<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html lang="en">
<head>
<meta content="text/html; charset=US-ASCII" http-equiv="Content-Type">
<title>
GitLab
</title>
<style>img {
max-width: 100%; height: auto;
}
</style>
</head>
<body>
<div class="content">
<p style="color: #777777;">
<a href="https://gitlab.com/civodul">civodul</a>
commented:
</p>
<div style="">
<p dir="auto">My reading of <a href="https://github.com/multipath-tcp/mptcp/commit/cf60af03ca4e71134206809ea892e49b92a88896" rel="nofollow noreferrer noopener" target="_blank">https://github.com/multipath-tcp/mptcp/commit/cf60af03ca4e71134206809ea892e49b92a88896</a> is that <code>MSG_FASTOPEN</code> must only be used on the first <code>sendmsg</code> call for a given host. Passing it on subsequent calls leads to <code>EISCONN</code>, just a <code>connect</code> call on an already-connected socket. This is confirmed by <a href="https://gitlab.com/gnutls/gnutls/uploads/e94f2265ed9b94c7599447270617b9dc/tcp-fastopen.c" data-link="true" class="gfm">this reproducer</a> (tested with Linux-libre 5.8.7).</p>
<p dir="auto">So the bug would be that GnuTLS passes <code>MSG_FASTOPEN</code> twice. Now, why does it happen?</p>
<p dir="auto">Outside Guix' isolated build environment, we get successful runs with this trace:</p>
<pre class="code highlight js-syntax-highlight plaintext" lang="plaintext" v-pre="true"><code><span id="LC1" class="line" lang="plaintext">socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 3</span>
<span id="LC2" class="line" lang="plaintext">getrandom("\xf1\xa7\x5d\xb1\x18\xb8\x4a\xd2\x77\x39\x36\x79\xba\x3e\xb6\x6f\x90\xcd\xdb\x0f\x35\x18\x03\x74\x92\xeb\xf0</span>
<span id="LC3" class="line" lang="plaintext">\xa3\xf8\x14\x38\xb0"..., 64, 0) = 64</span>
<span id="LC4" class="line" lang="plaintext">setsockopt(3, SOL_TCP, TCP_FASTOPEN, [1], 4) = 0</span>
<span id="LC5" class="line" lang="plaintext">sendmsg(3, {msg_name={sa_family=AF_INET, sin_port=htons(2137), sin_addr=inet_addr("127.0.0.1")}, msg_namelen=16, msg_iov=[{iov_base="\26\3\1\1b\1\0\1^\3\3\37\257as\245\1\33n\2202\240k\261|\275\177a\206-\274\320"..., iov_len=359}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, MSG_FASTOPEN) = 359</span>
<span id="LC6" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 40000) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC7" class="line" lang="plaintext">recvfrom(3, "\26\3\3\0{", 5, 0, NULL, NULL) = 5</span>
<span id="LC8" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 40000) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC9" class="line" lang="plaintext">recvfrom(3, "\2\0\0w\3\3m2>x\340\25(\341[m\233ji;uXyidgd\367?s\233\252"..., 123, 0, NULL, NULL) = 123</span>
<span id="LC10" class="line" lang="plaintext">sendmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\24\3\3\0\1\1", iov_len=6}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 6</span>
<span id="LC11" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 39997) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC12" class="line" lang="plaintext">recvfrom(3, "\24\3\3\0\1", 5, 0, NULL, NULL) = 5</span>
<span id="LC13" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 39997) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC14" class="line" lang="plaintext">recvfrom(3, "\1", 1, 0, NULL, NULL) = 1</span>
<span id="LC15" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 39997) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC16" class="line" lang="plaintext">recvfrom(3, "\27\3\3\0\35", 5, 0, NULL, NULL) = 5</span>
<span id="LC17" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 39997) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC18" class="line" lang="plaintext">recvfrom(3, "E\202UmH\20\37>Z\244R\214#.u\222<\260\303\210\350\260y\210\33\323YV\374", 29, 0, NULL, NULL) = 29</span>
<span id="LC19" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 39988) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC20" class="line" lang="plaintext">recvfrom(3, "\27\3\3\0B", 5, 0, NULL, NULL) = 5</span>
<span id="LC21" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 39988) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC22" class="line" lang="plaintext">recvfrom(3, "\221JO\26\256\341\303\274)Xa\277\1\253\223c\355[\267\341\5\22\2158\27\200}q\244\372\21\315"..., 66, 0, NULL, NULL) = 66</span>
<span id="LC23" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 39988) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC24" class="line" lang="plaintext">recvfrom(3, "\27\3\3\3\326", 5, 0, NULL, NULL) = 5</span>
<span id="LC25" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 39988) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC26" class="line" lang="plaintext">recvfrom(3, "\262\242~C\0242\244\316\263r\32\ny\23bb\220\226\216\277\276\323QSp\24\vU\336\246\241\22"..., 982, 0, NULL, NULL) = 982</span>
<span id="LC27" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 39987) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC28" class="line" lang="plaintext">recvfrom(3, "\27\3\3\1I", 5, 0, NULL, NULL) = 5</span>
<span id="LC29" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 39987) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC30" class="line" lang="plaintext">recvfrom(3, "\257!\233]m\260\276\342\236\266\304\270=Cb\223\317\364L\3030#j\372U\315bk\35\\\364\302"..., 329, 0, NULL, NULL) = 329</span>
<span id="LC31" class="line" lang="plaintext">openat(AT_FDCWD, "/gnu/store/cb6fakglpk69j8mz0g13rggp99l8v3x8-glibc-2.32/share/zoneinfo/UTC", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)</span>
<span id="LC32" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 39986) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC33" class="line" lang="plaintext">recvfrom(3, "\27\3\3\0E", 5, 0, NULL, NULL) = 5</span>
<span id="LC34" class="line" lang="plaintext">poll([{fd=3, events=POLLIN}], 1, 39986) = 1 ([{fd=3, revents=POLLIN}])</span>
<span id="LC35" class="line" lang="plaintext">recvfrom(3, "\177,\342&\26\212&r|\310Ipm\371\213\21o\243}2\21hI\340\367\314\6+\374a\350\203"..., 69, 0, NULL, NULL) = 69</span>
<span id="LC36" class="line" lang="plaintext">sendmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\27\3\3\0\31U\322\7\371/\341k<z\320\246I\207\250d\3345\242j_\226\221\r\215\373", iov_len=30}, {iov_base="\27\3\3\0E\253\3\215*\241\362\374\243F\312{$\373\315{\23\3\300\3340\262\200\313,D\2622"..., iov_len=74}], msg_iovlen=2, msg_controllen=0, msg_flags=0}, 0) = 104</span></code></pre>
<p dir="auto">The key difference is that there is <em>no</em> second <code>setsockopt</code> call and the second <code>sendmsg</code> call doesn't have <code>MSG_FASTOPEN</code>.</p>
<p dir="auto">The faulty case (i.e., in Guix' build environment) is as if <code>p->connect_addrlen = 0</code> in <code>tfo_writev</code> wasn't executed, leading to the second <code>setsockopt</code> and <code>MSG_FASTOPEN</code>, which in turn leads to <code>EISCONN</code>.</p>
<p dir="auto">Thoughts?</p>
</div>
</div>
<div class="footer" style="margin-top: 10px;">
<p style="font-size: small; color: #666;">
—
<br>
Reply to this email directly or <a href="https://gitlab.com/gnutls/gnutls/-/issues/1095#note_418757425">view it on GitLab</a>.
<br>
You're receiving this email because of your account on gitlab.com.
If you'd like to receive fewer emails, you can
<a href="https://gitlab.com/-/sent_notifications/26d3ccd5843177819859d2d94d302c77/unsubscribe">unsubscribe</a>
from this thread or
adjust your notification settings.
<script type="application/ld+json">{"@context":"http://schema.org","@type":"EmailMessage","action":{"@type":"ViewAction","name":"View Issue","url":"https://gitlab.com/gnutls/gnutls/-/issues/1095#note_418757425"}}</script>
</p>
</div>
</body>
</html>