SSL handshake fails between libcurl and libgnutls/MHD

[Christian Grothoff]

Dear all,

After a recent update of libcurl / libgnutls on my Debian unstable 
system, the fully automated tests of GNU libmicrohttpd for HTTPS started 
to fail.  These tests start an HTTPS server using libgnutls and GNU 
libmicrohttpd and then try downloading a site using libcurl.

Here is the key output:
$ cd libmicrohttpd/src/testcurl/https/; make check
curl version: libcurl/7.23.1 GnuTLS/2.12.14 zlib/1.2.3.4 libidn/1.23 
librtmp/2.3
# ...
curl_easy_perform failed: `SSL connect error'
Error: received handshake message out of context
Error (code: 4294967295)
FAIL: mhds_session_info_test

(this is not the only test that suddenly started to fail).

One of our tests also provokes a failure by selecting incompatible 
versions of the SSL protocol.  With older versions, that test produces ONCE:

curl version: libcurl/7.21.3 GnuTLS/2.8.6 zlib/1.2.3.4 libidn/1.18
curl_easy_perform failed: `SSL connect error'
Error: received handshake message out of context

With the latest version, the two lines are repeated several times (and 
the test now fails).


My guess right now is that there must have been some incompatible (!) 
protocol change in gnutls with itself (!?) or a significant change in 
how libcurl uses gnutls (i.e. change of supported ciphers, certificate 
checking, etc.).

I've not yet had the time to investigate which revision exactly 
introduced the problem; however, I've seen it on several systems now, so 
it is pretty real.  I suspect this is an unintended bug; however, if 
there was a change in how one should use the curl or gnutls APIs, I'd 
really appreciate some hints :-).

I'm collecting information about the bug in our bugtracker at
https://gnunet.org/bugs/view.php?id=2086

Help would be very welcome.


Happy hacking!

Christian