SSL handshake fails between libcurl and libgnutls/MHD

[Simon Josefsson]

Christian Grothoff <grothoff at in.tum.de> writes:

> Dear all,
>
> After a recent update of libcurl / libgnutls on my Debian unstable
> system, the fully automated tests of GNU libmicrohttpd for HTTPS
> started to fail.  These tests start an HTTPS server using libgnutls
> and GNU libmicrohttpd and then try downloading a site using libcurl.
>
> Here is the key output:
> $ cd libmicrohttpd/src/testcurl/https/; make check
> curl version: libcurl/7.23.1 GnuTLS/2.12.14 zlib/1.2.3.4 libidn/1.23
> librtmp/2.3
> # ...
> curl_easy_perform failed: `SSL connect error'
> Error: received handshake message out of context
> Error (code: 4294967295)
> FAIL: mhds_session_info_test
>
> (this is not the only test that suddenly started to fail).
>
> One of our tests also provokes a failure by selecting incompatible
> versions of the SSL protocol.  With older versions, that test produces
> ONCE:
>
> curl version: libcurl/7.21.3 GnuTLS/2.8.6 zlib/1.2.3.4 libidn/1.18
> curl_easy_perform failed: `SSL connect error'
> Error: received handshake message out of context
>
> With the latest version, the two lines are repeated several times (and
> the test now fails).
>
>
> My guess right now is that there must have been some incompatible (!)
> protocol change in gnutls with itself (!?) or a significant change in
> how libcurl uses gnutls (i.e. change of supported ciphers, certificate
> checking, etc.).
>
> I've not yet had the time to investigate which revision exactly
> introduced the problem; however, I've seen it on several systems now,
> so it is pretty real.  I suspect this is an unintended bug; however,
> if there was a change in how one should use the curl or gnutls APIs,
> I'd really appreciate some hints :-).
>
> I'm collecting information about the bug in our bugtracker at
> https://gnunet.org/bugs/view.php?id=2086
>
> Help would be very welcome.

I don't recognize any GnuTLS errors above, so before I can help I need
some backtrace or debug info pointing towards where a GnuTLS function
returns an error now but didn't before.  The 'SSL connect error' seems
pretty fundamental, so chances are that it is something simple.

/Simon