Interoperability issues (Debian Bug #348046)

[Simon Josefsson]

Marc Haber <mh+gnutls-devel at zugschlus.de> writes:

> Hi,
>
> Simon Josefsson has suggested to me (a member of the maintainer team
> for Exim's packages for the Debian Operating System) that it might be
> a good idea to move a technical debate from our blogs
> (http://blog.zugschlus.de/archives/585-exim4-vs.-OpenSSL-vs.-GnuTLS.html,
> http://blog.josefsson.org/2007/11/09/response-to-gnutls-in-exim-debate/)
> to gnutls-devel as this list is a better medium for archived discussion.
>
> I'll send a dedicated mail for each of Debian's bug reports, so that
> the threads are not going to intermix.
>
> Debian Bug #348046, http://bugs.debian.org/348046
> =================================================
>
> Simon writes:
>> Bug #348046 is so complex that I cannot judge it. If the submitters
>> are willing, it may be best to re-submit each problem separately. The
>> problem with TheBat is interesting, but given the non-free status of
>> TheBat and no other reports, it doesn???t seem serious. To reduce
>> entropy consumption is something we should work on, but it is a
>> ???wishlist??? kind of bug, and to some extent may have already been
>> solved by removing the DH generation code which depleats the entropy
>> pool quickly. The rest appears to be already solved or should be
>> tagged as ???wontfix???.
>
> To me, after cursory inspection, this looks like:
> (A) Original bug submitter complaining about "A TLS packet with
>     unexpected length was received" when the local Exim is a _client_
>     (Message 5).
> (B) jarek saying "me too", but saying that his local Exim says
>     "A TLS packet with unexpected length was received" while his Exim
>     is a _server_ (Message 45)

Right.

> (C) Ian Zimmerman trying to debug issue (B) but having trouble with
>     gnutls-cli (Message 58)

Ian's initial problem (in 58) appear to be with 'openssl s_client'.  The
problem with gnutls-cli-debug in message 63 was a user error,
gnutls-cli-debug doesn't work with TLS over SMTP, but you figured that
out.  In message 98 gnutls appears to work fine, but openssl s_client
does not seem to work, which was Ian's initial concern.

The reason the openssl command doesn't work is explained by adding
'debug' to the command line:

yxa-iv:~# openssl s_client -connect kniv.josefsson.org:25 -starttls smtp -debug
CONNECTED(00000003)
read from 080B1BB0 [080AC740] (8192 bytes => 72 (0x48))
0000 - 32 32 30 20 6b 6e 69 76-2e 6a 6f 73 65 66 73 73   220 kniv.josefss
0010 - 6f 6e 2e 6f 72 67 20 45-53 4d 54 50 20 45 78 69   on.org ESMTP Exi
0020 - 6d 20 34 2e 36 33 20 46-72 69 2c 20 30 34 20 4a   m 4.63 Fri, 04 J
0030 - 61 6e 20 32 30 30 38 20-31 32 3a 33 39 3a 31 36   an 2008 12:39:16
0040 - 20 2b 30 31 30 30 0d 0a-                           +0100..
write to 080B1BB0 [BF816D80] (10 bytes => 10 (0xA))
0000 - 53 54 41 52 54 54 4c 53-0d 0a                     STARTTLS..
read from 080B1BB0 [080AA738] (8192 bytes => 47 (0x2F))
0000 - 35 30 33 20 53 54 41 52-54 54 4c 53 20 63 6f 6d   503 STARTTLS com
0010 - 6d 61 6e 64 20 75 73 65-64 20 77 68 65 6e 20 6e   mand used when n
0020 - 6f 74 20 61 64 76 65 72-74 69 73 65 64 0d 0a      ot advertised..
write to 080B1BB0 [080B1BF8] (142 bytes => 142 (0x8E))
0000 - 80 8c 01 03 01 00 63 00-00 00 20 00 00 39 00 00   ......c... ..9..
0010 - 38 00 00 35 00 00 16 00-00 13 00 00 0a 07 00 c0   8..5............
0020 - 00 00 33 00 00 32 00 00-2f 03 00 80 00 00 66 00   ..3..2../.....f.
0030 - 00 05 00 00 04 01 00 80-08 00 80 00 00 63 00 00   .............c..
0040 - 62 00 00 61 00 00 15 00-00 12 00 00 09 06 00 40   b..a...........@
0050 - 00 00 65 00 00 64 00 00-60 00 00 14 00 00 11 00   ..e..d..`.......
0060 - 00 08 00 00 06 04 00 80-00 00 03 02 00 80 84 0f   ................
0070 - d2 b8 0b 21 74 5f 8a 9d-d5 42 3e 74 a0 63 5d 05   ...!t_...B>t.c].
0080 - b7 4d e1 8e 79 c5 52 1b-de 71 39 b4 3e cd         .M..y.R..q9.>.
read from 080B1BB0 [080B7158] (7 bytes => 7 (0x7))
0000 - 35 30 31 20 4e 55 4c                              501 NUL
12040:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:478:
yxa-iv:~# 

In other words, openssl doesn't implement the TLS over SMTP protocol
properly, and exim4 rejects the connection.  This is clearly an openssl
bug and has nothing to do with gnutls.

> (D) gnutls-cli-debug not having an --starttls option (is this a bug in
>     gnutls-cli-debug? What is gnutls-cli-debug's Differnence from
>     gnutls-cli in the first place?)

Right, gnutls-cli-debug doesn't support starttls.  If someone wants to
work on providing that capability, feel free to submit patches...  I
note that openssl doesn't have any similar tool with starttls support
either.

The difference between gnutls-cli and gnutls-cli-debug is that the
former is a simple interactive TLS client (with some starttls support)
and the latter is a non-interactive debug tool.

> (E) Marc F .Clemente saying "me too" to (B) (Message 110)

It would help if he could run gnutls-serv in debug mode and try to
reproduce the problem with thunderbird.  I'll ask him in the bug report
to do this.

I'd add another item here:

(EE) Vincent Lefevre says (Message 120) that the first message each
     morning fails with this error message too.

One theory here could be some firewall acting up the first time every
morning, what do you think?  As Andreas Metzler says in message 125,
there is nothing in the entropy code that could explain this.  The error
message is also not entropy related.

> (F) Vincent Lefevre saying (Message 130), that outgoign messages also
>     reduce entropy.

Which may be true.

I'd add another item here too:

(FF) Ronny Adsetts reports a problem with a different error message 'A
  record packet with illegal version was received.' and
  '(gnutls_handshake): timed out'.

To me this looks like a connection problem.  The first error typically
happens on data corruption (possibly caused by incorrect STARTTLS
negotiation) or implementation problems.

> (G) Andrew McGlashan finding it impossible to connect to gnutls-serv
>     with incredimail (giving debug output in Message 224).
>
> Can you comment about (D) and (G)?

First, Andrews' configuration seems confusing.  He's using
tls_on_connect_ports on ports 587?  No wonder OE doesn't work.  I don't
really understand what's not working and what he wants to do.  But I'll
try to answer only your question:

In the debug log, it seems IM is sending a SSLv2 client hello's.  GnuTLS
doesn't support SSLv2; for good reasons, its security has been broken.
We do support SSLv2 client hellos though, so it should work if the
client also supports SSLv3.  This may be an area where people have not
done enough testing, SSLv2 is almost extinct.  So a basic question: Does
IM support upwards negotiation from SSLv2 to SSLv3?  If not, then this
will not ever work.  The user might as well use unencrypted connections
instead.

Without more help from the IM community, I'd be inclined to sign this is
off as a IM problem.

>> Bug #348046
>>  This is a long bug report by several submitters. The initial report
>>  from Martin A. Brooks is stalled when he doesn???t respond to the
>>  (appropriate and relevant) questions that Marc Haber asks. The
>>  problem that Ian Zimmerman reports seems to be different, now GnuTLS
>>  clients work fine but OpenSSL clients fail to connect to the
>>  Exim4/GnuTLS server. The OpenSSL errors may suggest it only wants to
>>  talk SSLv2 for some reason (local configuration?). Debugging the
>>  OpenSSL problem further would be the appropriate response, and should
>>  likely be treated as an OpenSSL bug (!) until more evidence can be
>>  gathered. Later, Andrew McGlashan reports a problem where neither
>>  GnuTLS and OpenSSL works against the ???Incredimail??? MUA.
>>  Conclusion: The bug should really be forked into several problems,
>>  one for the initial reports where the submitter stopped responding,
>>  one for the OpenSSL problem, and one for the Incredimail problem (and
>>  as Incredimail isn???t a Debian package, it???s not Debian???s problem).
>>  Caveat: I may have missunderstood some parts of this bug report
>>  because different problems are discussed at the same time. Once that
>>  is done we can try to address each problem separetely.
>
> Which issue is the OpenSSL thing?

The 'openssl s_client' issue I discussed above, and by using the -debug
parameter I even resolved it... it is an openssl bug.

> I think this is a good case to show what happens when the error
> messages are too simple. This bug report is a mess of different issues
> since GnuTLS obviously returns the same, quite generic, error message
> text for a number of different issues. People look into the BTS for
> their error message and attach their issue to the existing bug report,
> leading to the horrible mess this bug report is. I'd like to use this
> opportunity to pledge for more distinctive error messages.

Before we know exactly what is the cause for the various problem, we
can't know what a better error message would be.  Reporting very narrow
error messages is known to lead to security problems, where the other
side can use different behaviour based on different error messages to
attack the server.  So some caution to be very verbose in error message
is warranted for security applications.

I'm not sure if this message will help much to move things further.
There are simply too many completely different problems in this bug
report, and the original submitter stopped responding long time ago.
But I tried to answer the questions you raised at least.

Thanks,
/Simon