[gnutls-help] 3.6.7 interoperability problems with earlier 3.6.x

[Andreas Metzler]

Hello,

this is https://bugs.debian.org/929907 submitted and boiled down to a
minimal (gnutls-cli) testcase by Dominik George. (The original issue was
openldap breaking.)

gnutls-serv from 3.6.4 3.6.5 and 3.6.6 does not work correctly with
gnutls-cli from 3.6.7.
------
gnutls-serv --echo --x509keyfile /etc/ssl/private/ssl-cert-snakeoil.key --x509certfile /etc/ssl/certs/ssl-cert-snakeoil.pem
...
pwgen 16383 | gnutls-cli --no-ca-verification --port 5556 localhost
> From a size of 16383 bytes onwards, I get:

> |<1>| Received packet with illegal length: 16385
> |<1>| Discarded message[1] due to invalid decryption
> *** Fatal error: A TLS record packet with invalid length was received.
> *** Server has terminated the connection abnormally.
------

gnutls-cli 3.6.8 works. Disabling TLS 1.3 or %NO_EXTENSIONS also works
around this.

I have run git bisect to find more info. The issue was introduced in
3fc7d37dd81a1f415afffbf5f733c13296e74824
ext/record_size_limit: account for content type octet in TLS 1.3

and was fixed for 3.6.8 by
2dc96e3b8d0e043bebf0815edaaa945f66ac0531
ext/record_size_limit: distinguish sending and receiving limits

I am now wondering on what to do with this bug for the next Debian
stable release ("buster").
* We are unlikely to upgrade to 3.6.8 since buster is already frozen.
* Both reverting 3fc7d37dd81a1f415afffbf5f733c13296e74824 and applying
  2dc96e3b8d0e043bebf0815edaaa945f66ac0531 makes the issue
  unreproducible. I don't like the former because it reverts a TLS 1.3 
  bugfix. Straightforward applying of the latter would add
  gnutls_record_set_max_recv_size(). (So I'd problably rename to
  _gnutls_record_set_max_recv_size and patch to not export.)
* I am not sure about the impact, whether we are unlucky we don't have a
  failing testcase with openssl or whether it only shows up with Record
  Size Limit Extension. However Ubuntu Cosmic Cuttlefish and Disco Dingo
  ship 3.6.4 and 3.6.5 respectively, so we can't just ignore this, since
  I really want Debian and Ubuntu release to work together. ;-)
* Do the git bisect result make sense?

TIA, cu Andreas

-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'