Handshake fails with Internal error in memory allocation

[Nikos Mavrogiannopoulos]

Simon Josefsson wrote:
> This error has come up lately, see:
> 
> http://bugs.debian.org/466477
> http://bugs.debian.org/478191
> 
> The cause seems clear, the server sends a huge list of CA certs and
> GnuTLS runs into some fixed size buffer or something.  This reproduces
> it:
> 
> gnutls-cli -p 25 -s mail3.mclemente.net
> ehlo foo
> starttls
> ^D
> 
> Nikos, do you have any idea?  I could look at it, but have little time
> right now.

There is this function that can be used to increase the default
handshake packet size. The current is 16kb and is used to protect from
denial of service.

/**
  * gnutls_handshake_set_max_packet_length - set the maximum length of a
handshake message
  * @session: is a #gnutls_session_t structure.
  * @max: is the maximum number.
  *
  * This function will set the maximum size of a handshake message.
  * Handshake messages over this size are rejected.  The default value
  * is 16kb which is large enough. Set this to 0 if you do not want to
  * set an upper limit.
  *
  **/