Handshake Packet Dropped

Peter Hendrickson pdh at wiredyne.com
Fri Jul 24 20:51:16 CEST 2009

A server initiates a session renegotiation by calling
gnutls_rehandshake().  If the connection is active, the client will
send application data followed by a ClientHello.  (The application
data may well have been sent before the Client received the
HelloRequest packet.)

According to the gnutls_handshake() documentation:
> If this function is called by a server after a rehandshake request
> `GNUTLS_E_WARNING_ALERT_RECEIVED' may be returned.  Note that these
> are non fatal errors, only in the specific case of a rehandshake.
> Their meaning is that the client rejected the rehandshake request.

So, apparently the server needs to clear out any incoming application
data before calling gnutls_handshake().  If it doesn't,
gnutls_handshake() just returns GNUTLS_E_GOT_APPLICATION_DATA.  And
that is apparently indistinguishable from the client rejecting the
renegotiation request.

The server wants to read all of the application data out before the
ClientHello packet appears.  And this is where I believe I have
found a bug.

If the server calls gnutls_record_recv(), it needs to do so repeatedly
until it runs out of application data.  This means that when the
ClientHello packet comes in, that it will need to be stored someplace
so that gnutls_handshake() will be able to use it shortly thereafter.

What actually happens is that gnutls_record_recv() calls
_gnutls_recv_int() and tells it to expect application data.  When the
ClientHello comes in, it is decrypted and stored in a gnutls_datum_t
called "tmp".  The sequence number is incremented and then
record_check_type() is called.  This routine notices that the data
received is a handshake and not application data and the result is
that _gnutls_recv_int() returns GNUTLS_E_REHANDSHAKE.

However, the ClientHello packet does not seem to get stored anywhere.
When gnutls_handshake() is subsequently called it just waits and waits
for a packet it will never see because the client already sent it --
it just got dropped on the floor.

(The memory in _gnutls_recv_int():tmp does not seem to get freed in
this case, so there may also be a little memory leak.)

This is observed in GnuTLS 2.9.2.  I doubt it matters, but I am using
nonblocking sockets.


More information about the Gnutls-devel mailing list