[gnutls-devel] Possible dereferencing error on pthread mutexes in 3.6.0

[Svein Olav Bjerkeset]

Hi,

During an attempt to build libmicrohttpd on RHEL5, I tried a combination 
of libmicrohttpd-0.9.55 and GnuTLS 3.6.0. It consistantly core dumped 
with a segfault when trying to increment at mutex value 
(gnutls_atomic_uint_st) during https connections. The increment occurs 
at lib/priority.c:563:

gnutls_atomic_increment(&priority->usage_cnt);

Running the test https server through gdb, I find that the reason for 
the crash, is that pthread_mutex_lock is eventually called with a null 
pointer argument from gnutls_system_mutex_lock.

Debuging seems to show that the mutex initialization works as intended, 
but not lock and unlock.

Here is the backtrace:

(gdb) bt
  #0  0x00000031c7a08dd0 in pthread_mutex_lock () from 
/lib64/libpthread.so.0
  #1  0x00002b9de625d2fc in gnutls_system_mutex_lock (priv=<value 
optimized out>) at system/threads.c:119
  #2  0x00002b9de624bd13 in gnutls_atomic_increment (session=0x13d64e20, 
priority=0x13d5e440)
     at atomic.h:58
  #3  gnutls_priority_set (session=0x13d64e20, priority=0x13d5e440) at 
priority.c:563
  #4  0x00002b9de5ffd5a0 in internal_add_connection (daemon=0x13d64bd8, 
client_socket=6,
      addr=0x2b9de88ff050, addrlen=16, external_add=false, non_blck=true) 
at daemon.c:2321
  #5  0x00002b9de5ffdd41 in MHD_accept_connection (daemon=0x13d64bd8) at 
daemon.c:2953
  #6  0x00002b9de5ffe1d8 in MHD_poll_all (daemon=0x13d64bd8, may_block=1) 
at daemon.c:3681
  #7  MHD_poll (daemon=0x13d64bd8, may_block=1) at daemon.c:3786
  #8  0x00002b9de5ffefae in MHD_select_thread (cls=<value optimized out>) 
at daemon.c:4372
  #9  0x00000031c7a0683d in start_thread () from /lib64/libpthread.so.0
#10 0x00000031c6ed4fdd in clone () from /lib64/libc.so.6

Unfortunately, I do not know the GnuTLS code, so I am not able to say 
where the error may be. I ended up using an older version of GnuTLS 
myself, but just wanted to report on my findings here.

Best regards,
Svein Olav Bjerkeset