[gnutls-help] DLL unload issues on Windows, MSVC2015 ReleaseDLL builds only

[Nikos Mavrogiannopoulos]

On Thu, Apr 6, 2017 at 3:35 PM, Lavrentiev, Anton (NIH/NLM/NCBI) [C]
<lavr at ncbi.nlm.nih.gov> wrote:
>> That's quite interesting case. Is that particular issue happening
>> because of the way you are using gnutls, or would this happen to most
>> users in windows?
>
> All code that uses GNUTLS in our builds made by MSVC2015/ReleaseDLL/x64 exhibits this problem of crashing at the infamous address 0x74 at process termination...
> Prior to using GNUTLS we set custom lock callbacks with gnutls_global_set_mutex()  (it's releasing all those locks that leads to calling free() with the app heap already gone).  Then we call gnutls_global_init() when begin using GNUTLS, and gnutls_global_deinit() when done with it.  We weren't aware of the (pretty big) change in behavior with auto-init from version 2 to 3 (somehow it escaped my attention).  As for the documentation,
>
> http://www.gnutls.org/manual/html_node/Initialization.html#Initialization
>
> says, "The resources allocated by the initialization process will be released on library deinitialization, or explicitly by calling gnutls_global_deinit."
> Well, that's not 100% accurate with auto-init (and what's exactly we were dealing with):  if user's code called gnutls_global_init() (which is no-operation with auto-init), so the user's gnutls_global_deinit() is no-operation just as well.  So either gnutls_global_init() must _not_ be called at all, or gnutls_global_deinit() must be called plus-one the number of times of explicit global_init's, to make the actual cleanup happen.

Thanks. I've committed a fix addressing that at documentation, by
removing the "or explicitly ... " part.
https://gitlab.com/gnutls/gnutls/commit/89d4aeb37f7028a033334361eb14f2bef095dc75

> I wasn't aware of the option to disable auto-init...  But it's rather cumbersome to use: the environment must be defined prior to the process start.  So any program that wants to avoid the auto-init must set it somehow.  If the binary is shipped out, there must be instructions, launch script or something to take care of the environment...  Simply put, it won't work easily.
>
> Lastly, gnutls_global_set_mutex should be documented of having a side effect of doing the global_deinit() / global_init() sequence internally -- this is important for counting the number of init / deinit pairs.
>
>> Do you have a suggestion on what can be improved to avoid these crashes?
> It seems that there's no one-fits-all solution here.  While auto-init is a great feature in general, you can make it a soft-init; so any explicit global_init() would override it and make the count of initializations restarted.  So last explicit global_deinit() will do the actual cleanup.  global_deinit() should be able to do the cleanup from the soft state as well (when there was no explicit global_init() issued from the user code, but only global_deinit()).

That unfortunately is too late to be added without causing problems to
existing programs. We can improve documentation though.

regards,
Nikos