[gnutls-devel] GnuTLS | nettle's gnutls_crypto_init() causes segfault in unrelated gmp code during static destructors (#1398)

[Read-only notification of GnuTLS library development activities]

Luke D'Alessandro created an issue: https://gitlab.com/gnutls/gnutls/-/issues/1398



## Description of problem:

The GMP library indirects dynamic memory allocation through a couple of function pointers, and allows client code to get and set those values through calls to `mp_set_memory_functions()`. 41c9c845 introduced code into `gnutls_crypto_init` that leverages this functionality to replace GMP's realloc and free functions to use `gnutls_realloc_zero` and `gnutls_free_zero`.

I have some static global `mpq_class` objects (using GMP's C++ API) that have their destructors run in `__run_exit_handlers`, after `main()`. These destructors attempt to use GMP's `__gmp_free_func` to free their data, which was set to `gnutls_free_zero`, however due to the vagaries of shared-object ctor/dtor ordering, libgnutls.so has **already been unloaded** and thus that call segfaults.

I do not personally load libgnutls, that is done transitively by OpenMPI in this case.

As an end user, I don't really know of a great way to workaround this. I can manually reset GMP's pointers to their default after gnutls resets them now that I know this is a problem. I think the fix might be to have gnutls remember these values and reset them during `deinit()`, but that assumes everyone is correctly calling that function.

## Version of gnutls used:

## Distributor of gnutls (e.g., Ubuntu, Fedora, RHEL)

Various means, this occurs on all platforms that I run on.

## How reproducible:

Forgive me, I don't know how to actually call any functions in gnutls, but the following code reproduces the problem when compiled with mpic++.

```C++
#include <gmpxx.h>
#include <mpi.h>

static mpq_class _;

int main()
{
    MPI_Init(NULL, NULL);
    MPI_Finalize();
}
```

## Actual results:

```
ldalessa at portland:~/temp$ mpic++ -o test test.cpp -lgmpxx -lgmp
ldalessa at portland:~/temp$ ./test
[portland:74170] *** Process received signal ***
[portland:74170] Signal: Segmentation fault (11)
[portland:74170] Signal code: Address not mapped (1)
[portland:74170] Failing at address: 0x7fab9d15d0e0
Segmentation fault
```

## Expected results:

No SEGFAULT.

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/gnutls/-/issues/1398
You're receiving this email because of your account on gitlab.com.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20220820/1ccbd065/attachment-0001.html>