libgcrypt performance issue, diagnostic & workaround

Christian Grothoff grothoff at
Tue Aug 14 10:51:19 CEST 2018


Running the GNU Taler exchange (which uses libgcrypt) on a many-core
(32-thread) system I noticed that futex syscall was dominating CPU
utilization.  Digging deeper into it, the reason is the
SECMEM_LOCK/SECMEM_UNLOCK logic, despite the fact that I (properly)
disable the use of SECMEM via gcry_control on startup

 * Initialize libgcrypt.
void __attribute__ ((constructor))
GNUNET_CRYPTO_random_init ()
  gcry_error_t rc;

  if (! gcry_check_version (NEED_LIBGCRYPT_VERSION))
    FPRINTF (stderr,
             _("libgcrypt has not the expected version (version %s is
    GNUNET_assert (0);
  /* Disable use of secure memory */
  if ((rc = gcry_control (GCRYCTL_DISABLE_SECMEM, 0)))
    FPRINTF (stderr,
             "Failed to set libgcrypt option %s: %s\n",
	     gcry_strerror (rc));
  /* Otherwise gnunet-ecc takes forever to complete, besides
     we are fine with "just" using GCRY_STRONG_RANDOM */
  if ((rc = gcry_control (GCRYCTL_ENABLE_QUICK_RANDOM, 0)))
    FPRINTF (stderr,
	     "Failed to set libgcrypt option %s: %s\n",
	     gcry_strerror (rc));

Sample stack trace:

#2  0x00007fbab01077ea in _gcry_secmem_free (a=0x7fba4000faf0) at
#3  0x00007fbab01068f6 in _gcry_private_free (a=0x7fba4000faf0) at
#4  0x00007fbab0100c8d in _gcry_free (p=0x7fba4000faf0) at global.c:1033
#5  0x00007fbab01018e9 in _gcry_sexp_release (sexp=0x7fba4000faf0) at
#6  0x00007fbab01025e1 in _gcry_sexp_cadr (list=0x7fba40005dc0) at
#7  0x00007fbab00fbc6a in gcry_sexp_cadr (list=0x7fba40005dc0) at
#8  0x00007fbab0052a1e in key_from_sexp (array=0x7fba4fffe4b0,
sexp=0x7fba4001a5f0, topname=0x7fbab009244d "public-key",
elems=0x7fbab00924b6 "n") at crypto_rsa.c:103
#9  0x00007fbab00534d0 in GNUNET_CRYPTO_rsa_public_key_decode (
    buf=0x7fba40003630 "(public-key \n (rsa \n  (n
6758C9739EA5B94C51"..., len=310) at crypto_rsa.c:380
#10 0x00007fbab0017129 in parse_rsa_public_key (cls=0x0,
root=0x7fba4001dd60, spec=0x7fba4fffe680) at json_helper.c:799
#11 0x00007fbab00144f6 in GNUNET_JSON_parse (root=0x7fba4001cb40,
spec=0x7fba4fffe610, error_json_name=0x7fba4fffe5c0,
error_line=0x7fba4fffe5bc) at json.c:62
#12 0x000055680f9af868 in TEH_PARSE_json_data
(connection=0x7fba4001e370, root=0x7fba4001cb40, spec=0x7fba4fffe610) at
#13 0x000055680f9a96ea in TEH_DEPOSIT_handler_deposit (rh=0x55680f9c7988
<handlers+616>, connection=0x7fba4001e370,
connection_cls=0x7fba4001e3c8, upload_data=0x0,

Note that 'mainpool' clearly was never setup:

(gdb) print mainpool
$4 = {next = 0x0, mem = 0x0, size = 0, okay = 0, is_mmapped = 0,
cur_alloced = 0, cur_blocks = 0}
(gdb) print no_secure_memory
$5 = 1

My understanding is that the bug is that stdmem.c::_gcry_private_free()
first tries to call _gcry_secmem_free() before calling free(), thus
uselessly grabbing the lock.  Here, a test whether secmem is disabled
instead of grabbing the lock ought to be inserted.

As a workaround, I've added this call to our libgcrypt initialization

  /* set custom allocators */
  gcry_set_allocation_handler (&w_malloc, // wrapper around calloc()
                               &w_malloc, // wrapper around calloc()
                               &w_check,  // return 'false'

After that, the GNU Taler exchange could do > 10x the number of
operations per second on my system (might be more, but the bottleneck is
now elsehwere).

Happy hacking!


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <>

More information about the Gcrypt-devel mailing list