pinentry-gnome3: (libgcrypt): Cannot allocate memory

[Andreas Stieger]

Hello,

On 05/06/15 00:06, Andreas Stieger wrote:
> As I was packaging pinentry 0.9.3 with the new /usr/bin/pinentry-gnome3
> I noticed the following:
> 
> $ /usr/bin/pinentry-gnome3
> OK Your orders please
> getpin
> couldn't lock 16384 bytes of memory (libgcrypt): Cannot allocate memory
> ERR 83886179 canceled
> 
> strace:
> 
> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
> = 0x7efc75498000
> mmap(NULL, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x7efc75494000
> mlock(0x7efc75494000, 16384)            = -1 ENOMEM (Cannot allocate memory)
> write(2, "couldn't lock 16384 bytes of mem"..., 72couldn't lock 16384
> bytes of memory (libgcrypt): Cannot allocate memory
> 
> This is on a system with a fairly standard limit:
> max locked memory       (kbytes, -l) 64
> 
> Where would we start looking to debug and look for a possible solution?
> Do you think the gnome3 variant could receive a change to stay below a
> 64k lock limit?


tl;dnr: DEFAULT_POOLSIZE was set to 64k, leaving no more for mlock in
the -gnome3 UI with standard limits.

Debugged this... When running pinentry-gnome3, mlock called twice, first
via pinentry_init -> secmem_init

$ gdb pinentry-gnome3
done.
(gdb) catch syscall mlock
Catchpoint 1 (syscall 'mlock' [149])
(gdb) run
Starting program: /usr/bin/pinentry-gnome3
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Catchpoint 1 (call to syscall mlock), 0x00007ffff63a9307 in mlock ()
from /lib64/libc.so.6
(gdb) bt
#0  0x00007ffff63a9307 in mlock () at /lib64/libc.so.6
#1  0x0000000000409d14 in secmem_init (n=65536, p=0x7ffff7fe7000) at
secmem.c:167
#2  0x0000000000409d14 in secmem_init (n=65536) at secmem.c:246
#3  0x0000000000409d14 in secmem_init (n=65536, n at entry=1) at secmem.c:306
#4  0x00000000004052cf in pinentry_init (pgmname=pgmname at entry=0x40c2e7
"pinentry-gnome3") at pinentry.c:512
#5  0x0000000000403196 in main (argc=1, argv=0x7fffffffdce8) at
pinentry-gnome3.c:254
(gdb)
(gdb) c
Continuing.

Catchpoint 1 (returned from syscall mlock), 0x00007ffff63a9307 in mlock
() from /lib64/libc.so.6
(gdb) c
Continuing.
[New Thread 0x7fffec027700 (LWP 8640)]
OK Your orders please
getpin
[New Thread 0x7fffeb826700 (LWP 8641)]

And then again with via the GUI in -gnome3.


Catchpoint 1 (call to syscall mlock), 0x00007ffff63a9307 in mlock ()
from /lib64/libc.so.6
(gdb) bt
#0  0x00007ffff63a9307 in mlock () at /lib64/libc.so.6
#1  0x00007ffff6b41865 in  () at /usr/lib64/libgcr-base-3.so.1
#2  0x00007ffff5ff0f29 in  () at /usr/lib64/libgcrypt.so.20
#3  0x00007ffff5ff22c3 in  () at /usr/lib64/libgcrypt.so.20
#4  0x00007ffff608c71b in  () at /usr/lib64/libgcrypt.so.20
#5  0x00007ffff608c8d7 in  () at /usr/lib64/libgcrypt.so.20
#6  0x00007ffff6b3c0ca in  () at /usr/lib64/libgcr-base-3.so.1
#7  0x00007ffff6b148b6 in  () at /usr/lib64/libgcr-base-3.so.1
#8  0x00007ffff6b151f8 in gcr_secret_exchange_receive () at
/usr/lib64/libgcr-base-3.so.1
#9  0x00007ffff6b186d9 in  () at /usr/lib64/libgcr-base-3.so.1
#10 0x00007ffff5d3a7e1 in  () at /usr/lib64/libgio-2.0.so.0
#11 0x00007ffff76768e5 in g_main_context_dispatch () at
/usr/lib64/libglib-2.0.so.0
#12 0x00007ffff7676c48 in  () at /usr/lib64/libglib-2.0.so.0
#13 0x00007ffff7676f0a in g_main_loop_run () at /usr/lib64/libglib-2.0.so.0
#14 0x00007ffff6b17315 in  () at /usr/lib64/libgcr-base-3.so.1
#15 0x00007ffff5ccd2ba in g_initable_new_valist () at
/usr/lib64/libgio-2.0.so.0
#16 0x00007ffff5ccd39c in g_initable_new () at /usr/lib64/libgio-2.0.so.0
#17 0x00007ffff6b191a1 in gcr_system_prompt_open_for_prompter () at
/usr/lib64/libgcr-base-3.so.1
#18 0x00000000004033b2 in create_prompt (pe=pe at entry=0x611ae0
<pinentry>, confirm=confirm at entry=0)
    at pinentry-gnome3.c:80
#19 0x00000000004035f8 in gnome3_cmd_handler (pe=0x611ae0 <pinentry>) at
pinentry-gnome3.c:176
#20 0x0000000000404f78 in cmd_getpin (ctx=0x7ffff7fe7008,
line=<optimized out>) at pinentry.c:1168
#21 0x0000000000408eea in process_request (linelen=<optimized out>,
line=0x7ffff7fe7056 "", ctx=0x7ffff7fe7008) at assuan-handler.c:435
#22 0x0000000000408eea in process_request (ctx=0x7ffff7fe7008) at
assuan-handler.c:458
#23 0x0000000000409470 in assuan_process (ctx=0x7ffff7fe7008) at
assuan-handler.c:526
#24 0x000000000040579d in pinentry_loop2 (infd=infd at entry=0,
outfd=outfd at entry=1) at pinentry.c:1426
#25 0x000000000040585c in pinentry_loop () at pinentry.c:1446
#26 0x00000000004031c4 in main (argc=1, argv=0x7fffffffdce8) at
pinentry-gnome3.c:267
(gdb)

Of course an initial mlock call of 64k will exhaust the lock limit,
leaving none for the second. The 64k was due to a local change of
DEFAULT_POOLSIZE to address a problem that has since vanished. Local
problem.

Andreas