'pubring.kbx' growing

[Bernhard Voelker]

Hi *,

when moving our GPGME-based server application from SLES12 (gpg 2.0.24) to SLES15 (gpg 2.2.27),
I noticed that the performance got 3-10 times slower within one day.
I nailed it down with strace that all commands have to do a lot of read(2)s on the 'pubring.kbx' file.

Now the special thing about my application is probably that it always ensures that only
exactly 1 key is in the keyring (either private or public).
It does this via gpgme_op_import(), then doing some action like sign/verify/encrypt/decrypt
followed by deleting the key again from the keyring via gpgme_op_delete_ext().

This is to ensure that e.g. a signature verified was really created by a certain key.
I'm aware that this procedure might not be necessary anymore, and the keys could stay in the
keyring files nowadays.
At the time I wrote that application (back in 2013), the GPGME result structures did not
turn out to be that reliable in that regard yet.

As a rough workaround for the immediate issue, I have now added an unlink() on the 'keyring.kbx'
file in my application.

Still, I find it worth reporting that the 'pubring.kbx' file is growing with many import/delete-key
actions.  Here's a reproducer based on plain GPG (also tested with newer GPG-2.4.5 on Tumbleweed)
which shows the growing file size and also demonstrates that a 'gpg --verify' becomes slower.
It uses a public key file, its fingerprint and a signature file - see the top of the script.

--->8--->8--->8--->8--->8--->8--->8--->8--->8---
#!/bin/sh

# Some public key file and its fingerprint.
KEY='/tmp/E7C0CD48F8BB8CFF481AC5D4408E548D1D380ED9_pub.gpg'
FPR='E7C0CD48F8BB8CFF481AC5D4408E548D1D380ED9'

# A signature file created with the secret part of the above key.
SIG="/tmp/sig"

# Set up fresh GNUPGHOME.
mkdir -m 0700 GNUPGHOME \
    && cd GNUPGHOME \
    && export GNUPGHOME="$( pwd )" \
    || exit 1

for r in $( seq 10); do
    # Run 1000x import + delete-key.
    seq 1000 \
      | while read x; do \
          gpg --batch --import "${KEY}" \
            && gpg --batch --delete-key "${FPR}"; \
        done >/dev/null 2>&1

    # Get size of pubring.
    s="$( du -h $GNUPGHOME/pubring.kbx | cut -f1 )"

    # Measure time for 'gpg --verify'.
    t="$( env time -f 'real:%e user:%U sys:%S' \
            gpg --verify "${SIG}" 2>&1 \
            | grep '^real:' )"

    printf "== After %d000 import/delete-key runs: pubring size: %s, gpg-verify time: %s\n" \
      "$r" "$s" "$t"
done
---8<---8<---8<---8<---8<---8<---8<---8<---8<---

Here's the output:
$ ./gnupg-pubring-growing.sh
== After 1000 import/delete-key runs: pubring size: 1.4M, gpg-verify time: real:0.00 user:0.00 sys:0.00
== After 2000 import/delete-key runs: pubring size: 2.7M, gpg-verify time: real:0.01 user:0.00 sys:0.01
== After 3000 import/delete-key runs: pubring size: 4.0M, gpg-verify time: real:0.02 user:0.00 sys:0.01
== After 4000 import/delete-key runs: pubring size: 5.3M, gpg-verify time: real:0.02 user:0.00 sys:0.01
== After 5000 import/delete-key runs: pubring size: 6.6M, gpg-verify time: real:0.02 user:0.00 sys:0.01
== After 6000 import/delete-key runs: pubring size: 7.9M, gpg-verify time: real:0.02 user:0.00 sys:0.02
== After 7000 import/delete-key runs: pubring size: 9.2M, gpg-verify time: real:0.03 user:0.00 sys:0.02
== After 8000 import/delete-key runs: pubring size: 11M, gpg-verify time: real:0.03 user:0.00 sys:0.03
== After 9000 import/delete-key runs: pubring size: 12M, gpg-verify time: real:0.04 user:0.00 sys:0.04
== After 10000 import/delete-key runs: pubring size: 14M, gpg-verify time: real:0.04 user:0.00 sys:0.04

Please note that not only the time of 'gpg-verify' increases but also the import and delete-key
commands take longer after each iteration.

FWIW: my application is doing like 200.000-400.000 of such actions per day in more busy installations.

Is this a known issue?

Thanks & have a nice day,
Berny