Proposal: New keydb format

[Guilhem Moulin]

Hi Neal,

On Mon, 23 Nov 2015 at 15:15:19 +0100, Neal H. Walfield wrote:
> I rebased against master and fixed a few bugs including the large
> memory use that you reported.  Please get the latest code from the
> neal/kdb branch.

Awesome, thanks a lot for looking into this!  Yeah, the max res set is
kept small now.  I ran the whole benchmark from
<20151101184906.GA28098 at localhost.localdomain>, but only write down the
updates worth mentioning here

>> * Listing keys *
>>[…]
>>  ~$ time ./g10/gpg2 --homedir /run/shm/gnupg-kdb --list-keys >/dev/null
>>  0:00.73 (0.66 user, 0.07 sys)  108716k maxres
0:00.74 (0.68 user, 0.05 sys)  35192k maxres

>> * Listing sigs without nested lookups (--fast-list-mode) *
>>[…]
>>  ~$ time ./g10/gpg2 --homedir /run/shm/gnupg-kdb --fast-list-mode --list-sigs >/dev/null
>>  0:00.96 (0.88 user, 0.08 sys)  108692k maxres
0:00.97 (0.89 user, 0.07 sys)  35116k maxres

>> * Listing sigs with nested lookups *
>>[…]
>>  ~$ time ./g10/gpg2 --homedir /run/shm/gnupg-kdb --list-sigs >/dev/null
>>  0:05.93 (5.75 user, 0.18 sys)  107972k maxres
0:06.01 (5.84 user, 0.16 sys)  41888k maxres


So all in all, while the the running time is unchanged, the memory usage
is reduced by a factor of 2 or 3.  Sweet!

I've also benchmarked keyring merges (only new sigs) and key import.  No
bad surprise here either:

  $ time gpg --homedir /run/shm/gnupg-ring --import </tmp/key.gpg
  0:00.14 (0.12 user, 0.02 sys)  6784k maxres
  $ time gpg2 --homedir /run/shm/gnupg-kbx --import </tmp/key.gpg
  0:00.15 (0.09 user, 0.06 sys)  13420k maxres
  $ time ./g10/gpg2 --homedir /run/shm/gnupg-kdb --import </tmp/key.gpg
  0:00.07 (0.06 user, 0.00 sys)  10112k maxres

  $ time gpg --homedir /run/shm/gnupg-ring --import </tmp/newkey.asc
  0:00.19 (0.12 user, 0.07 sys)  3788k maxres
  $ time gpg2 --homedir /run/shm/gnupg-kbx --import </tmp/newkey.asc
  0:00.14 (0.03 user, 0.10 sys)  6516k maxres
  $ time ./g10/gpg2 --homedir /run/shm/gnupg-kdb --import </tmp/newkey.asc
  0:00.01 (0.00 user, 0.00 sys)  5240k maxres


There is however a typo which fails any attempt to delete a key.  The
patch is trivial:

-8<-------------------------------------------------------------------------->8-
diff --git a/g10/kdb.c b/g10/kdb.c
index da99573..b1c5544 100644
--- a/g10/kdb.c
+++ b/g10/kdb.c
@@ -909,8 +909,8 @@ kdb_delete (KDB_HANDLE hd)
 
   rc = sqlite3_exec_printf
     (hd->resource->db, NULL, NULL, &err,
-     "delete from keys where primary = %d;\n"
-     "delete from uids where primary = %d;\n"
+     "delete from keys where primary_key = %d;\n"
+     "delete from uids where primary_key = %d;\n"
      "delete from primaries where oid = %d;\n",
      hd->found.key, hd->found.key, hd->found.key);
   if (rc)
-8<-------------------------------------------------------------------------->8-

Cheers,
-- 
Guilhem.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: </pipermail/attachments/20151123/ab8a96eb/attachment.sig>