Proposal: New keydb format

[Neal H. Walfield]

Hi Guilhem,

Thanks for taking a look at the code!

At Thu, 29 Oct 2015 15:28:28 +0100,
Guilhem Moulin wrote:
> On Thu, 29 Oct 2015 at 11:52:46 +0100, Neal H. Walfield wrote:
> > I'd be interested in any feedback as well as some more measurements of
> > the performance in real conditions.  Note: the code is incomplete, but
> > the essentials should work.
> 
> Thanks for working on this!  I'd like to see if this new format also
> improves --list-sigs, which is very slow on a keybox, especially on 32
> bits platforms (see issue1938).
> 
> Assuming an index on 64-bit key IDs, I expect a significant performance
> boost when looking up the signer's key to display their primary UID.
> Unfortunately I wasn't able to benchmark issue1938 with the new format
> as the signers' primary UID is never printed (as if --fast-list-mode was
> always set):
> 
>   ~$ ./g10/gpg2 --homedir /run/shm/gnupg-kdb --with-colons --list-sigs | \
>      grep -E '^(sig|rev):' | \
>      grep -vE '^(sig|rev):([^:]*:){8}\[User ID not found\]:'
> 
> Care to look into that?  I'd be happy to resume the benchmark later.

I spent a little bit of time on this this evening and found the bug.
I think --with-colons --list-sigs should now work.

> Also, probably due to the slow “full scan” you mentioned, note that
> merely looking all sigs but avoiding nested lookups with
> ‘--fast-list-mode --list-sigs’ is much slower with kdb than with a
> keybox:

I added a couple of optimizations as well.

Please feel free to run some benchmarks.


FWIW, I probably won't be working on this much in the near future as
it is not currently a priority for us.

:) Neal