GnuPG 2.3 Beta

[Marco Ricci]

Greetings.

Thus spoke Werner Koch:
> On Tue, 23 Feb 2021 10:48, Marco Ricci said:
>
>> Forgive my ignorance, but why exactly do you set up a new daemon for
>> this? Just based on the description of the feature, I would have rather
>
> We use SQLite and transactions are very expensive if more than one
> process is accessing the DB.  So what we do is what all other database
> engines do: run a single process with exclusive access to the DB.  There
> is some overhead due to IPC but overall things are much faster.

I see. This makes me really sad... even though I can see the temptation
to solve the performance issue this way.

> The reason why several processes need to access the DB is that there are
> often several gpg processes running and working on the same keyring.
> This needs to be synchronized.  Without the keyboxd we lock the keyring
> and all other gpg and gpgsm processes need to wait until they can
> continue even with a read.  This multi-process serialization slows down
> everything.

I grepped the current git HEAD -- I have no idea which commit the beta
is supposed to be, I couldn't find a tag or anything -- and saw that you
don't use SQLite's WAL mode [1], which is designed for better
reader/writer-concurrency than naive full file locking. Have you tried
that already? From what the documentation says, SQLite's native Unix and
Windows drivers can make use of shared-memory primitives on both
systems, so you wouldn't need to go the slower route of synchronizing
via full file locks... on Unix and Windows at least.

  [1]: https://sqlite.org/wal.html

(I checked dev.gnupg.org, but couldn't find any mention of WAL mode.)

(Or is the design already at the WONTFIX stage?)

On that note: Both the beta and git HEAD require SQLite 3.7 in
configure/configure.ac. But in T4510 you started shipping (requiring?)
SQLite 3.28. Perhaps you need to update and/or re-evaluate which SQLite
version you require at a minimum? This might then be a good time to
check if requiring a newer SQLite and then using WAL mode solves your
performance problems.

Cheers,
Marco


-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 228 bytes
Desc: OpenPGP digital signature
URL: <https://lists.gnupg.org/pipermail/gnupg-devel/attachments/20210224/cabbb4f3/attachment.sig>