TOFU performance / DB format
aheinecke at intevation.de
Tue Oct 27 15:30:23 CET 2015
On Friday 23 October 2015 19:09:24 Neal H. Walfield wrote:
> Unfortunately, this change only helps with the flat database format.
> For the split forat, we're still looking at over 90 seconds for the
> initial listing and 3 seconds for subsequent listings.
I can confirm that thanks to your performance improvements initializing the flat
db is now really as fast as I would expect. << 1second. :-)
> I'd be interested to hear people's opinions about whether the split
> format (with its ability to be more easily synced) is still
> interesting despite the slow performance.
I'm not sure that keeping two database formats and making them configurable is
a good idea.
- It makes things harder to maintain. Obviously you have two codepaths with
different issues. e.g. The performance for the flat format is now fine while the
performance for the split format is still bad. You will receive bug reports
and you will always have to ask "are you using flat / split"
The majority will use whatever is default but you will still have to maintain
the code for the minority that uses another option.
- It makes things harder to document. "If you are using tofu-db-split format
you will have to sync that folder. If you are using tofo-db-flat format you
will have to sync that db." etc.
- I don't see how the split format solves two way sync:
-- Two way sync is not really supported in GnuPG. We have the monolithic
keyring and the trust.db and It's just not supported afaik two merge
two gnupg home dirs.
-- How should such a two-way sync work? If I would only use the "last
updated" db from two systems how would I merge them in case of conflicts?
With regards to easier sync. It can be easier Just to copy bulk data if it is
small then lots and lots of different files. E.g with rsync to a system
connected over the internet i see the following values:
(Ok that is cheating a bit as the tofu.db does not really contain any data but
rsync -rvzP tofu.d remote:/tmp/tofu.d 0.54s user 0.10s system 3% cpu 20.231
rsync -rvzP tofu.db remote:/tmp/tofu.db 0.06s user 0.00s system 1% cpu 4.323
rsync -rvzP tofu.d remote:/tmp/tofu.d 0.39s user 0.08s system 9% cpu 4.739
rsync -rvzP tofu.db remote:/tmp/tofu.db 0.02s user 0.00s system 1% cpu 1.498
Andre Heinecke | ++49-541-335083-262 | http://www.intevation.de/
Intevation GmbH, Neuer Graben 17, 49074 Osnabrück | AG Osnabrück, HR B 18998
Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 181 bytes
Desc: This is a digitally signed message part.
More information about the Gnupg-devel