TOFU performance / DB format
aheinecke at intevation.de
Wed Oct 28 12:03:21 CET 2015
On Wednesday 28 October 2015 11:12:32 Neal H. Walfield wrote:
> I think there might be a bit of confusion. rsync doesn't do two way
> synchronization. It can be used create a backup and update that
> backup. But, it doesn't do a two-way sync.
I've tried to make two arguments. First that I don't see how the split DB
solves two way sync. Secondly that I don't think that is a great improvement
for backup / one way sync (Using the rsync example).
> Unison works by considering the files the basic unit of
> synchronization and memorizes the state of each file (by saving a
> check sum and the last modification time). When it synchronizes L and
> D, it sees that D has the original copy of 'l' and sends L's updated
> version of 'l' to D. Likewise it sees that L has the original version
> of 'd' and sends D's updated version of 'd' to L. Only if a file is
> updated on both L and D does the user need to manually intervene and
> either choose a copy or manually merge the changes.
> This approach isn't perfect, but in practice it works great: I rarely
> have to manually fix something. (I've been using this setup for about
> a decade, I think.)
> The difficulty with the TOFU data is that we update the DB whenever we
> see a new signed message. Thus, for active users of GnuPG, we expect
> there to be a fair amount of churn. Further, it is not possible to
> merge two DBs by hand. (We could and probably will write a tool to do
> this. Nevertheless, we'd then have to teach users about it, which is
> a pain.) By splitting the DB isn't many small, atomic chunks, the
> chance of a conflict decreases dramatically. But, equally important,
> if two chunks do diverge, choosing one copy randomly still results in
> a usable DB with little data loss.
My problem with two way sync was exactly the "Merge" case. When I work on my
Laptop and Desktop usually I'm in communication with similar partners. So I
would expect a tofu db conflict would be the "usual" case. Similarly my pubring
/ trustdb files would diverge anyway, breaking a two way merge. For me it is
easier to do a one way sync from time to time from / to my laptop if I know
I've made changes I want to have on the other system.
You say that the split DB makes it easier to do a two way merge, I can see the
point of that. But as it does not really "solve" two way merges imo (as there
still will be individual conflicts and other data with similar problems) I
personally don't see that it has enough value to justify having redundant code
and yet another config option to individualize a setup.
> Personally, I find this improved ability to do two-synchronization is
> a big win.
Ok. If you think the advantages outweigh the problems I'm totally fine with it.
I just wanted to raise my concerns from a support / maintenance perspective.
Even the slightest behavioral differences tend to lead to individual Bugs.
Even if the difference in the code is small now I think that the behavior of
that code might be quite different between the split / flat format.
And this is GnuPG once something is released it is usually maintained for a
veeery long time ;-) So we probably have to decide now If we want to maintain
two tofu database formats / layouts "forever"
> But, given the huge performance difference, I think it is
> reasonable to make the flat format the default DB format.
Yes please :-) I have not tested it yet but I expect that the performance
difference will be even more pronounced on Windows.
Andre Heinecke | ++49-541-335083-262 | http://www.intevation.de/
Intevation GmbH, Neuer Graben 17, 49074 Osnabrück | AG Osnabrück, HR B 18998
Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 181 bytes
Desc: This is a digitally signed message part.
More information about the Gnupg-devel