Preserving public keyserver network (Re: Which keyserver)

[Andrew Gallagher]

On 23/10/2020 10:14, Bernhard Reiter wrote:
> So yes, I also believe that improvements to hockeypuck or a fresh 
> implementation could step by step get the public keyserver network up again.

I've thought about this quite a bit after my previous attempts to
reconcile recon with selective retention. I believe the solution is to
segregate the "recon" part of the process from the "retention" part.

Our current recon model requires that all packets that exist in the
keyserver network be reconned via the same method. This causes problems
when trying to apply retention policies to certain packets and not
others. For example, we almost always want revocation packets to recon,
almost *never* want user-attribute packets, and other packets such as
user-id fall somewhere in between.

If we segregate the recon and retention components, we can recon an
agreed bare minimum of packet types, without needing to apply per-key
filters and thereby avoiding any split-brain or sync-storm failures.
This minimal list of packet types would have to include primary keys and
revocation keys, but little (perhaps nothing?) else.

Along with these packets each keyserver would gossip a list of
associated hints from other keyservers. These hints would declare that
an "authoritative keyserver" exists that is serving the full key,
presumably having performed validation. The full set of packets would
not be advertised for recon, but would be available through hkp(s) as
normal (for the purposes of this section, the existence of an entry in
WKD would count as an "authoritative keyserver"). Other keyservers would
gossip that they have recently been able to download the full key from
one or more authoritative locations. Such hints would not be
cryptographically part of the key in question, so they should have an
expiration date in order to prevent stale info accumulating in the network.

A keyserver that is not willing to retain the full set of packets for a
given key could instead choose to serve them via a caching proxy or an
HTTP redirect to a server that is willing. This would allow for the full
public key to be discovered and refreshed by the usual methods, but
without every keyserver in the network having to retain its own copy.

It would still be advisable for a user to upload their full key to more
than one validating keyserver, in order to guard against service
outages, but even in the case where the only copy of the full key
becomes unavailable indefinitely, primary and revocation packets
associated with it would continue to recon. This model also has the
advantage of significantly reducing the number of packets in the recon
database.

Some other initial ideas:

* The new pool would have to be completely separate from the old pool,
because the deltas would be astronomical.

* Existing validating keyservers could cheaply "join the new pool" by
setting up a separate reconning keyserver in the pool that a) advertises
the appropriate hints on behalf of the validating keyserver and b)
submits any newly-synced packets into the validating keyserver.

* Hints could take the form of fake preferred-keyserver subpackets, in a
similar manner to fake "fpr:DEADBEEF" user-id packets that have been
previously discussed to support UID-less key refresh on legacy systems
(could both be combined in a single fake BIND sig?). These would be
amenable to recon, and naturally come with a timestamp that could be
used to expire them from the cache. Cryptographic non-verification
should ensure that real preferred-keyserver subpackets would override
such hints in client applications.

Thoughts?

-- 
Andrew Gallagher

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.gnupg.org/pipermail/gnupg-users/attachments/20201023/ba8b7b98/attachment.sig>