Generating bitwise identical keyrings with GnuPG 1 + 2

[Mihai Moldovan]

Hi


I am working on a software package alike to Debian's debian-archive-keyring.
Within that, public keys are stored in a jetring-compatible format. At build
time, these keys are essentially read/imported and concatenated in a
non-kbx-keyring.

Maintainers, for sanity, are supposed to build the keyrings locally and sign
them. The signature (only) is then included in the package.

At build time, the keyrings are generated on-the-fly using jetring and the
signature checked.

Obviously, for this to work correctly, the generated keyrings must be bitwise
identical.

Now the fun part: gpg 1.4 (which is used by older distro versions... and even if
it weren't, I'd only have gpg 2.0 as an alternative which likewise doesn't
include the changes from 2.1) and 2.2 generate different data and I'm trying to
find a way to get 2.2 to generate identical data to what 1.4 would spit out.

All of this is best illustrated by an example, emulating what jetring-build
would do (for sake of simplicity, I'll use file sizes here, even if that is not
a replacement for secure hash functions, of course):

rm -f keyringfile-gpg2.2.gpg*
# It took me quite a while to figure out that this is CRUCIAL. If the target
# keyring does not exist, gpg 2.1+ will create new keybox files by default.
# If it exists, however, they will just append data to the (empty) keyring when
# importing keys, which more or less corresponds to the old keyring format.
touch keyringfile-gpg2.2.gpg
# The tail ... pipe just reads in ASCII-armored public key data.
for i in active-keys/add-{d,n,g,X}*; do gpg2 --command-fd 0
--no-auto-check-trustdb --options /dev/null --no-default-keyring --batch --yes
--keyring ./keyringfile-gpg2.2.gpg --secret-keyring ./seckeyring.gpg --import
<(tail -n +5 $i | sed -e 's/^ \+//g'); done; rm -f keyringfile-gpg2.2.gpg~
wc -c keyringfile-gpg2.2.gpg

Output:

gpg: key E1F958385BFE2B6E: public key "X2go Debian/Ubuntu Packaging
<debian at x2go.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
gpg: key C509840B96F89133: public key "Oleksandr Shneyder
<oleksandr.shneyder at obviously-nice.de>" imported
gpg: Total number processed: 1
gpg:               imported: 1
gpg: key F4A7678C9C6B0B2B: public key "X2go Git Administrator
<git-admin at x2go.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
gpg: key 817F33270D65FBF2: public key "X2Go Packages Automatic Signing Key
<x2go-admin at x2go.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
5521 keyringfile-gpg2.2.gpg


Okay, good. That's about the same thing jetring-build would have done, with
minor difference that we don't care about. For instance, jetring-build would run
each import command in a new, empty, temporary GNUPGHOME environment and also
put seckeyring.gpg in there. It doesn't matter here because we'll not be using
anything within GNUPGHOME anyway and have no secret keys.

Now, gnupg 2.1.14 introduced a new filter functionality that can be used to
sanitize keyrings and remove "bad" data. Let's use that:

gpg2 --no-options --no-default-keyring --no-auto-check-trustdb --trustdb-name
./trustdb.gpg --no-keyring --import-options import-export --import <
./keyringfile-gpg2.2.gpg > ./keyringfile-gpg2.2.gpg.san
wc -c keyringfile-gpg2.2.gpg.san

Output:

gpg: Total number processed: 4
5345 ./keyringfile-gpg2.2.gpg.san


This is the final keyring I care for, generated with gpg 2.2 and sets our
baseline. 5345 bytes in size.


Let's do the same thing with gpg 1.4:

rm -f keyringfile-gpg1.4.gpg*
# Not actually needed for 1.4, but better safe than sorry.
touch keyringfile-gpg1.4.gpg
for i in active-keys/add-{d,n,g,X}*; do gpg1 --command-fd 0
--no-auto-check-trustdb --options /dev/null --no-default-keyring --batch --yes
--keyring ./keyringfile-gpg1.4.gpg --secret-keyring ./seckeyring.gpg --import
<(tail -n +5 $i | sed -e 's/^ \+//g'); done; rm -f keyringfile-gpg1.4.gpg~
wc -c keyringfile-gpg1.4.gpg

Output (I'll drop gnupg's output since that is not interesting):

5377 keyringfile-gpg1.4.gpg


Crap, that doesn't match the previous 5521 bytes as generated by gpg 2.2.

Sanitizing the keyring is also difficult, because 1.4 naturally doesn't have the
import-export import functionality/filter, but just for the sake of it, we'll
try to emulate it:

gpg1 --no-options --no-default-keyring --no-auto-check-trustdb --trustdb-name
./trustdb.gpg --keyring ./keyringfile-gpg1.4.gpg --export | gpg1 --no-options
--no-default-keyring --no-auto-check-trustdb --trustdb-name ./trustdb.gpg
--keyring ./keyringfile-gpg1.4.gpg.san --import; rm -f keyringfile-gpg1.4.gpg.san~
wc -c keyringfile-gpg1.4.gpg.san

Output:

5377 keyringfile-gpg1.4.gpg.san


Crap2, this seems to essentially have been a no-op, and it's confirmed by checksums:

c0943200ee62d1b2cac320e53d19a8d07f3d66c7f0f3d36d95a0b96dcb982024
*keyringfile-gpg1.4.gpg
c0943200ee62d1b2cac320e53d19a8d07f3d66c7f0f3d36d95a0b96dcb982024
*keyringfile-gpg1.4.gpg.san


Also note that the keyring generated by gpg1 doesn't match neither the first
keyring generated by gpg 2.2 nor the sanitized one.


I wonder what would happen if we'd just take the sanitized version generated by
2.2 and read it through gpg 1.4...

gpg1 --no-options --no-default-keyring --no-auto-check-trustdb --trustdb-name
./trustdb.gpg --keyring ./keyringfile-gpg2.2.gpg.san --export | gpg1
--no-options --no-default-keyring --no-auto-check-trustdb --trustdb-name
./trustdb.gpg --keyring ./keyringfile-gpg1.4.from2.2san.gpg --import; rm -f
keyringfile-gpg1.4.from2.2san.gpg~
wc -c keyringfile-gpg1.4.from2.2san.gpg

Output:

5377 keyringfile-gpg1.4.from2.2san.gpg

Oh, interesting. Same size as all the other keyring files generated through gpg
1.4. Does the checksum match as well?

c0943200ee62d1b2cac320e53d19a8d07f3d66c7f0f3d36d95a0b96dcb982024
*keyringfile-gpg1.4.gpg
c0943200ee62d1b2cac320e53d19a8d07f3d66c7f0f3d36d95a0b96dcb982024
*keyringfile-gpg1.4.from2.2san.gpg


So, to summarize, if I process a keyring file generated by gpg 2.2 with a 1.4
binary, i.e., read-in the former, export all keys and import it again, gpg 1.4
generates exactly the same file as it would when importing the keys directly.

I'm looking for a way to generate exactly the same file through gpg 2.2, though.
Is that possible? Not having to force maintainers to install gpg 1.4 would be great.



Mihai

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 899 bytes
Desc: OpenPGP digital signature
URL: <https://lists.gnupg.org/pipermail/gnupg-users/attachments/20190818/101080cc/attachment.sig>