Keyrings file format

[Werner Koch]

On Tue,  6 Jan 2009 16:18, rjh at sixdemonbag.org said:

> Is there any possibility of the new GnuPG keyring format being something
> easier to parse?  Not necessarily XML (although I think that would be a

That depends on your needs.  It will be easier to parse than OpenPGP but
still be an opaque format.  However it uses length headers to describe a
keyblock and thus skipping over a keyblock does not need intimate
knowledge of OpenPGP.  In fact it is the same format as used by gpgsm
and already defined for years (gnupg/kbx/keybox-blob.c):


  The KeyBox uses an augmented OpenPGP/X.509 key format.  This makes
  random access to a keyblock/certificate easier and also gives the
  opportunity to store additional information (e.g. the fingerprint)
  along with the key.  All integers are stored in network byte order,
  offsets are counted from the beginning of the Blob.
  
  The first record of a plain KBX file has a special format:
  
   u32  length of the first record
   byte Blob type (1)
   byte version number (1)
   byte reserved
   byte reserved
   u32  magic 'KBXf'
   u32  reserved
   u32  file_created_at
   u32  last_maintenance_run
   u32  reserved
   u32  reserved
  
  The OpenPGP and X.509 blob are very similiar, things which are
  X.509 specific are noted like [X.509: xxx]
  
   u32  length of this blob (including these 4 bytes)
   byte Blob type (2) [X509: 3]
   byte version number of this blob type (1)
   u16  Blob flags
  	bit 0 = contains secret key material
          bit 1 = ephemeral blob (e.g. used while quering external resources)
  
   u32  offset to the OpenPGP keyblock or X509 DER encoded certificate
   u32  and its length
   u16  number of keys (at least 1!) [X509: always 1]
   u16  size of additional key information
   n times:
     b20	The keys fingerprint
  	(fingerprints are always 20 bytes, MD5 left padded with zeroes)
     u32	offset to the n-th key's keyID (a keyID is always 8 byte)
          or 0 if not known which is the case only for X509.
     u16	special key flags
  	 bit 0 = qualified signature (not yet implemented}
     u16	reserved
   u16  size of serialnumber(may be zero) 
     n  u16 (see above) bytes of serial number
   u16  number of user IDs
   u16  size of additional user ID information
   n times:
     u32	offset to the n-th user ID
     u32	length of this user ID.
     u16	special user ID flags.
  	 bit 0 =
     byte validity
     byte reserved
     [For X509, the first user ID is the Issuer, the second the Subject
     and the others are subjectAltNames]
   u16  number of signatures
   u16  size of signature information (4)
     u32	expiration time of signature with some special values:
  	0x00000000 = not checked
  	0x00000001 = missing key
  	0x00000002 = bad signature
  	0x10000000 = valid and expires at some date in 1978.
  	0xffffffff = valid and does not expire
   u8	assigned ownertrust [X509: not used]
   u8	all_validity 
             OpenPGP:  see ../g10/trustdb/TRUST_* [not yet used]
             X509: Bit 4 set := key has been revoked.  Note that this value
                                matches TRUST_FLAG_REVOKED
   u16	reserved
   u32	recheck_after
   u32	Newest timestamp in the keyblock (useful for KS syncronsiation?)
   u32	Blob created at
   u32	size of reserved space (not including this field)
        reserved space
  
      Here we might want to put other data
  
      Here comes the keyblock
  
      maybe we put a signature here later.
  
   b16	MD5 checksum  (useful for KS syncronisation), we might also want to use
      a mac here.
   b4    reserved
  
  
Not exactly what you desire but very helpful implementation wise.  There
is also the idea of an internal key listing daemon to get rid of file
locking.  There is a utility kbxdump which can be used to dump such a
keybox file in a human readable format.

The main advantage of this format is that there is no more
need to parse an OpenPGP keyring all the way down to the desired key
which for example includes the computation of fingerprints for all keys
before the one we want.  It also allows to get rid of sequential scans
by having index entries, but experiments have shown that this is not the
major performance hurdle.  The OpenPGP parsing with the fingerprint
computation is what makes operations on large keyrings very slow.

The primary reason why the format is not yet used is that it will take
some effort to automagically migrate to the new format without
introducing too many hassles.




-- 
Die Gedanken sind frei.  Auschnahme regelt ein Bundeschgesetz.