Need to implement a gpg/gpg2-compatible tool to encrypt millions of files in unsupervised mode

[Kynn Jones]

Hi everyone,

First, please allow me to define a bit of ad-hoc
nomenclature.  I will use the uppercase tems "ENCRYPT",
"ENCRYPTION", etc. as shorthands for "compress and
AES256-encrypt", "compression and AES256 encryption", etc.
Likewise, I will use "DECRYPT", etc. as shorthands for
"[AES256] decrypt and decompress", etc.

I need to ENCRYPT ~20 million files (~150TB) for long-term
(>15y) storage.  This ENCRYPTION will be done in several
batches, and will take place over many months (due to CPU and
bandwidth limitations).

The ideal solution would produce ENCRYPTED files that can be
decrypted using standard off-the-shelf gpg/gpg2. [1]

In my search for a library I could use to do this, I found
gpgme and libgcrypt.  I tried the former, and found it not
suitable, due to frequent gpg-agent-related failures.

libgcrypt, on the other hand, is a bit too low-level for
someone who is not acquainted with the fine details of gpg's
ENCRYPTION to replicate it.  (AFAICT, using straight-up
gcry_cipher_encrypt would not necessarily produce an
encrypted file (let alone an ENCRYPTED file) that could be
decrypted/DECRYPTED with standard gpg/gpg2.)

Is there something in-between gpgme and libgcrypt that would
allow me implement the required tool?

*Alternatively*, can someone tell me of a more efficient way
than reading the gpg2 source code for me to learn how to
implement gpg-compatible ENCRYPTION/DECRYPTION using
libgcrypt?

Thank you all in advance!

kj


[1] This gpg/gpg2 compatibility requirement is important, as
an insurance that the files will be DECRYPTABLE in the
"distant" future (10-15y), even the my tool is not properly
enough maintained to be operational then.  This, of course,
assumes that gpg will have greater longevity than a privately
implemented, single-user tool like mine.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnupg-users/attachments/20190725/dbf70114/attachment.html>