cv25519 scalar byte order

[NIIBE Yutaka]

Vincent Breitmoser <look at my.amazin.horse> wrote:
> See attached secret key and secret message with those values. This fails
> to decrypt for me if I use k for the X25519 input (before SP800-56A),
> but succeeds with reverse(k).
>
> While implementing to spec, this was unexpected for me, and it took me
> quite a while to figure out what GnuPG was doing differently. I also
> double-checked, for EdDSA the MPI value is not similarly reversed before
> it is handed as a scalar to the algorithm.

Well, I admit that it's confusing, and it would be considered a kind of
problem in GnuPG implementation.  Let us consider how we can go forward.

FWIW, I debug with the following:

  $ gpg --debug=crypto -a -o /tmp/OUTPUT.asc -a -r <ECCKEY> -e SOMEFILE
  $ gpg --debug=crypto,ipc /tmp/OUTPUT.asc


I think that there are two things; input handling to
encryption/decryption functions and output handling from those
functions.  The latter is troublesome, I suppose.


(1) INPUT handling for ECDH

In the libgcrypt API of gcry_pk_encrypt, scalar input is defined as MPI
(big-endian order).

When GnuPG calls gcry_pk_encrypt, no conversion is done for scalar
input; The octet sequence is interpreted as standard MPI (big-endian).

# In the test code of X25519 (libgcrypt/tests/t-cv25519.c), the scalar
# input K is needed to be converted from little-endian (defined in
# X25519) to big-endian, thus, we have reverse_buffer.

While gcry_pk_decrypt supports scalar input with the prefix 0x40 for
little-endian public key, gcry_pk_encrypt doesn't support scalar input
with the prefix 0x40 (yet).

I think that this is just a little problem.  It is possible for
libgcrypt to support the prefix 0x40, so that t-cv25519.c can be
less confusion.


(2) OUTPUT handling for ECDH

Both of gcry_pk_encrypt and gcry_pk_decrypt output the scalar with the
prefix 0x40.  After the prefix, it's little-endian.

In GnuPG (pk_ecdh_encrypt_with_shared_point in gnupg/g10/ecdh.c),

    For shared secret, result comes with the prefix 0x40 from libgcrypt,
    then, the octet sequence after 0x40 is used as-is (with no
    conversion of endian) to the input to KDF function.

I think that we have interpretation problem here.

In the section 7. Key Derivation Function in RFC 6637, KDF is described
referring NIST-SP800-56A and RFC 6090.

For the prefix 0x40, I interpret it as "extracting X part from the
representation of Z", no change of byte-order.
--