cv25519 scalar byte order

[Vincent Breitmoser]

Hey folks,

I've been working recently on finally getting cv25519 to work in
OpenKeychain, and I did achieve an interoperable implementation in the
end. But there is a part that doesn't work as I expected from the spec,
maybe someone here has an explanation.

So, since we already had ECDH working for other curves, I had all the
KDF parts in place. I created an encrypted file in GnuPG and built a
unit test from that to work against. Switching out the curve operation
for X25519 didn't take long - but the test didn't work.

I got some intermediate crypto parameters from gnupg, and compared them
to my implementation. The difference in numbers happened around X25519
itself: I would give it the same numbers that GnuPG used as input for
the operation, but receive a different result and the aes-unwrap failed
afterwards.

The difference I found was that GnuPG apparently reverses the order of
bytes in the scalar before doing the X25519 operation. In its tests, it
uses the vectors from RFC 7748 as input for `test_cv(k, u)` function,
but the bytes of k are reversed before being handed to the actual
implementation:

https://github.com/gpg/libgcrypt/blob/master/tests/t-cv25519.c#L146

The input format for the X25519 parameters is defined in 4880-bis as
"0x40, followed by the custom format defined by the operation". I'm also
fairly sure I didn't similarly reverse the input scalar for EdDSA. So
this was quite the curveball for me (haha) while implementing this to
spec.

I spent some time looking at this, but couldn't find the part of the
spec that would require this reversing of bytes, and nobody I asked so
far could explain this behavior to me. Is GnuPG behaving weirdly? Or
what am I missing?

Cheers

 - V