ecc: compact representation of an elliptic curve point

[NIIBE Yutaka]

Hello,

This week, I look ECC implementation and I found a bug in compact
representation handling.

The reference is now:

    https://www.ietf.org/archive/id/draft-jivsov-ecc-compact-05.txt

The problem is that we have a path for MPI_EC_EDWARDS which handles
x-coordinate.  But comparison will be done with y-coordinate.

I wrote a patch like following.  I think that other Edwards curves
would follow the practice of Ed25519 for its secret key tweak, so,
this is likely to be a dead path.

Nevertheless, to avoid confusion, it makes sence to appy this fix.


diff --git a/cipher/ecc.c b/cipher/ecc.c
index 51621f8..1933978 100644
--- a/cipher/ecc.c
+++ b/cipher/ecc.c
@@ -195,15 +195,22 @@ nist_generate_key (ECC_secret_key *sk, elliptic_curve_t *E, mpi_ec_t ctx,
   else
     {
       gcry_mpi_t negative;
+      int comparison;

       negative = mpi_new (pbits);

       if (E->model == MPI_EC_WEIERSTRASS)
-        mpi_sub (negative, E->p, y);      /* negative = p - y */
+        {
+          mpi_sub (negative, E->p, y);      /* negative = p - y */
+          comparison = mpi_cmp (negative, y);
+        }
       else
-        mpi_sub (negative, E->p, x);      /* negative = p - x */
+        {
+          mpi_sub (negative, E->p, x);      /* negative = p - x */
+          comparison = mpi_cmp (negative, x);
+        }

-      if (mpi_cmp (negative, y) < 0)   /* p - y < p */
+      if (comparison < 0)   /* p - y < p (or p - x < p) */
         {
           /* We need to end up with -Q; this assures that new Q's y is
              the smallest one */
--