Old bug in gcry_mpi_invm producing wrong result

[NIIBE Yutaka]

Guido Vranken wrote:
> It says that InvMod(18446744073709551615,
> 340282366762482138434845932244680310781) is
> 170141183381241069226646338154899963903 but that's not true, because
> 170141183381241069226646338154899963903 * 18446744073709551615 %
> 340282366762482138434845932244680310781 is 4294967297, not 1.

Thank you for your report.  With libgcrypt 1.8, it works correctly.

It is tracked by: https://dev.gnupg.org/T5970

The fix I pushed is:

diff --git a/mpi/mpih-const-time.c b/mpi/mpih-const-time.c
index b527ad79..9d74d190 100644
--- a/mpi/mpih-const-time.c
+++ b/mpi/mpih-const-time.c
@@ -204,6 +204,13 @@ _gcry_mpih_cmp_ui (mpi_ptr_t up, mpi_size_t usize, unsigned long v)
     is_all_zero &= (up[i] == 0);
 
   if (is_all_zero)
-    return up[0] - v;
+    {
+      if (up[0] < v)
+        return -1;
+      else if (up[0] > v)
+        return 1;
+      else
+        return 0;
+    }
   return 1;
 }



The expression of up[0] - v is only correct on 32-bit architecture.
It may return wrong result on 64-bit architecture.
--