gcry_mpi_invm succeeds if the inverse does not exist

[Jussi Kivilinna]

Hello,

On 8.5.2020 1.24, Guido Vranken via Gcrypt-devel wrote:
> I'm running my crypto library fuzzer on Google OSS-Fuzz and it keeps finding bugs in gcry_mpi_invm.
> 
> jussi kivilinna receives these reports. Would you mind taking a look?

I did see "Issue 21708 in oss-fuzz: cryptofuzz:cryptofuzz-openssl: ASSERT: Botan-libgcrypt-BignumCalc-(no algorithm)-difference" 2020-04-15 and follow-up discussion and that resulted bug-report to this mailing list. I have not seen any further reports from OSS-Fuzz. 

If this keeps causing too much noise as repeated bug reports, cannot you disable the BigNum fuzzing of libgcrypt until this gets fixed?

-Jussi

> 
> On Tue, Apr 14, 2020 at 8:32 AM NIIBE Yutaka <gniibe at fsij.org <mailto:gniibe at fsij.org>> wrote:
> 
>     Hello,
> 
>     Thank you for your report with a test case.  It helps a lot.
> 
>     Guido Vranken wrote:
>     > This is not in accordance with the documentation:
>     >
>     > Function: int gcry_mpi_invm (gcry_mpi_t x, gcry_mpi_t a, gcry_mpi_t m)
>     > Set x to the multiplicative inverse of a \bmod m. Return true if the
>     > inverse exists.
> 
>     Indeed.  It seems that API was changed to return the result
>     (success/failure) in libgcrypt 1.3.2 in 2007.  Before that, the API was
>     void (no return value).  Since the initial change, it never returns
>     correct value.
> 
>     I'll fix for 1.9.
>     -- 
> 
> 
> _______________________________________________
> Gcrypt-devel mailing list
> Gcrypt-devel at gnupg.org
> http://lists.gnupg.org/mailman/listinfo/gcrypt-devel
>