[PATCH 0/2] Changes to rsa_decrypt behavior on bad keys

[David Galos]

Apologies for the repost if you're subscribed to gnupg-devel as well as
this list. When reading the HACKING file I didn't realize that I
could've sent right to this list, so my bad.

There are two following patches, and I think at least one of them is
pretty important because the failure rate of a gcrypt-based crypto
oracle could probably leak info about factor-proximity based on failure
rate.

Below is the same text I sent to gnupg-devel (which was the wrong list)
but I think it still provides good explanation and context:

What follows are two patches based on some surprising behavior I
experienced dealing with some OpenPGP keys generated outside of GnuPG.

I'll start with the second patch first, a key with p > q but a correctly
calculated u will work for decryption about 80% of the time, but will
overflow some percentage of the time, yielding an incorrect result, and
a confusing error message from gpg -d. There are two better choices in
this case: 1. error out. 2. always work. The patch attached choses the
second option, because it would quietly make some things work, rather
than breaking them. There is an argument for the first: broken things
should break, and while from an RFC4880 point of view, the key is bad,
RSA itself doesn't specify which key needs to be bigger.

In the first patch I just validate the relationship between p, q, and u,
because I have no idea how the code would behave if that relationship
doesn't hold, but ERR_BAD_KEY seems like the right move. If Option 1
from above is chosen above, I will expand this patch to ensure p < q.

David Galos (2):
   Validate CRT factors in rsa_decrypt
   Fix intermittent failure in rsa_decrypt when p>q.

  cipher/rsa.c | 26 +++++++++++++++++++++++---
  1 file changed, 23 insertions(+), 3 deletions(-)

-- 
2.55.0.795.g602f6c329a-goog