status of S/MIME and interoperability?

Ed Finnerty edfinnerty at
Thu Feb 18 13:13:17 CET 2016

On 02/18/2016 05:33 AM, Daiki Ueno wrote:
> Daiki Ueno <ueno at> writes:
>> Ed Finnerty <edfinnerty at> writes:
>>>> I've actually logged the problem here and even attached a script that
>>>> easily tests for the issue and can be modified to produce trivial test
>>>> output for comparison purposes.
>>>> Nobody seemed to really care. So if you do S/MIME with GPG, and use
>>>> Thunderbird, every once in a while you'll get an undecryptable message
>>>> that you have to save to disk and try to deal with in some way with GPG.
>>> Of course, I forgot to add the link:
>> Although I am not sure if this is related to the original issue of this
>> thread, I have the same impression that this is an NSS bug.
> Actually, reading the CMS specification more closely, my initial
> impression seems wrong, and this looks really like an issue in gpgsm
> (actually in libksba, see below).
>> In my test, it happens only when the encrypted session key is shorter
>> than 256 bytes.  However, I don't see any problem in the CMS data which
>> gpgsm produced.
> When rsaEncryption is used, the length of the encrypted session key
> should never change, as long as the same key is used.
> The rationale is:
> - The section 4.2 of RFC3370 states:
>    The RSA key transport algorithm is the RSA encryption scheme defined
>    in RFC 2313 [PKCS#1], block type is 02, where the message to be
>    encrypted is the content-encryption key.
>   ...
>    Note that the same RSA encryption scheme is also defined in RFC 2437
>    [NEWPKCS#1].  Within RFC 2437, this RSA encryption scheme is called
>    RSAES-PKCS1-v1_5.
> - In the section 7.2.1 of RFC2437, the step 4 of RSAES-PKCS1-v1_5 calls
>   I2OPS as:
>    4. Convert the ciphertext representative c to a ciphertext C of
>    length k octets: C = I2OSP (c, k)
>   where k is the length in octets of the modulus n
> - I2OSP(x,l) adds leading zeros if the length of x in octets is shorter
>   than l, as noted in the step 2 of I2OSP
> In libksba/src/cms.c, ksba_cms_set_enc_val removes the first leading
> zero:
>   1834    if (n > 1 && !*s)
>   1835      { /* We might have a leading zero due to the way we encode
>   1836           MPIs - this zero should not go into the OCTECT STRING.  */
>   1837        s++;
>   1838        n--;
>   1839      }
> I think this should be bypassed, at least when rsaEncryption is used.
> With the attached patch, I could no longer reproduce the issue with the
> provided script (with ~1000 iterations).

Thanks! Hopefully this patch will make it into GPG.

More information about the Gnupg-devel mailing list