[gnutls-help] too few bits from gnutls_dh_params_generate2() ?

[Daniel Kahn Gillmor]

Hi Pierre--

On Mon 2014-11-10 08:25:56 -1000, Pierre Ossman wrote:
> We're having some interoperability issues between Java's SSLEngine and
> GnuTLS in TigerVNC.

what version of Java and its SSLEngine are you using?

> Java will throw this at us sometimes (actually, rather often):
>
>> Caused by: java.security.InvalidAlgorithmParameterException: Prime size must be multiple of 64, and can only range from 512 to 2048 (inclusive)
>> 	at com.sun.crypto.provider.DHKeyPairGenerator.initialize(DHKeyPairGenerator.java:120)
>> 	at java.security.KeyPairGenerator$Delegate.initialize(KeyPairGenerator.java:658)
>> 	at sun.security.ssl.DHCrypt.<init>(DHCrypt.java:127)
>> 	... 10 more
>
> After some debugging it turns out that the failing criteria is that
> multiple of 64 bits requirement[1]. For some reason I've gotten a 1023
> bit prime, even though I called gnutls_dh_params_generate2() with 1024
> as the argument.

ugh.  Java is at fault here -- there's no sense in this particular
severe limitation.  if they're willing to use 512-bit DHE parameters and
1024-bit DHE parameters, they should be willing to use 1023-bit DHE
parameters.

That said, i suppose it's possible that gnutls could always ensure that
the high bit is set when generating a prime of a given size.

> One example set of parameters I've gotten:
>
>>  TLS:         DH prime:
>>               691e93a4e2dcd04a785abd633b6c066c404809815b6983f140fa8e0cad702ffffd15e7b8361e9924858494df07a7cff50d1b971e4ce1ab396647183b4222aded580f7a079203980c952e8443e2dde055793307c407c686c34af4a5309077023f078e0443bb4b5662c20af6af6958a8d2a2c52a50267428dac8e15d7777b49d6b
>>  TLS:         DH generator:
>>               5783a44a1aae0e098a9474b191251397812fc201f4e38d58e9ea96f2a83793a2468f9bbc55c82b6e4c55e6674ef23db59de38f3446d1c6b84f5837f350d9b1598abe09c79a83c39402bcc53c9f4444b76bdb0f6b4c0a5ccbd3bf76a794f4e307912127bffcc81261ae4ae3bf36a20a02ec65251e4778a8e58e11f22e685bbf59
>>  TLS:         DH bits: 158


what is this output from?  I'm not sure how to reconcile the "DH bits:
158" with the other data.

> This is with GnuTLS 3.2.15 and nettle 2.7.1 on Windows.
>
> Who's to blame here? GnuTLS? Java? Us? Everybody? :)
>
> And what do I do about it? Keep calling gnutls_dh_params_generate2()
> until I get what I need?

arguably, gnutls could keep the high bit set in its generated primes,
just to coddle broken peers like this java implementation.

> [1] Is that even a valid requirement? I cannot find any reference for
> this except that Java code.

have you reported this bug to java?  it sounds like they should not be
doing this.

      --dkg