[gnutls-help] too few bits from gnutls_dh_params_generate2() ?

[Pierre Ossman]

Hi,

We're having some interoperability issues between Java's SSLEngine and
GnuTLS in TigerVNC.

Java will throw this at us sometimes (actually, rather often):

> Caused by: java.security.InvalidAlgorithmParameterException: Prime size must be multiple of 64, and can only range from 512 to 2048 (inclusive)
> 	at com.sun.crypto.provider.DHKeyPairGenerator.initialize(DHKeyPairGenerator.java:120)
> 	at java.security.KeyPairGenerator$Delegate.initialize(KeyPairGenerator.java:658)
> 	at sun.security.ssl.DHCrypt.<init>(DHCrypt.java:127)
> 	... 10 more

After some debugging it turns out that the failing criteria is that
multiple of 64 bits requirement[1]. For some reason I've gotten a 1023
bit prime, even though I called gnutls_dh_params_generate2() with 1024
as the argument.

One example set of parameters I've gotten:

>  TLS:         DH prime:
>               691e93a4e2dcd04a785abd633b6c066c404809815b6983f140fa8e0cad702ffffd15e7b8361e9924858494df07a7cff50d1b971e4ce1ab396647183b4222aded580f7a079203980c952e8443e2dde055793307c407c686c34af4a5309077023f078e0443bb4b5662c20af6af6958a8d2a2c52a50267428dac8e15d7777b49d6b
>  TLS:         DH generator:
>               5783a44a1aae0e098a9474b191251397812fc201f4e38d58e9ea96f2a83793a2468f9bbc55c82b6e4c55e6674ef23db59de38f3446d1c6b84f5837f350d9b1598abe09c79a83c39402bcc53c9f4444b76bdb0f6b4c0a5ccbd3bf76a794f4e307912127bffcc81261ae4ae3bf36a20a02ec65251e4778a8e58e11f22e685bbf59
>  TLS:         DH bits: 158

This is with GnuTLS 3.2.15 and nettle 2.7.1 on Windows.

Who's to blame here? GnuTLS? Java? Us? Everybody? :)

And what do I do about it? Keep calling gnutls_dh_params_generate2()
until I get what I need?

[1] Is that even a valid requirement? I cannot find any reference for
this except that Java code.

Rgds
-- 
Pierre Ossman           Software Development
Cendio AB		https://cendio.com
Teknikringen 8		https://twitter.com/ThinLinc
583 30 Linköping	https://facebook.com/ThinLinc
Phone: +46-13-214600	https://plus.google.com/+CendioThinLinc

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: not available
URL: </pipermail/attachments/20141110/4aa3879b/attachment.sig>