2.5.7 gnutls_x509_privkey_generate() returns GNUTLS_E_INVALID_REQUEST

[Sam Varshavchik]

Simon Josefsson writes:

> Sam Varshavchik <mrsam at courier-mta.com> writes:
> 
>> The following short test program runs when compiled against
>> 2.4.0. Compiling it against 2.5.7 causes it to report a
>> GNUTLS_E_INVALID_REQUEST from the second call to
>> gnutls_x509_privkey_generate().
> 
> I can't reproduce this, adding this somewhere:
> 
>   printf ("vers %s %s\n", LIBGNUTLS_VERSION, gnutls_check_version (NULL));
> 
> Does print 2.5.7 for both, confirming that I really use 2.5.7.  So it
> seems something else is required to reproduce this.  Can you try to
> debug gnutls_x509_privkey_generate and see what happens?
> 
> Does 'certtool -p' trigger the same problem for you?

The bug seems to be easy to spot. I think this is it:

int
gnutls_x509_privkey_generate (gnutls_x509_privkey_t key,
                              gnutls_pk_algorithm_t algo, unsigned int bits,
                              unsigned int flags)
{
  int ret;
  unsigned int params_len;

// . . .

    ret = _gnutls_rsa_generate_params (key→params, &params_len, bits);

This goes into:

static int
_generate_params (int algo, bigint_t *resarr, unsigned int *resarr_len,
                  int bits)

// . . .

  if (resarr && resarr_len && *resarr_len > params.params_nr)
                              ===========

Looks like *resarr_len points to uninitialized memory at this point. 
gnutls_x509_privkey_generate() never initialized params_len, as far as I can 
tell.




-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: </pipermail/attachments/20080917/b8e49461/attachment.pgp>