RSA sign/verify and hash generation functions

[Murray S. Kucherawy]

I got a fair bit further, but I'm stuck now with GnuTLS not generating the same signature as OpenSSL under the same circumstances.  Anyone that can spot what I've missed, please do let me know.

Here's what I'm doing:

gnutls_datum_t rsa_out; /* output signature */
gnutls_datum_t dd; /* SHA256 digest; size = 20 */
gnutls_datum_t key; /* private key in PEM form */
gnutls_x509_privkey_t privkey;
gnutls_privkey_t rsa_key;

key.data = <buffer containing PEM formatted private key>
key.len = strlen(key.data);

dd.data = (SHA256 of object to sign);
dd.size = 20 (size of a SHA256 digest);

assert(gnutls_privkey_init(&rsa_key) == GNUTLS_E_SUCCESS);
assert(gnutls_x509_privkey_init(&privkey) == GNUTLS_E_SUCCESS);
assert(gnutls_x509_privkey_import(privkey, &key, GNUTLS_X509_FMT_PEM) == GNUTLS_E_SUCCESS);
assert(gnutls_privkey_import_x509(rsa_key, privkey, 0) == GNUTLS_E_SUCCESS);
assert(gnutls_privkey_sign_hash(rsa_key, &dd, &rsa_out == GNUTLS_E_SUCCESS);

At this point, comparing "rsa_out" in the GnuTLS case to the OpenSSL case reveals that rsa_out.size is the same, but the data in rsa_out.data is not.

I haven't tried the public key verification code nor the public key extraction (In fact I haven't even found that yet).

Thanks,
-MSK