[gnutls-help] SSL priority and handshake error

[Michal Suchanek]

Hello,

I tried to dust off a piece of old GnuTLS code and I get this error:

./httpfs2-ssl: SSL init: loaded 173 CA certificate(s).
Thread main initializing SSL socket.
./httpfs2-ssl: invalid SSL priority
 NORMAL
 ^
./httpfs2-ssl: sourceforge.net:443 - ./httpfs2-ssl: SSL connection
failed: -12 Handshake failed.

The code was developed with older GnuTLS on Debian 7. The requirements
say GnuTLS >= 2.10.

It is difficult to install such an old version of GnuTLS without
breaking my system.

However, Debian still carries 2.12.20-8+deb7u5 which gives different error:

./httpfs2-ssl: SSL init: loaded 173 CA certificate(s).
Thread main initializing SSL socket.
./httpfs2-ssl: sourceforge.net:443 - ./httpfs2-ssl: SSL connection
failed: -12 Handshake failed.

3.x version available in Debian (3.5.2-1, 3.4.14-1, 3.4.13-1,
3.3.8-6+deb8u3) all have this issue.

The problem is that the r = gnutls_priority_set_direct(url->ss, ps,
&errp); line sets errp even when no error is reported.

Extra line if (!r) errp = NULL; is required to provide correct
diagnostics in GnuTLS 3.x.

So now that this turns out to be a hanshake error how do you diagnose these?

Is there some sample code for printing intelligible diagnostic in the
event handshake fails?

Thanks

Michal

The code in question looks like this (ripped from some example):

        /* Make SSL connection. */
        int r = 0;
        const char * ps = "NORMAL"; /* FIXME allow user setting */
        const char * errp = NULL;
        if (!url->ssl_initialized) {
            r = gnutls_global_init();
            if (!r)
                r = gnutls_certificate_allocate_credentials
(&url->sc); /* docs suggest to share creds */
            if (url->cafile) {
                if (!r)
                    r = gnutls_certificate_set_x509_trust_file
(url->sc, url->cafile, GNUTLS_X509_FMT_PEM);
                if (r>0)
                    fprintf(stderr, "%s: SSL init: loaded %i CA
certificate(s).\n", argv0, r);
                if (r>0) r = 0;
            }
            if (!r)
                gnutls_certificate_set_verify_function (url->sc,
verify_certificate_callback);
            gnutls_certificate_set_verify_flags (url->sc,
GNUTLS_VERIFY_ALLOW_X509_V1_CA_CRT /* suggested */
                    | url->md5 | url->md2 ); /* oprional for old cert compat */
            if (!r) url->ssl_initialized = 1;
            gnutls_global_set_log_level((int)url->ssl_log_level);
            gnutls_global_set_log_function(&logfunc);
        }
        if (r) {
            ssl_error(r, url->ss, "SSL init");
            return -1;
        }

        fprintf(stderr, "Thread %s initializing SSL socket.\n", url->tname);
        r = gnutls_init(&url->ss, GNUTLS_CLIENT);
        if (!r) gnutls_session_set_ptr(url->ss, url); /* used in cert
verifier */
        if (!r) r = gnutls_priority_set_direct(url->ss, ps, &errp);
        if (!r) errp = NULL;
        //if (!r) gnutls_set_default_priority(url->ss);
        if (!r) r = gnutls_credentials_set(url->ss,
GNUTLS_CRD_CERTIFICATE, url->sc);
        if (!r) gnutls_transport_set_ptr(url->ss,
(gnutls_transport_ptr_t) (intptr_t) url->sockfd);
        if (!r) r = gnutls_handshake (url->ss); /* FIXME
gnutls_error_is_fatal is recommended here */
        if (r) {
            close(url->sockfd);
            if (errp) fprintf(stderr, "%s: invalid SSL priority\n %s\n
%*s\n", argv0, ps, (int)(errp - ps), "^");
            fprintf(stderr, "%s: %s:%d - ", argv0, url->host, url->port);
            ssl_error(r, url->ss, "SSL connection failed");
            fprintf(stderr, "Thread %s closing SSL socket.\n", url->tname);
            gnutls_deinit(url->ss);
            errno = EIO;
            return -1;
        }