From dmacks at netspace.org  Wed Jun  4 23:00:34 2014
From: dmacks at netspace.org (Daniel E. Macks)
Date: Wed, 4 Jun 2014 21:00:34 +0000 (UTC)
Subject: [gnutls-help] Self-test fail: apple's libtool is weird
Message-ID: <slrnlov27i.2cq9.dmacks@macks-factor.home>

Building gnutls-3.3.4 on OS X 10.8 with valgrind available gives
several self-test failures in cert-tests: aki, pathlen, pem-decoding.
In all cases, the log indicates that libtool failed due to invalid
flags being used. The "libtool" program Apple supplies is not GNU
libtool, but instead an OS X specific tool for handling some binary
archives. These tests want to call GNU libtool in order to run the GNU
libtool generated programs it just built, where for example from
tests/cert-tests/aki:

  VALGRIND="libtool --mode=execute ${VALGRIND}"

But that syntax finds apple's libtool (via PATH search). The right
libtool isn't in PATH at all and isn't present on OS X at all. But it
is available right in the build dir, where it was generated to build
gnutls itself. So the solution is to hardcode use of that local one:

  VALGRIND="../../libtool --mode=execute ${VALGRIND}"

and also in tests/cert-tests/pathlen and tests/cert-tests/pem-decoding

This might only be *required* for platforms where GNU libtool is not
first in PATH that also do have valgrind (either from vendor or
manually installed). But I also don't see a down-side for others.

dan

-- 
Daniel Macks
dmacks at netspace.org




From nmav at gnutls.org  Thu Jun  5 09:15:42 2014
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Thu, 5 Jun 2014 09:15:42 +0200
Subject: [gnutls-help] Self-test fail: apple's libtool is weird
In-Reply-To: <slrnlov27i.2cq9.dmacks@macks-factor.home>
References: <slrnlov27i.2cq9.dmacks@macks-factor.home>
Message-ID: <CAJU7zaJhri6Z2wL_Qj+hSOgyOvqJ4ptnQ1zVj_ziOB=9YbCS=w@mail.gmail.com>

On Wed, Jun 4, 2014 at 11:00 PM, Daniel E. Macks <dmacks at netspace.org> wrote:
> Building gnutls-3.3.4 on OS X 10.8 with valgrind available gives
> several self-test failures in cert-tests: aki, pathlen, pem-decoding.
> In all cases, the log indicates that libtool failed due to invalid
> flags being used. The "libtool" program Apple supplies is not GNU
> libtool, but instead an OS X specific tool for handling some binary
> archives. These tests want to call GNU libtool in order to run the GNU
> libtool generated programs it just built, where for example from
> tests/cert-tests/aki:
>   VALGRIND="libtool --mode=execute ${VALGRIND}"

Thank you for reporting that. I've solved that a bit differently. I
now pass the $LIBTOOL variable to scripts and they will use that one
(normally in your system it would be the shipped libtool).

regards,
Nikos


From bennyboysmith at gmail.com  Thu Jun  5 16:28:48 2014
From: bennyboysmith at gmail.com (Ben Mohamoodally)
Date: Thu, 5 Jun 2014 15:28:48 +0100
Subject: [gnutls-help] gnutls and libtasn1 causing SSL issues with SVN
Message-ID: <CALBV0a3s2CVLVfp0oz-jhVzywCt+fzd1PeZGuZ5bG2u8CL2hCQ@mail.gmail.com>

Hello,

Linux machines (SL6.1 to 6.4) on my estate use GnuTLS to securely connect
to our SVN server.

However over the last two days I've noticed that an updated version of
gnutls was installed on a the Linux machines.

(/var/log/yum.log)

Jun 04 05:10:12 Updated: gnutls-2.8.5-14.el6_5.x86_64

The odd thing I noticed was that an additional package was also installed
for the first time from core-0 repo.

Jun 04 05:10:11 Updated: libtasn1-2.3-6.el6_5.x86_64

I am trying to understand the link between the two packages. I understand
that libtasn1 is a library suite that GnuTLS uses.

Since this update occurred, all Linux servers weren't able to connect to
the SVN server.

I was seeing the following error depending on the machine:

svn: OPTIONS of 'https://svn.xxxxxxx.co.uk/xxxxxx': SSL handshake failed,
client certificate was requested: SSL socket write failed (
https://svn.xxxxxx.co.uk)

Or

SSL handshake failed, client certificate was requested: SSL error: GnuTLS
internal error.

The only way to fix this was to run 'yum downgrade libtasn1' which removed
the package.

This resolved the issue and everything is OK again even with the
gnutls-2.8.5-14.el6_5.x86_64 left as is. i,e not downgrading gnutls!

My question is what part of libtasn1 would cause SSL connections to to svn
to break?
Second question is, is it ok to not have libtasn1 installed?

I appreciate any advice you can give.

Best Regards,

Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/attachments/20140605/3cbf7976/attachment.html>

From nmav at gnutls.org  Thu Jun  5 17:38:31 2014
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Thu, 5 Jun 2014 17:38:31 +0200
Subject: [gnutls-help] gnutls and libtasn1 causing SSL issues with SVN
In-Reply-To: <CALBV0a3s2CVLVfp0oz-jhVzywCt+fzd1PeZGuZ5bG2u8CL2hCQ@mail.gmail.com>
References: <CALBV0a3s2CVLVfp0oz-jhVzywCt+fzd1PeZGuZ5bG2u8CL2hCQ@mail.gmail.com>
Message-ID: <CAJU7zaLsh83EVecSsYBph9QZ-j-SP_v8TbwdQMhMM2h+_7o4VQ@mail.gmail.com>

On Thu, Jun 5, 2014 at 4:28 PM, Ben Mohamoodally
<bennyboysmith at gmail.com> wrote:
> Hello,
> Linux machines (SL6.1 to 6.4) on my estate use GnuTLS to securely connect to
> our SVN server.
> However over the last two days I've noticed that an updated version of
> gnutls was installed on a the Linux machines.
> (/var/log/yum.log)
> Jun 04 05:10:12 Updated: gnutls-2.8.5-14.el6_5.x86_64
> The odd thing I noticed was that an additional package was also installed
> for the first time from core-0 repo.
> Jun 04 05:10:11 Updated: libtasn1-2.3-6.el6_5.x86_64
> I am trying to understand the link between the two packages. I understand
> that libtasn1 is a library suite that GnuTLS uses.

That's true.

> Since this update occurred, all Linux servers weren't able to connect to the
> SVN server.
> I was seeing the following error depending on the machine:
> svn: OPTIONS of 'https://svn.xxxxxxx.co.uk/xxxxxx': SSL handshake failed,
> client certificate was requested: SSL socket write failed
> (https://svn.xxxxxx.co.uk)

You most probably need to open a bug in your OS vendor about the
libtasn1 component.

> SSL handshake failed, client certificate was requested: SSL error: GnuTLS
> internal error.

Could you send me the name of the server in private? I'd be interested
to check whether other versions of libtasn1 are affected and whether
there is something there to fix.

regards,
Nikos


From paul.morriss at tokenbay.co.uk  Sat Jun  7 13:19:27 2014
From: paul.morriss at tokenbay.co.uk (Paul Morriss)
Date: Sat, 07 Jun 2014 12:19:27 +0100
Subject: [gnutls-help] Windows build for 3.3
Message-ID: <a1c1b59dc913fc5e3213035854919b01@tokenbay.co.uk>

Are there any instructions on how to build GnuTLS for Visual Studio 
(2010 onwards) please?  Or, is there going to be an update to the 
precompiled Windows Binary to add 3.3 binaries?

Thank you in advance :)

Paul
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Please give generously to Great Ormond Street Hospital, London, UK
A specialist hospital for sick children who need every penny!

Great Ormond Street Hospital Children's Charity (GOSHCC) website:
http://www.gosh.org/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


From m.postman at mafrigo.net  Wed Jun 11 19:50:49 2014
From: m.postman at mafrigo.net (m.postman at mafrigo.net)
Date: Wed, 11 Jun 2014 19:50:49 +0200
Subject: [gnutls-help] Create csr with netscape extension = server
Message-ID: <539896F9.1060906@mafrigo.net>

Hi,
i've been working on this problem quite long now.
OpenLDAP on my OpenSuSE 13.1 is compiled with gnutls apparently.
But connecting to the OpenLDAP server fails with the following message:

# ldapsearch -h localhost -W -D uid=admin,dc=example,dc=net -b 
dc=example,dc=net -s sub "(uid=user1)" -v -ZZ
ldap_initialize( ldap://localhost )
ldap_start_tls: Connect error (-11)
         additional info: error:14090086:SSL 
routines:SSL3_GET_SERVER_CERTIFICATE:_certificate verify failed 
(unsupported certificate purpose)__
__
_Tracking down this error lead to a missing "Netscape Extension" called 
"server".
[Source: 
http://www.openldap.org/lists/openldap-software/200704/msg00278.html]

Well... _how do I create a CSR with gnutls/certtool with this 
extension??_ :)

I simply can't figure it out... maybe I missed something?!
In openssl there is a directive "nsCertType = server"... I suppose 
that's what I am looking for :)

I appreciate any help. Thank you very much in advance!
Marc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/attachments/20140611/476ab19a/attachment.html>

From nmav at gnutls.org  Thu Jun 12 11:12:28 2014
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Thu, 12 Jun 2014 11:12:28 +0200
Subject: [gnutls-help] Create csr with netscape extension = server
In-Reply-To: <539896F9.1060906@mafrigo.net>
References: <539896F9.1060906@mafrigo.net>
Message-ID: <CAJU7zaL=0i4Rgbcyt82gpy7+kWxJ3F_8au7NLb63M8YJOhJjfw@mail.gmail.com>

On Wed, Jun 11, 2014 at 7:50 PM,  <m.postman at mafrigo.net> wrote:
> Hi,
> i've been working on this problem quite long now.
> OpenLDAP on my OpenSuSE 13.1 is compiled with gnutls apparently.
> But connecting to the OpenLDAP server fails with the following message:
> # ldapsearch -h localhost -W -D uid=admin,dc=example,dc=net -b
> dc=example,dc=net -s sub "(uid=user1)" -v -ZZ
> ldap_initialize( ldap://localhost )
> ldap_start_tls: Connect error (-11)
>         additional info: error:14090086:SSL
> routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed (unsupported
> certificate purpose)

This is not a gnutls error. Most likely is comes from openssl. My
guess would be that your server certificate doesn't have the correct
purpose set, or has some purpose set that is unknown to it.

> Tracking down this error lead to a missing "Netscape Extension" called
> "server".

I doubt that any software would use that extension. It has been dead
since a decade.
Most likely you need to consult the key purpose extensions. My guess
would be that it requires the "tls_www_server" option to the certtool
template.

regards,
Nikos


From lavr at ncbi.nlm.nih.gov  Thu Jun 12 22:28:11 2014
From: lavr at ncbi.nlm.nih.gov (Lavrentiev, Anton (NIH/NLM/NCBI) [C])
Date: Thu, 12 Jun 2014 20:28:11 +0000
Subject: [gnutls-help] Regression bug between 2.x and 3.2?
Message-ID: <5F8AAC04F9616747BC4CC0E803D5907D0C90115D@MLBXv04.nih.gov>

Hello,

Our code is using gnutls_transport_set_{push|pull}_function() and is having problems since
we've switched from an old GNUTLS 2.4.2 to newer 3.2.13.  Namely, when the push callback
experiences a short write (i.e. reporting fewer written bytes than it was requested to
write), gnutls_record_send() returns an error, rather than the number of bytes that
it has succeeded to write before the underlying short write (and which it used to do formerly).

Is that the expected behavior?  gnutls_record_send() seems to be documented to be able
to return a partial write.

Anton Lavrentiev
Contractor NIH/NLM/NCBI



From lavr at ncbi.nlm.nih.gov  Thu Jun 12 23:37:37 2014
From: lavr at ncbi.nlm.nih.gov (Lavrentiev, Anton (NIH/NLM/NCBI) [C])
Date: Thu, 12 Jun 2014 21:37:37 +0000
Subject: [gnutls-help] Regression bug between 2.x and 3.2?
Message-ID: <5F8AAC04F9616747BC4CC0E803D5907D0C9012D1@MLBXv04.nih.gov>

I am correcting my message to add the following.  Formerly, short writes by the push callback
were allowed to be followed by another try by GNUTLS.  Current GNUTLS release stops the tries after
seeing the first short write.  This means the callback is no longer following the "send()"
paradigm, i.e. the callback itself is expected to wait even though it has written
some portion of the requested buffer (but not all of it).  send() returns immediately
once at least one byte has been sent (and any remaining portion would have caused the blocking).

I was wrong saying that gnutls_record_send() used to return after a short write.
It did not -- but since the loop "while (left > 0)" was only broken if the callback
returned -1, it let the callback to manage waiting for the output device if it was not
ready _prior_ to output, not after it.

Anton Lavrentiev
Contractor NIH/NLM/NCBI

> -----Original Message-----
> From: Lavrentiev, Anton (NIH/NLM/NCBI) [C]
> Sent: Thursday, June 12, 2014 4:28 PM
> To: gnutls-help at gnutls.org
> Subject: Regression bug between 2.x and 3.2?
> 
> Hello,
> 
> Our code is using gnutls_transport_set_{push|pull}_function() and is
> having problems since
> we've switched from an old GNUTLS 2.4.2 to newer 3.2.13.  Namely, when
> the push callback
> experiences a short write (i.e. reporting fewer written bytes than it
> was requested to
> write), gnutls_record_send() returns an error, rather than the number of
> bytes that
> it has succeeded to write before the underlying short write (and which
> it used to do formerly).
> 
> Is that the expected behavior?  gnutls_record_send() seems to be
> documented to be able
> to return a partial write.
> 
> Anton Lavrentiev
> Contractor NIH/NLM/NCBI



From nmav at gnutls.org  Fri Jun 13 11:12:50 2014
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Fri, 13 Jun 2014 11:12:50 +0200
Subject: [gnutls-help] Regression bug between 2.x and 3.2?
In-Reply-To: <5F8AAC04F9616747BC4CC0E803D5907D0C9012D1@MLBXv04.nih.gov>
References: <5F8AAC04F9616747BC4CC0E803D5907D0C9012D1@MLBXv04.nih.gov>
Message-ID: <CAJU7zaKXmSbsmo3e0woN7oKRO_EbpDTpxuMN5PVyrCuBk9KEuw@mail.gmail.com>

On Thu, Jun 12, 2014 at 11:37 PM, Lavrentiev, Anton (NIH/NLM/NCBI) [C]
<lavr at ncbi.nlm.nih.gov> wrote:
> I am correcting my message to add the following.  Formerly, short writes by the push callback
> were allowed to be followed by another try by GNUTLS.  Current GNUTLS release stops the tries after
> seeing the first short write.  This means the callback is no longer following the "send()"
> paradigm, i.e. the callback itself is expected to wait even though it has written
> some portion of the requested buffer (but not all of it).  send() returns immediately
> once at least one byte has been sent (and any remaining portion would have caused the blocking).

Hello Anton,
 Could you provide a gnutls trace (use GNUTLS_DEBUG_LEVEL=6 or more),
or better a reproducer for the issue? (e.g., you could base it on some
of the existing gnutls tests like mini-eagain.c)

regards,
Nikos


From lavr at ncbi.nlm.nih.gov  Fri Jun 13 18:21:23 2014
From: lavr at ncbi.nlm.nih.gov (Lavrentiev, Anton (NIH/NLM/NCBI) [C])
Date: Fri, 13 Jun 2014 16:21:23 +0000
Subject: [gnutls-help] Regression bug between 2.x and 3.2?
In-Reply-To: <CAJU7zaKXmSbsmo3e0woN7oKRO_EbpDTpxuMN5PVyrCuBk9KEuw@mail.gmail.com>
References: <5F8AAC04F9616747BC4CC0E803D5907D0C9012D1@MLBXv04.nih.gov>
 <CAJU7zaKXmSbsmo3e0woN7oKRO_EbpDTpxuMN5PVyrCuBk9KEuw@mail.gmail.com>
Message-ID: <5F8AAC04F9616747BC4CC0E803D5907D0C902274@MLBXv04.nih.gov>

>  Could you provide a gnutls trace (use GNUTLS_DEBUG_LEVEL=6 or more),

With GNUTLS 3.2.13, here's the point of failure (SSOCK#1000 is our callback's output that shows that it
could write only 528 bytes -- GNUTLS was pushing 1053).  After that the control gets back from
gnutls_record_send() with a GNUTLS_E_EAGAIN return code rather than the callback is tried one
more time (it's how send() should have been used).  Note that we're using non-blocking output
in the callback.  So when the device is full, it won't accept any data at all and won't block,
so the callback does a short wait, then tries again and returns immediately if anything at all
has been written successfully.

---BEGIN---
06/13/14 11:12:59 GNUTLS7: WRITE FLUSH: 1053 bytes in buffer.
06/13/14 11:12:59 SSOCK#1000[3]@130.14.29.110:443: Written at offset 17469
#################### [BEGIN] Raw Data (528 bytes):
\27\3\3\4\30\0\0\0\0\0\0\0\22\222\321\351\267\357\22343[\243\361\213\341,\275\36\342\214\35\200S\241\240\27\324\n\
q7\347  \\\377\257\20\2\20W%Z\241\224\241JX\250\"_\240\23iX3\205f\373<@G\344\260\246KS\262\33\313*\216\231I\356\223\260\342\343\b\216\267\234\372\256&\256\300\367+\350iLmq\226=\21
h\245@\343\364\223t\220+\34%kTs\37}\234\335-\307\216\205\202\357@:\254\253\223{/\265\237i \3678\1|t\2441%v\252\246|\34\335\31\355\215\3\207\311\20\27\316\36CG[^\331\245\336d\261R\
#################### [END] Raw Data

06/13/14 11:12:59 GNUTLS7: WRITE: wrote 528 bytes, 525 bytes left.
---END---
At this point I get GNUTLS_E_EAGAIN from gnutls_record_send(), presumably because this gets executed in gnutls_buffers.c:
        if (sent < tosend) {
                return gnutls_assert_val(GNUTLS_E_AGAIN);
        }


GNUTLS 2.4.2 behaves differently, it calls the push function again even after a short write.
Unable to write anything right away, our push function waits for the output device to become
ready, and writes the next portion of the buffer, thus the entire writing process advances.

---BEGIN---
06/13/14 11:59:33 GNUTLS4: REC[15f42120]: Sending Packet[2033] Application Data(23) with length: 1024
06/13/14 11:59:33 GNUTLS7: WRITE: Will write 1301 bytes to 368319552.
06/13/14 11:59:33 SSOCK#1000[0]@130.14.29.110:443: Written at offset 2403083
#################### [BEGIN] Raw Data (480 bytes):
\27\3\2\5\20\306Uq\367\t\36#\330D\257x\v\341:})z\246o\334\213y4kN\2213>\236\345\350h\21\245O\305\21Q\263\270\203\245V`\236\200\330\250Xr\264xQ\225_\267*\30r#\350\237*\257:\251n\32
\245\v\6\252\4R\21\34\241\364$o@\250\353\222/\235\336\355\0\334\351\270\206\350\5g\20\b\333\372:5\300\350\373\330\265\265%\261F\266!\36\0[\207\364\343\b\242Xr\221\346X\251k\301\n\
\210\212kdA\334\321\276\'\374%\332t\3204\5\216\3354\335;o&\277\262\206\265\376\337V&\25&\26\25\0236\310^\n\
\247\301Yk\324\235\202\261\277}vt\3069<\375vo\n\
 \262\250\272\21xTL(\240\207\317\333I\347C\316Z3\372\260;G\2479D\353\24\20\200!\240\37o\26\225\350\324\206sf\305i{\a\331\20*\252/\302\314`\35472\256#/6;R\367\351\2535\22\326\331\3
#################### [END] Raw Data

06/13/14 11:59:33 GNUTLS7: WRITE: wrote 480 bytes to 368319552. Left 821 bytes. Total 1301 bytes.
...
06/13/14 11:59:33 SSOCK#1000[0]@130.14.29.110:443: Written at offset 2403563
#################### [BEGIN] Raw Data (821 bytes):
\206\363\273;\aj\306U\210pc9D\a\fU+=\5\367\33\"\324\253\224\27\254v(\27\325e\216\2023\247\310\233Es\325s\343\242H\2704rx\tD\23\361zS\252R\v\201bl\231\222Q\377\334n\205l\330o\322a\
\337(\305H\242n\260\252@\210\260\6\272\214\311\4\317R\234\344,Vu\0045\325z3\235\177\311&\344\307e\327\242\261\344\374\350t\204&W\253\205\361\271\251\326U\330\355\327\255{\331Zb\26
\335L\2W\v\215\271\334!\202c_\\)\304\215Y\303N2\363L\265\221s\356.\17h\215 o7\376\6\225\344\26\276\370I\240\226\260Npe\301\321\231\231\3361\36[\357\206n\304c\302\302\201\307\233F\
\216\240\306\222\357\207/C\230=\2126~y<VIR8\303H\372\337u_q<^\"\351 gf\262@\25[\23tjm\221\354\353\252\b\177\217\227\222\370^\2336\300x\246\370\5\32\353\303\246PB%\r\210.p\315\',\3
#################### [END] Raw Data

06/13/14 11:59:33 GNUTLS7: WRITE: wrote 821 bytes to 368319552. Left 0 bytes. Total 1301 bytes.
---END---


mini-eagain.c is not a good test, IMO.  It operates buffers that fit into internal socket
buffers entirely, and uses very short messaging that can't cause the data flood in one
direction, stalling the connection (it's when send() will experience the short write
situation).  Or the client has to be faster pushing the data to the server so that the
connection becomes full at one point.  Using blocking I/O will save the trouble, but
non-blocking will demonstrate the regression.  I'll try to come up with a useful and
short reproducer, though...


Anton Lavrentiev
Contractor NIH/NLM/NCBI



From m.postman at mafrigo.net  Sat Jun 14 16:03:12 2014
From: m.postman at mafrigo.net (m.postman at mafrigo.net)
Date: Sat, 14 Jun 2014 16:03:12 +0200
Subject: [gnutls-help] Create csr with netscape extension = server
In-Reply-To: <CAJU7zaL=0i4Rgbcyt82gpy7+kWxJ3F_8au7NLb63M8YJOhJjfw@mail.gmail.com>
References: <539896F9.1060906@mafrigo.net>
 <CAJU7zaL=0i4Rgbcyt82gpy7+kWxJ3F_8au7NLb63M8YJOhJjfw@mail.gmail.com>
Message-ID: <539C5620.5070108@mafrigo.net>

Hey,
Actually you are right, openldap on opensuse 13.1 is compiled with openssl.
I misread the output of "ldd", sorry for the inconvenience!

Thanks for your help anyway :)
Marc

Am 12.06.2014 11:12, schrieb Nikos Mavrogiannopoulos:
> On Wed, Jun 11, 2014 at 7:50 PM,  <m.postman at mafrigo.net> wrote:
>> Hi,
>> i've been working on this problem quite long now.
>> OpenLDAP on my OpenSuSE 13.1 is compiled with gnutls apparently.
>> But connecting to the OpenLDAP server fails with the following message:
>> # ldapsearch -h localhost -W -D uid=admin,dc=example,dc=net -b
>> dc=example,dc=net -s sub "(uid=user1)" -v -ZZ
>> ldap_initialize( ldap://localhost )
>> ldap_start_tls: Connect error (-11)
>>          additional info: error:14090086:SSL
>> routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed (unsupported
>> certificate purpose)
> This is not a gnutls error. Most likely is comes from openssl. My
> guess would be that your server certificate doesn't have the correct
> purpose set, or has some purpose set that is unknown to it.
>
>> Tracking down this error lead to a missing "Netscape Extension" called
>> "server".
> I doubt that any software would use that extension. It has been dead
> since a decade.
> Most likely you need to consult the key purpose extensions. My guess
> would be that it requires the "tls_www_server" option to the certtool
> template.
>
> regards,
> Nikos



From nmav at gnutls.org  Sun Jun 15 22:47:29 2014
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sun, 15 Jun 2014 22:47:29 +0200
Subject: [gnutls-help] Regression bug between 2.x and 3.2?
In-Reply-To: <5F8AAC04F9616747BC4CC0E803D5907D0C902274@MLBXv04.nih.gov>
References: <5F8AAC04F9616747BC4CC0E803D5907D0C9012D1@MLBXv04.nih.gov>
 <CAJU7zaKXmSbsmo3e0woN7oKRO_EbpDTpxuMN5PVyrCuBk9KEuw@mail.gmail.com>
 <5F8AAC04F9616747BC4CC0E803D5907D0C902274@MLBXv04.nih.gov>
Message-ID: <1402865249.4438.11.camel@nomad.lan>

On Fri, 2014-06-13 at 16:21 +0000, Lavrentiev, Anton (NIH/NLM/NCBI) [C]
wrote:
> >  Could you provide a gnutls trace (use GNUTLS_DEBUG_LEVEL=6 or more),
> 
> With GNUTLS 3.2.13, here's the point of failure (SSOCK#1000 is our callback's output that shows that it
> could write only 528 bytes -- GNUTLS was pushing 1053).  After that the control gets back from
> gnutls_record_send() with a GNUTLS_E_EAGAIN return code rather than the callback is tried one
> more time (it's how send() should have been used).

>From that description I think that this is pretty much expected. A call
to gnutls_record_send() can be interrupted, and had to be called again
(I believe that was the case in all gnutls versions). If that's not
clear from the documentation please let me know what could be improved.

>   Note that we're using non-blocking output
> in the callback.  So when the device is full, it won't accept any data at all and won't block,
> so the callback does a short wait, then tries again and returns immediately if anything at all
> has been written successfully.

That sounds a pretty typical use case. The way to make that work in the
old and new gnutls, would be to call gnutls_record_send() until it
doesn't return GNUTLS_E_AGAIN or GNUTLS_E_INTERRUPTED, with the same
parameters as before (or null). 

> mini-eagain.c is not a good test, IMO.

It's purpose is to test whether interruption of gnutls calls works as
expected, but had proved to be too limited, and thus the
tests/suite/ecore was added, which was a check more to real world. I
haven't checked these tests for years though. If you think that there is
something missing from being checked don't hesitate to enhance the
suite.

regards,
Nikos




From lavr at ncbi.nlm.nih.gov  Mon Jun 16 16:54:01 2014
From: lavr at ncbi.nlm.nih.gov (Lavrentiev, Anton (NIH/NLM/NCBI) [C])
Date: Mon, 16 Jun 2014 14:54:01 +0000
Subject: [gnutls-help] Regression bug between 2.x and 3.2?
In-Reply-To: <1402865249.4438.11.camel@nomad.lan>
References: <5F8AAC04F9616747BC4CC0E803D5907D0C9012D1@MLBXv04.nih.gov>
 <CAJU7zaKXmSbsmo3e0woN7oKRO_EbpDTpxuMN5PVyrCuBk9KEuw@mail.gmail.com>
 <5F8AAC04F9616747BC4CC0E803D5907D0C902274@MLBXv04.nih.gov>
 <1402865249.4438.11.camel@nomad.lan>
Message-ID: <5F8AAC04F9616747BC4CC0E803D5907D0C909914@MLBXv04.nih.gov>

> From that description I think that this is pretty much expected. A call
> to gnutls_record_send() can be interrupted, and had to be called again
> (I believe that was the case in all gnutls versions). If that's not
> clear from the documentation please let me know what could be improved.

Here's the problem:  there was no interruption, there was a short write.
Previous version of GNUTLS tolerated that by calling the push callback
once again, and again, and again, giving up only when nothing _at all_
was written (-1 returned).  Current version bails out immediately.
This is a change in behavior, which is not backward compatible.
The code in gnutls_buffers.c has changed significantly:  it was presumably
necessary to accommodate a vector write operation (writev), and could have
resulted in the inadvertent change for the plain push.

Anyhow, the push callback is documented to be a send()-like thing.
Which means it is allowed to write fewer bytes than it was requested to,
and that does not constitute an error.  Previous GNUTLS version treated
that exactly so, by re-trying the write until unsuccessful (and advancing
with writes, most of the time).  Current implementation considers the
short write as a fault and returns EAGAIN.  Thus, the callback is
longer compatible with send():  if the callback wants GNUTLS to
keep writing it must ensure to push as many bytes as it was told to,
which means, it must be doing send()/wait()/send() sequences internally.

P.S.  I'm preparing a test case which should demonstrate the changed behavior.

Anton Lavrentiev
Contractor NIH/NLM/NCBI


From nmav at gnutls.org  Mon Jun 16 18:31:56 2014
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Mon, 16 Jun 2014 18:31:56 +0200
Subject: [gnutls-help] Regression bug between 2.x and 3.2?
In-Reply-To: <5F8AAC04F9616747BC4CC0E803D5907D0C909914@MLBXv04.nih.gov>
References: <5F8AAC04F9616747BC4CC0E803D5907D0C9012D1@MLBXv04.nih.gov>
 <CAJU7zaKXmSbsmo3e0woN7oKRO_EbpDTpxuMN5PVyrCuBk9KEuw@mail.gmail.com>
 <5F8AAC04F9616747BC4CC0E803D5907D0C902274@MLBXv04.nih.gov>
 <1402865249.4438.11.camel@nomad.lan>
 <5F8AAC04F9616747BC4CC0E803D5907D0C909914@MLBXv04.nih.gov>
Message-ID: <1402936316.4149.6.camel@nomad.lan>

On Mon, 2014-06-16 at 14:54 +0000, Lavrentiev, Anton (NIH/NLM/NCBI) [C]
wrote:
> > From that description I think that this is pretty much expected. A call
> > to gnutls_record_send() can be interrupted, and had to be called again
> > (I believe that was the case in all gnutls versions). If that's not
> > clear from the documentation please let me know what could be improved.
> 
> Here's the problem:  there was no interruption, there was a short write.
> Previous version of GNUTLS tolerated that by calling the push callback
> once again, and again, and again, giving up only when nothing _at all_
> was written (-1 returned).  Current version bails out immediately.
> This is a change in behavior, which is not backward compatible.
> The code in gnutls_buffers.c has changed significantly:  it was presumably
> necessary to accommodate a vector write operation (writev), and could have
> resulted in the inadvertent change for the plain push.

Hello Anton,
 Indeed. However, note that this change is backwards compatible. The
push function doesn't need to change at all. Whether gnutls calls it
once or twice on short reads is an internal behavior of gnutls, you
shouldn't have depended on.

What is the only requirement of the gnutls API is that the caller of
gnutls_record_send(), retries the call if interrupted with
GNUTLS_E_AGAIN or GNUTLS_E_INTERRUPTED, that was always the case. If
that is followed it wouldn't matter whether gnutls would retry the push
function or not. Does making that change address your issue?

> Anyhow, the push callback is documented to be a send()-like thing.
> Which means it is allowed to write fewer bytes than it was requested to,
> and that does not constitute an error.  Previous GNUTLS version treated
> that exactly so, by re-trying the write until unsuccessful (and advancing
> with writes, most of the time). 

Yes, that's the only requirement of the push callback. However, the way
gnutls will call it, is an internal matter that may change at any time
(it's not part of the API nor described anywhere).

regards,
Nikos




From lavr at ncbi.nlm.nih.gov  Mon Jun 16 19:38:32 2014
From: lavr at ncbi.nlm.nih.gov (Lavrentiev, Anton (NIH/NLM/NCBI) [C])
Date: Mon, 16 Jun 2014 17:38:32 +0000
Subject: [gnutls-help] Regression bug between 2.x and 3.2?
Message-ID: <5F8AAC04F9616747BC4CC0E803D5907D0C90A9A8@MLBXv04.nih.gov>

Hello Nikos,

> Does making that change address your issue?

It's not that I cannot reissue gnutls_record_send() upon EAGAIN, it's that I can no longer distinguish
two different write faults that I was able to, formerly:  whether send() failed to complete because of
the internal device timeout or just because send() was not able to write everything at once.

Our push callback is an extended send()-like call, with only a simple addition:  if the socket is
not ready to accept data, it does block only for a certain amount of time before failing with EAGAIN
(if socket becomes ready sooner, it will do send() and return).  If a blocking send() was used, it would
have blocked indefinitely for the socket to become ready.  Non-blocking send() would've returned immediately
with EAGAIN.  So our callback is half-blocking send(), so that the waited time can be controlled.

Now compare:

OLD (2.4.2 in our case):

gnutls_record_send():
  push():
    send() > 0 ? return many sent
    else wait for ready or timeout:
      ready ? send() and return how many sent
      timeout ? return -1 (EAGAIN)
  nothing written ? bail out with an error (including EAGAIN)
  not all written ? repeat push()
  else return success
error == EAGAN ? timeout : some other error

NEW (3.2.13 for us):

gnutls_record_send():
   push():
     send() > 0 ? return how many sent
     else wait for ready or timeout:
       ready ? send() and return how many sent
       timeout ? return -1 (EAGAIN)
   not all written ? bail out with EAGAIN
   error ? bail out with an error (including EAGAIN)
error == EAGAIN ???

So the outer code can no longer figure out what the problem has been (there could not have been any, actually).
The push callback cannot set up any other errnos, because only EAGAIN/EINTR are treated specially, with
all others being de-facto fatal.

The new flow control has merged the two formerly distinct states into one.  There is no way to distinguish
from a time-out situation without having to re-write the callback the way, which is not send()-like.

The new shortcoming is not very logical -- there is a very little cost to add another pass when the
push callback cannot send everything, instead of complete stack unwind up to the point of
gnutls_record_send(), then re-doing the entire sequence.  To work around the timeout issue what we're
having to distinguish in particular, we will have to track this down all the way with an additional
context (and even more CPU cycles), which was not necessary before.

Anton Lavrentiev
Contractor NIH/NLM/NCBI




From nmav at gnutls.org  Tue Jun 17 11:06:44 2014
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Tue, 17 Jun 2014 11:06:44 +0200
Subject: [gnutls-help] Regression bug between 2.x and 3.2?
In-Reply-To: <5F8AAC04F9616747BC4CC0E803D5907D0C90A9A8@MLBXv04.nih.gov>
References: <5F8AAC04F9616747BC4CC0E803D5907D0C90A9A8@MLBXv04.nih.gov>
Message-ID: <CAJU7za+2W=d9f4nLB1T9qkwDmsssAo5SCvRqnH9Vq0b+Co4bfg@mail.gmail.com>

On Mon, Jun 16, 2014 at 7:38 PM, Lavrentiev, Anton (NIH/NLM/NCBI) [C]
<lavr at ncbi.nlm.nih.gov> wrote:
> Hello Nikos,
>> Does making that change address your issue?
> It's not that I cannot reissue gnutls_record_send() upon EAGAIN, it's that I can no longer distinguish
> two different write faults that I was able to, formerly:  whether send() failed to complete because of
> the internal device timeout or just because send() was not able to write everything at once.
[...]
> So the outer code can no longer figure out what the problem has been (there could not have been any, actually).
> The push callback cannot set up any other errnos, because only EAGAIN/EINTR are treated specially, with
> all others being de-facto fatal.

I see. So for you it is mostly the issue that GNUTLS_E_AGAIN doesn't
always map to EAGAIN. My suggestions would be to communicate the
failure issue using the transport_ptr (e.g., by making it a
structure).  Modifying gnutls to emulate the previous behavior I don't
think would help you either, as there are more cases where
GNUTLS_E_AGAIN doesn't directly map to EAGAIN errno.

regards,
Nikos


From lavr at ncbi.nlm.nih.gov  Tue Jun 17 15:45:39 2014
From: lavr at ncbi.nlm.nih.gov (Lavrentiev, Anton (NIH/NLM/NCBI) [C])
Date: Tue, 17 Jun 2014 13:45:39 +0000
Subject: [gnutls-help] Regression bug between 2.x and 3.2?
In-Reply-To: <CAJU7za+2W=d9f4nLB1T9qkwDmsssAo5SCvRqnH9Vq0b+Co4bfg@mail.gmail.com>
References: <5F8AAC04F9616747BC4CC0E803D5907D0C90A9A8@MLBXv04.nih.gov>
 <CAJU7za+2W=d9f4nLB1T9qkwDmsssAo5SCvRqnH9Vq0b+Co4bfg@mail.gmail.com>
Message-ID: <5F8AAC04F9616747BC4CC0E803D5907D0C90AC00@MLBXv04.nih.gov>

> as there are more cases where GNUTLS_E_AGAIN doesn't directly map to EAGAIN errno.

Studying the current GNUTLS source, it looks like the added case is for exactly short writes,
which errno_to_gerr() supports.  The other place is for short read, but since the pull
callback is wrapped into the "while (left > 0)" loop (gnutls_stream_read), it is a legitimate
case, which also maps correctly.  I still can't understand why the similar loop "while (left > 0)"
was dropped off gnutls_io_write_flush (formerly part of gnutls_io_write_buffered) between 2.x and 3.x.

Anyhow, I'm going to modify our push callback behavior to do a tight loop around the former
push callback.  Doing so I notice that reading and writing are no longer symmetrical:
the pull callback does not require the looping on short reads as it will be done by GNUTLS,
and the push callback requires the looping because GNUTLS now bails out on short writes.

So for the sake of performance, the push callback should not return prematurely in order to
save on a long chain of returns with the only purpose to be followed by just a repeat of
gnutls_record_send(), which thus, can be avoided altogether...

Anton Lavrentiev
Contractor NIH/NLM/NCBI



From nmav at gnutls.org  Wed Jun 18 10:11:39 2014
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Wed, 18 Jun 2014 10:11:39 +0200
Subject: [gnutls-help] Regression bug between 2.x and 3.2?
In-Reply-To: <5F8AAC04F9616747BC4CC0E803D5907D0C90AC00@MLBXv04.nih.gov>
References: <5F8AAC04F9616747BC4CC0E803D5907D0C90A9A8@MLBXv04.nih.gov>
 <CAJU7za+2W=d9f4nLB1T9qkwDmsssAo5SCvRqnH9Vq0b+Co4bfg@mail.gmail.com>
 <5F8AAC04F9616747BC4CC0E803D5907D0C90AC00@MLBXv04.nih.gov>
Message-ID: <CAJU7zaLWK+VW2XgZS4vE5X-XRWozqdcYy9bC=TzfQcey_SnyNQ@mail.gmail.com>

On Tue, Jun 17, 2014 at 3:45 PM, Lavrentiev, Anton (NIH/NLM/NCBI) [C]
<lavr at ncbi.nlm.nih.gov> wrote:
>> as there are more cases where GNUTLS_E_AGAIN doesn't directly map to EAGAIN errno.
> Studying the current GNUTLS source, it looks like the added case is for exactly short writes,
> which errno_to_gerr() supports.  The other place is for short read, but since the pull
> callback is wrapped into the "while (left > 0)" loop (gnutls_stream_read), it is a legitimate
> case, which also maps correctly.  I still can't understand why the similar loop "while (left > 0)"
> was dropped off gnutls_io_write_flush (formerly part of gnutls_io_write_buffered) between 2.x and 3.x.
> Anyhow, I'm going to modify our push callback behavior to do a tight loop around the former
> push callback.  Doing so I notice that reading and writing are no longer symmetrical:
> the pull callback does not require the looping on short reads as it will be done by GNUTLS,
> and the push callback requires the looping because GNUTLS now bails out on short writes.

I believe the reason was, that if there was a short send(), that is
equivalent to try again (EAGAIN), or the underlying layer would have
to block to send the rest. Thus it looked more efficient to return
GNUTLS_E_AGAIN rather than retrying, and making immediately after the
short send() a new system call that would fail with EAGAIN.

regards,
Nikos


From Erick.Euzebio at prudential.com  Wed Jun 18 16:03:57 2014
From: Erick.Euzebio at prudential.com (Erick.Euzebio at prudential.com)
Date: Wed, 18 Jun 2014 11:03:57 -0300
Subject: [gnutls-help] GnuTLS - Compress File RPM - Where is it ?
Message-ID: <OFCADC7534.8B4B1D16-ON83257CFB.004CD323-83257CFB.004D4484@prudential.com>

Hello Srs,

Could you help me please ? We are updating GnuTLS in our company, but when 
we were checking the compress file, we didn't find RPM, how can we do 
download of RPM packet ?

Follow the update that we wanna to do:

http://article.gmane.org/gmane.comp.encryption.gpg.gnutls.devel/7341

Thanks in Advance.
_____________________________________________________
Att,
Erick Euz?bio
Analista de Seguran?a da Informa??o
Prudential do Brasil Seguros de Vida S/A 
TEL: 55 21 3088-2065 
EMAIL: erick.euzebio at prudential.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/attachments/20140618/25ed4fa4/attachment.html>

From sandeepdas.cse at gmail.com  Thu Jun 19 07:26:39 2014
From: sandeepdas.cse at gmail.com (Sandeep Kumar)
Date: Thu, 19 Jun 2014 10:56:39 +0530
Subject: [gnutls-help] DTLS Handshake not taking place as expected.
Message-ID: <CAPtPDjbkc-ih6r4vU4ro5J6AevA=mJk87G-a2Gtud+yYmX5hCw@mail.gmail.com>

Hi,

I'm trying to simulate a DTLS client and server communication over SCTP as
transport. I've used sample code provided by GNUTLS and modified it a
little bit to achieve my desired result. I was referring to "RFC 4347
Figure 1. Message flights for full handshake" for verifying the Handshake
procedure whether its taking place properly or not.

I've used following link "https://help.ubuntu.com/community/GnuTLS" to
create secret keys and certificate for server and i did not create
certificate for client as its optional part in handshake. The initial steps
are taking place properly except the part that i'm not able to see
"Finished" message from either of the side. So i believe its not completed
without Finished message. I might be wrong but please verify.

There is one message which is "encrypted alert". I'm not able to understand
its role in communication. What is requirement of sending this message
after sending encrypted data or any control message?

I'm sharing the code which i'm using to simulate this scenario. In addition
to that i'm also sharing the wireshark trace which i captured while running
this simulation. In case you want secret keys and certificate then please
let me know. You'll need to use decode as option to view the DTLS packets
otherwise they will appear as m3ua packets in the trace which i've shared.

GNUTLS Version : 3.2.15
Nettle Version: 2.7.1
GMP Version: 5.1.1-2
OS: Fedora 19
Kernel: 3.14.4-100.fc19.x86_64

Thanks,
Sandeep
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/attachments/20140619/3616711c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dtls_sctp_client.c
Type: text/x-csrc
Size: 5055 bytes
Desc: not available
URL: </pipermail/attachments/20140619/3616711c/attachment-0002.c>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Makefile
Type: application/octet-stream
Size: 439 bytes
Desc: not available
URL: </pipermail/attachments/20140619/3616711c/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stream_sctp_server.c
Type: text/x-csrc
Size: 14980 bytes
Desc: not available
URL: </pipermail/attachments/20140619/3616711c/attachment-0003.c>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: GNUTLS-STREAMBASED-HANDSHAKE.pcap
Type: application/vnd.tcpdump.pcap
Size: 6132 bytes
Desc: not available
URL: </pipermail/attachments/20140619/3616711c/attachment-0001.pcap>

From nmav at gnutls.org  Thu Jun 19 09:28:11 2014
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Thu, 19 Jun 2014 09:28:11 +0200
Subject: [gnutls-help] DTLS Handshake not taking place as expected.
In-Reply-To: <CAPtPDjbkc-ih6r4vU4ro5J6AevA=mJk87G-a2Gtud+yYmX5hCw@mail.gmail.com>
References: <CAPtPDjbkc-ih6r4vU4ro5J6AevA=mJk87G-a2Gtud+yYmX5hCw@mail.gmail.com>
Message-ID: <1403162891.5500.3.camel@nomad.lan>

On Thu, 2014-06-19 at 10:56 +0530, Sandeep Kumar wrote:
> Hi,
> I'm trying to simulate a DTLS client and server communication over
> SCTP as transport. I've used sample code provided by GNUTLS and
> modified it a little bit to achieve my desired result. I was referring
> to "RFC 4347 Figure 1. Message flights for full handshake" for
> verifying the Handshake procedure whether its taking place properly or
> not. 
> [...]
> There is one message which is "encrypted alert". I'm not able to
> understand its role in communication. What is requirement of sending
> this message after sending encrypted data or any control message? 

Hello,
 The Finished messages are the messages mentioned as "Encrypted
Handshake Message", as finished is only sent encrypted. The encrypted
alert is probably the session termination alert since you call
gnutls_bye(). Except for a three times sent client hello, everything
seems fine.

regards,
Nikos





From sandeepdas.cse at gmail.com  Mon Jun 23 09:14:11 2014
From: sandeepdas.cse at gmail.com (Sandeep Kumar)
Date: Mon, 23 Jun 2014 12:44:11 +0530
Subject: [gnutls-help] DTLS Handshake between server and client.
Message-ID: <CAPtPDjYONZF8erVMjgPC1kQ7y1aBV0bhNB6mOEFuSZQB6A6o3g@mail.gmail.com>

Hi,

I've implemented a test program for server and client using the existing
example of gnu-tls. This program emulates DTLS handshake over SCTP.
There are several messages starting from client hello then hello verify
request etc.
All i want to know is that whether is it mandatory for server to verify the
cookie for DTLS because if its case of SCTP the same is already done while
complete SCTP Handshake.
The example which i've is originally derived from DTLS over UDP and hence
that part of code is inherited from there itself.
If i comment the cookie verify part of DTLS then gnutls_handshake(session);
of server does not entertain Client Hello request at all.
Is it because prestate not being set properly?
PFA the code of both client and server for your reference. I'm more
concerned about code starting from line 238 till 286 in
stream_sctp_server.c.
How to take out this piece of code because that's an optional part in
standard DTLS handshake.



Regards,
Sandeep
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/attachments/20140623/60fb5ee8/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dtls_sctp_client.c
Type: text/x-csrc
Size: 5035 bytes
Desc: not available
URL: </pipermail/attachments/20140623/60fb5ee8/attachment-0002.c>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stream_sctp_server.c
Type: text/x-csrc
Size: 16021 bytes
Desc: not available
URL: </pipermail/attachments/20140623/60fb5ee8/attachment-0003.c>

From apoikos at debian.org  Mon Jun 23 14:14:00 2014
From: apoikos at debian.org (Apollon Oikonomopoulos)
Date: Mon, 23 Jun 2014 15:14:00 +0300
Subject: [gnutls-help] Issues with libcurl and GnuTLS 3.2
Message-ID: <20140623121359.GA14827@marvin.ws.skroutz.gr>

Hi,

(Please Cc me on reply, I'm not subscribed to the list. Thanks.)

I'm trying to debug a (rather painful) issue that apparently lies either in
libcurl, or GnuTLS 3.2. It all started out with Debian bug #751886 [1],
where the software at hand (Ganeti) uses Haskell's FFI to call libcurl, which
in Debian is currently linked with GnuTLS 3.2.

The luxid daemon initiates RPC calls over an HTTPS transport, using a
certificate effectively as a shared secret, as both client and server cert.
Every once in a while, during such an RPC call, luxid will either segfault, or
receive a TLS decrypt error alert from the server (which makes me think that
somewhere something is getting corrupted during the handshake).

I managed to obtain a few meaningful backtraces from the segfaulting instances,
all of them consistently happening during the handshake and all of them 
with the same call trace like the one at the bottom of this message. Also,
debug output from a corrupted and a crashed handshake follow.

A few more bits that might be of some value:

 - With the same version of libcurl (7.37.0) linked against gnutls 2.12, luxid
   works reliably. Actually the bug appeared when Debian switched to a gnutls
   3.2-linked version of libcurl. I tried out different combinations of 
   libcurl and GnuTLS versions, the only failing combinations were with 
   GnuTLS 3.2. 

 - The binary does not use the threaded Haskell runtime, so it's
   single-threaded.

 - Luxid uses the non-blocking curl multi interface to multiplex RPC 
   calls to multiple nodes (but it's failing with even a single node).

 - When the other end (Python + OpenSSL) sends the decrypt error alert, 
   it reports:
   HttpError: Error in SSL handshake: ([('rsa routines', 'INT_RSA_VERIFY', 'wrong signature length'), ('SSL routines', 'SSL3_GET_CERT_VERIFY', 'bad signature')],)

Does anyone have even the slightest idea what might be going on, or at least
what else to look at?

[1] https://bugs.debian.org/751886

Regards,
Apollon

--- GnuTLS debug output from a failed handshake

gnutls[2]: Enabled GnuTLS logging...
gnutls[2]: Intel SSSE3 was detected
gnutls[2]: p11: loaded provider 'p11-kit-trust'
gnutls[2]: p11: loaded provider 'gnome-keyring'
gnutls[2]: ASSERT: pkcs11.c:431
gnutls[4]: REC[0x2f19ed0]: Allocating epoch #0
gnutls[2]: ASSERT: x509_b64.c:299
gnutls[2]: Could not find '-----BEGIN RSA PRIVATE KEY'
gnutls[2]: ASSERT: x509_b64.c:299
gnutls[2]: Could not find '-----BEGIN DSA PRIVATE KEY'
gnutls[2]: ASSERT: x509_b64.c:299
gnutls[2]: Could not find '-----BEGIN EC PRIVATE KEY'
gnutls[2]: ASSERT: privkey.c:481
gnutls[2]: Falling back to PKCS #8 key decoding
gnutls[2]: ASSERT: gnutls_constate.c:583
gnutls[4]: REC[0x2f19ed0]: Allocating epoch #1
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_ECDSA_AES_128_GCM_SHA256 (C0.2B)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_ECDSA_AES_256_GCM_SHA384 (C0.2C)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_ECDSA_CAMELLIA_128_GCM_SHA256 (C0.86)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_ECDSA_CAMELLIA_256_GCM_SHA384 (C0.87)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_ECDSA_AES_128_CBC_SHA1 (C0.09)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_ECDSA_AES_128_CBC_SHA256 (C0.23)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_ECDSA_AES_256_CBC_SHA1 (C0.0A)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_ECDSA_AES_256_CBC_SHA384 (C0.24)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_ECDSA_CAMELLIA_128_CBC_SHA256 (C0.72)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_ECDSA_CAMELLIA_256_CBC_SHA384 (C0.73)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_ECDSA_3DES_EDE_CBC_SHA1 (C0.08)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_RSA_AES_128_GCM_SHA256 (C0.2F)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_RSA_AES_256_GCM_SHA384 (C0.30)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_RSA_CAMELLIA_128_GCM_SHA256 (C0.8A)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_RSA_CAMELLIA_256_GCM_SHA384 (C0.8B)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_RSA_AES_128_CBC_SHA1 (C0.13)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_RSA_AES_128_CBC_SHA256 (C0.27)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_RSA_AES_256_CBC_SHA1 (C0.14)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_RSA_AES_256_CBC_SHA384 (C0.28)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_RSA_CAMELLIA_128_CBC_SHA256 (C0.76)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_RSA_CAMELLIA_256_CBC_SHA384 (C0.77)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: ECDHE_RSA_3DES_EDE_CBC_SHA1 (C0.12)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_AES_128_GCM_SHA256 (00.9C)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_AES_256_GCM_SHA384 (00.9D)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_CAMELLIA_128_GCM_SHA256 (C0.7A)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_CAMELLIA_256_GCM_SHA384 (C0.7B)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_AES_128_CBC_SHA1 (00.2F)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_AES_128_CBC_SHA256 (00.3C)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_AES_256_CBC_SHA1 (00.35)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_AES_256_CBC_SHA256 (00.3D)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_CAMELLIA_128_CBC_SHA1 (00.41)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_CAMELLIA_128_CBC_SHA256 (00.BA)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_CAMELLIA_256_CBC_SHA1 (00.84)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_CAMELLIA_256_CBC_SHA256 (00.C0)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: RSA_3DES_EDE_CBC_SHA1 (00.0A)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_AES_128_GCM_SHA256 (00.9E)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_AES_256_GCM_SHA384 (00.9F)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_CAMELLIA_128_GCM_SHA256 (C0.7C)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_CAMELLIA_256_GCM_SHA384 (C0.7D)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_AES_128_CBC_SHA1 (00.33)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_AES_128_CBC_SHA256 (00.67)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_AES_256_CBC_SHA1 (00.39)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_AES_256_CBC_SHA256 (00.6B)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_CAMELLIA_128_CBC_SHA1 (00.45)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_CAMELLIA_128_CBC_SHA256 (00.BE)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_CAMELLIA_256_CBC_SHA1 (00.88)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_CAMELLIA_256_CBC_SHA256 (00.C4)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_RSA_3DES_EDE_CBC_SHA1 (00.16)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_AES_128_GCM_SHA256 (00.A2)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_AES_256_GCM_SHA384 (00.A3)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_CAMELLIA_128_GCM_SHA256 (C0.80)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_CAMELLIA_256_GCM_SHA384 (C0.81)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_AES_128_CBC_SHA1 (00.32)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_AES_128_CBC_SHA256 (00.40)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_AES_256_CBC_SHA1 (00.38)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_AES_256_CBC_SHA256 (00.6A)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_CAMELLIA_128_CBC_SHA1 (00.44)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_CAMELLIA_128_CBC_SHA256 (00.BD)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_CAMELLIA_256_CBC_SHA1 (00.87)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_CAMELLIA_256_CBC_SHA256 (00.C3)
gnutls[3]: HSK[0x2f19ed0]: Keeping ciphersuite: DHE_DSS_3DES_EDE_CBC_SHA1 (00.13)
gnutls[3]: EXT[0x2f19ed0]: Sending extension STATUS REQUEST (5 bytes)
gnutls[3]: EXT[0x2f19ed0]: Sending extension SAFE RENEGOTIATION (1 bytes)
gnutls[3]: EXT[0x2f19ed0]: Sending extension SESSION TICKET (0 bytes)
gnutls[3]: EXT[0x2f19ed0]: Sending extension SUPPORTED ECC (12 bytes)
gnutls[3]: EXT[0x2f19ed0]: Sending extension SUPPORTED ECC POINT FORMATS (2 bytes)
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (4.1) RSA-SHA256
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (4.2) DSA-SHA256
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (4.3) ECDSA-SHA256
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (5.1) RSA-SHA384
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (5.3) ECDSA-SHA384
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (6.1) RSA-SHA512
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (6.3) ECDSA-SHA512
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (3.1) RSA-SHA224
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (3.2) DSA-SHA224
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (3.3) ECDSA-SHA224
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (2.1) RSA-SHA1
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (2.2) DSA-SHA1
gnutls[3]: EXT[0x2f19ed0]: sent signature algo (2.3) ECDSA-SHA1
gnutls[3]: EXT[0x2f19ed0]: Sending extension SIGNATURE ALGORITHMS (28 bytes)
gnutls[3]: HSK[0x2f19ed0]: CLIENT HELLO was queued [239 bytes]
gnutls[4]: REC[0x2f19ed0]: Preparing Packet Handshake(22) with length: 239 and min pad: 0
gnutls[9]: ENC[0x2f19ed0]: cipher: NULL, MAC: MAC-NULL, Epoch: 0
gnutls[4]: REC[0x2f19ed0]: Sent Packet[1] Handshake(22) in epoch 0 and length: 244
gnutls[2]: ASSERT: gnutls_buffers.c:1075
gnutls[2]: ASSERT: gnutls_buffers.c:518
2014-06-23 15:01:50,103210000000 EEST: ganeti-luxid pid=31318 DEBUG mcode: CurlmOK, remaining: 1
gnutls[2]: ASSERT: gnutls_buffers.c:1075
gnutls[4]: REC[0x2f19ed0]: SSL 3.3 Handshake packet received. Epoch 0, length: 53
gnutls[4]: REC[0x2f19ed0]: Expected Packet Handshake(22)
gnutls[4]: REC[0x2f19ed0]: Received Packet Handshake(22) with length: 53
gnutls[4]: REC[0x2f19ed0]: Decrypted Packet[0] Handshake(22) with length: 53
gnutls[3]: HSK[0x2f19ed0]: SERVER HELLO (2) was received. Length 49[49], frag offset 0, frag length: 49, sequence: 0
gnutls[3]: HSK[0x2f19ed0]: Server's version: 3.3
gnutls[3]: HSK[0x2f19ed0]: SessionID length: 0
gnutls[3]: HSK[0x2f19ed0]: SessionID: 00
gnutls[3]: HSK[0x2f19ed0]: Selected cipher suite: RSA_AES_128_GCM_SHA256
gnutls[3]: HSK[0x2f19ed0]: Selected compression method: NULL (0)
gnutls[3]: EXT[0x2f19ed0]: Parsing extension 'SAFE RENEGOTIATION/65281' (1 bytes)
gnutls[3]: EXT[0x2f19ed0]: Parsing extension 'SESSION TICKET/35' (0 bytes)
gnutls[3]: HSK[0x2f19ed0]: Safe renegotiation succeeded
gnutls[2]: ASSERT: gnutls_buffers.c:1075
gnutls[4]: REC[0x2f19ed0]: SSL 3.3 Handshake packet received. Epoch 0, length: 700
gnutls[4]: REC[0x2f19ed0]: Expected Packet Handshake(22)
gnutls[4]: REC[0x2f19ed0]: Received Packet Handshake(22) with length: 700
gnutls[4]: REC[0x2f19ed0]: Decrypted Packet[1] Handshake(22) with length: 700
gnutls[3]: HSK[0x2f19ed0]: CERTIFICATE (11) was received. Length 696[696], frag offset 0, frag length: 696, sequence: 0
gnutls[2]: ASSERT: gnutls_buffers.c:1075
gnutls[4]: REC[0x2f19ed0]: SSL 3.3 Handshake packet received. Epoch 0, length: 79
gnutls[4]: REC[0x2f19ed0]: Expected Packet Handshake(22)
gnutls[4]: REC[0x2f19ed0]: Received Packet Handshake(22) with length: 79
gnutls[4]: REC[0x2f19ed0]: Decrypted Packet[2] Handshake(22) with length: 79
gnutls[3]: HSK[0x2f19ed0]: CERTIFICATE REQUEST (13) was received. Length 71[75], frag offset 0, frag length: 71, sequence: 0
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (6.1) RSA-SHA512
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (6.2) (null)
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (6.3) ECDSA-SHA512
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (5.1) RSA-SHA384
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (5.2) (null)
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (5.3) ECDSA-SHA384
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (4.1) RSA-SHA256
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (4.2) DSA-SHA256
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (4.3) ECDSA-SHA256
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (3.1) RSA-SHA224
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (3.2) DSA-SHA224
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (3.3) ECDSA-SHA224
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (2.1) RSA-SHA1
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (2.2) DSA-SHA1
gnutls[3]: EXT[0x2f19ed0]: rcvd signature algo (2.3) ECDSA-SHA1
gnutls[2]: ASSERT: gnutls_buffers.c:1075
gnutls[3]: HSK[0x2f19ed0]: SERVER HELLO DONE (14) was received. Length 0[0], frag offset 0, frag length: 1, sequence: 0
gnutls[2]: ASSERT: gnutls_buffers.c:1310
gnutls[3]: HSK[0x2f19ed0]: CERTIFICATE was queued [700 bytes]
gnutls[3]: HSK[0x2f19ed0]: CLIENT KEY EXCHANGE was queued [262 bytes]
gnutls[2]: sign handshake cert vrfy: picked RSA-SHA512 with SHA512
gnutls[3]: HSK[0x2f19ed0]: CERTIFICATE VERIFY was queued [258 bytes]
gnutls[3]: REC[0x2f19ed0]: Sent ChangeCipherSpec
gnutls[9]: INT: PREMASTER SECRET[48]: 03034a8de47664248edbf8d5834367edc2736eba2f0194e6d74966656cdde8174113985ef1333a90b40bd910c514ee44
gnutls[9]: INT: CLIENT RANDOM[32]: 53a8175f38f6896910b842c4f694a36d97978e50aaf91b82036fd9a8ffcdd25b
gnutls[9]: INT: SERVER RANDOM[32]: 5863ecbd32afb8b28379e5e3a07210f2d1c98178cb4c283b5d2b3ae74d6494c7
gnutls[9]: INT: MASTER SECRET: 49503ca54bcca26bb5e90126e277d60edf02aea192c831359f89e7c7900ef50b1b0437998974b0116b37f341403fca6a
gnutls[4]: REC[0x2f19ed0]: Initializing epoch #1
gnutls[9]: INT: KEY BLOCK[40]: e39b5b0b698ab3c0b67e3441648ef487fd5d7da3f842e4dd906d2063874d7411
gnutls[9]: INT: CLIENT WRITE KEY [16]: e39b5b0b698ab3c0b67e3441648ef487
gnutls[9]: INT: SERVER WRITE KEY [16]: fd5d7da3f842e4dd906d2063874d7411
gnutls[4]: REC[0x2f19ed0]: Epoch #1 ready
gnutls[3]: HSK[0x2f19ed0]: Cipher Suite: RSA_AES_128_GCM_SHA256
gnutls[3]: HSK[0x2f19ed0]: Initializing internal [write] cipher sessions
gnutls[3]: HSK[0x2f19ed0]: recording tls-unique CB (send)
gnutls[3]: HSK[0x2f19ed0]: FINISHED was queued [16 bytes]
gnutls[4]: REC[0x2f19ed0]: Preparing Packet Handshake(22) with length: 700 and min pad: 0
gnutls[9]: ENC[0x2f19ed0]: cipher: NULL, MAC: MAC-NULL, Epoch: 0
gnutls[4]: REC[0x2f19ed0]: Sent Packet[2] Handshake(22) in epoch 0 and length: 705
gnutls[4]: REC[0x2f19ed0]: Preparing Packet Handshake(22) with length: 262 and min pad: 0
gnutls[9]: ENC[0x2f19ed0]: cipher: NULL, MAC: MAC-NULL, Epoch: 0
gnutls[4]: REC[0x2f19ed0]: Sent Packet[3] Handshake(22) in epoch 0 and length: 267
gnutls[4]: REC[0x2f19ed0]: Preparing Packet Handshake(22) with length: 258 and min pad: 0
gnutls[9]: ENC[0x2f19ed0]: cipher: NULL, MAC: MAC-NULL, Epoch: 0
gnutls[4]: REC[0x2f19ed0]: Sent Packet[4] Handshake(22) in epoch 0 and length: 263
gnutls[4]: REC[0x2f19ed0]: Preparing Packet ChangeCipherSpec(20) with length: 1 and min pad: 0
gnutls[9]: ENC[0x2f19ed0]: cipher: NULL, MAC: MAC-NULL, Epoch: 0
gnutls[4]: REC[0x2f19ed0]: Sent Packet[5] ChangeCipherSpec(20) in epoch 0 and length: 6
gnutls[4]: REC[0x2f19ed0]: Preparing Packet Handshake(22) with length: 16 and min pad: 0
gnutls[9]: ENC[0x2f19ed0]: cipher: AES-128-GCM, MAC: AEAD, Epoch: 1
gnutls[4]: REC[0x2f19ed0]: Sent Packet[1] Handshake(22) in epoch 1 and length: 45
gnutls[2]: ASSERT: gnutls_buffers.c:1075
gnutls[2]: ASSERT: gnutls_buffers.c:518
2014-06-23 15:01:50,120672000000 EEST: ganeti-luxid pid=31318 DEBUG mcode: CurlmOK, remaining: 1
2014-06-23 15:01:50,131738000000 EEST: ganeti-luxid pid=31318 DEBUG mcode: CurlmOK, remaining: 1
2014-06-23 15:01:50,142315000000 EEST: ganeti-luxid pid=31318 DEBUG mcode: CurlmOK, remaining: 1
2014-06-23 15:01:50,153039000000 EEST: ganeti-luxid pid=31318 DEBUG mcode: CurlmOK, remaining: 1
gnutls[2]: ASSERT: gnutls_buffers.c:1075
gnutls[4]: REC[0x2f19ed0]: SSL 3.3 Alert packet received. Epoch 0, length: 2
gnutls[4]: REC[0x2f19ed0]: Expected Packet Handshake(22)
gnutls[4]: REC[0x2f19ed0]: Received Packet Alert(21) with length: 2
gnutls[4]: REC[0x2f19ed0]: Decrypted Packet[3] Alert(21) with length: 2
gnutls[4]: REC[0x2f19ed0]: Alert[2|51] - Decrypt error - was received
gnutls[2]: ASSERT: gnutls_record.c:771
gnutls[2]: ASSERT: gnutls_record.c:778
gnutls[2]: ASSERT: gnutls_record.c:1293
gnutls[2]: ASSERT: gnutls_buffers.c:1326
gnutls[2]: ASSERT: gnutls_handshake.c:1414
gnutls[2]: ASSERT: session_ticket.c:649
gnutls[2]: ASSERT: gnutls_handshake.c:2800
gnutls[2]: ASSERT: gnutls_record.c:342
gnutls[4]: REC[0x2f19ed0]: Start of epoch cleanup
gnutls[4]: REC[0x2f19ed0]: End of epoch cleanup
gnutls[4]: REC[0x2f19ed0]: Epoch #0 freed
gnutls[4]: REC[0x2f19ed0]: Epoch #1 freed

--- GnuTLS debug output from a request that caused a crash

gnutls[4]: REC[0x2f23800]: Allocating epoch #0
gnutls[2]: ASSERT: x509_b64.c:299
gnutls[2]: Could not find '-----BEGIN RSA PRIVATE KEY'
gnutls[2]: ASSERT: x509_b64.c:299
gnutls[2]: Could not find '-----BEGIN DSA PRIVATE KEY'
gnutls[2]: ASSERT: x509_b64.c:299
gnutls[2]: Could not find '-----BEGIN EC PRIVATE KEY'
gnutls[2]: ASSERT: privkey.c:481
gnutls[2]: Falling back to PKCS #8 key decoding
gnutls[2]: ASSERT: gnutls_constate.c:583
gnutls[4]: REC[0x2f23800]: Allocating epoch #1
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_ECDSA_AES_128_GCM_SHA256 (C0.2B)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_ECDSA_AES_256_GCM_SHA384 (C0.2C)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_ECDSA_CAMELLIA_128_GCM_SHA256 (C0.86)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_ECDSA_CAMELLIA_256_GCM_SHA384 (C0.87)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_ECDSA_AES_128_CBC_SHA1 (C0.09)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_ECDSA_AES_128_CBC_SHA256 (C0.23)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_ECDSA_AES_256_CBC_SHA1 (C0.0A)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_ECDSA_AES_256_CBC_SHA384 (C0.24)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_ECDSA_CAMELLIA_128_CBC_SHA256 (C0.72)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_ECDSA_CAMELLIA_256_CBC_SHA384 (C0.73)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_ECDSA_3DES_EDE_CBC_SHA1 (C0.08)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_RSA_AES_128_GCM_SHA256 (C0.2F)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_RSA_AES_256_GCM_SHA384 (C0.30)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_RSA_CAMELLIA_128_GCM_SHA256 (C0.8A)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_RSA_CAMELLIA_256_GCM_SHA384 (C0.8B)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_RSA_AES_128_CBC_SHA1 (C0.13)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_RSA_AES_128_CBC_SHA256 (C0.27)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_RSA_AES_256_CBC_SHA1 (C0.14)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_RSA_AES_256_CBC_SHA384 (C0.28)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_RSA_CAMELLIA_128_CBC_SHA256 (C0.76)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_RSA_CAMELLIA_256_CBC_SHA384 (C0.77)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: ECDHE_RSA_3DES_EDE_CBC_SHA1 (C0.12)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_AES_128_GCM_SHA256 (00.9C)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_AES_256_GCM_SHA384 (00.9D)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_CAMELLIA_128_GCM_SHA256 (C0.7A)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_CAMELLIA_256_GCM_SHA384 (C0.7B)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_AES_128_CBC_SHA1 (00.2F)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_AES_128_CBC_SHA256 (00.3C)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_AES_256_CBC_SHA1 (00.35)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_AES_256_CBC_SHA256 (00.3D)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_CAMELLIA_128_CBC_SHA1 (00.41)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_CAMELLIA_128_CBC_SHA256 (00.BA)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_CAMELLIA_256_CBC_SHA1 (00.84)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_CAMELLIA_256_CBC_SHA256 (00.C0)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: RSA_3DES_EDE_CBC_SHA1 (00.0A)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_AES_128_GCM_SHA256 (00.9E)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_AES_256_GCM_SHA384 (00.9F)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_CAMELLIA_128_GCM_SHA256 (C0.7C)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_CAMELLIA_256_GCM_SHA384 (C0.7D)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_AES_128_CBC_SHA1 (00.33)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_AES_128_CBC_SHA256 (00.67)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_AES_256_CBC_SHA1 (00.39)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_AES_256_CBC_SHA256 (00.6B)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_CAMELLIA_128_CBC_SHA1 (00.45)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_CAMELLIA_128_CBC_SHA256 (00.BE)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_CAMELLIA_256_CBC_SHA1 (00.88)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_CAMELLIA_256_CBC_SHA256 (00.C4)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_RSA_3DES_EDE_CBC_SHA1 (00.16)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_AES_128_GCM_SHA256 (00.A2)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_AES_256_GCM_SHA384 (00.A3)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_CAMELLIA_128_GCM_SHA256 (C0.80)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_CAMELLIA_256_GCM_SHA384 (C0.81)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_AES_128_CBC_SHA1 (00.32)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_AES_128_CBC_SHA256 (00.40)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_AES_256_CBC_SHA1 (00.38)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_AES_256_CBC_SHA256 (00.6A)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_CAMELLIA_128_CBC_SHA1 (00.44)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_CAMELLIA_128_CBC_SHA256 (00.BD)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_CAMELLIA_256_CBC_SHA1 (00.87)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_CAMELLIA_256_CBC_SHA256 (00.C3)
gnutls[3]: HSK[0x2f23800]: Keeping ciphersuite: DHE_DSS_3DES_EDE_CBC_SHA1 (00.13)
gnutls[3]: EXT[0x2f23800]: Sending extension STATUS REQUEST (5 bytes)
gnutls[3]: EXT[0x2f23800]: Sending extension SAFE RENEGOTIATION (1 bytes)
gnutls[3]: EXT[0x2f23800]: Sending extension SESSION TICKET (0 bytes)
gnutls[3]: EXT[0x2f23800]: Sending extension SUPPORTED ECC (12 bytes)
gnutls[3]: EXT[0x2f23800]: Sending extension SUPPORTED ECC POINT FORMATS (2 bytes)
gnutls[3]: EXT[0x2f23800]: sent signature algo (4.1) RSA-SHA256
gnutls[3]: EXT[0x2f23800]: sent signature algo (4.2) DSA-SHA256
gnutls[3]: EXT[0x2f23800]: sent signature algo (4.3) ECDSA-SHA256
gnutls[3]: EXT[0x2f23800]: sent signature algo (5.1) RSA-SHA384
gnutls[3]: EXT[0x2f23800]: sent signature algo (5.3) ECDSA-SHA384
gnutls[3]: EXT[0x2f23800]: sent signature algo (6.1) RSA-SHA512
gnutls[3]: EXT[0x2f23800]: sent signature algo (6.3) ECDSA-SHA512
gnutls[3]: EXT[0x2f23800]: sent signature algo (3.1) RSA-SHA224
gnutls[3]: EXT[0x2f23800]: sent signature algo (3.2) DSA-SHA224
gnutls[3]: EXT[0x2f23800]: sent signature algo (3.3) ECDSA-SHA224
gnutls[3]: EXT[0x2f23800]: sent signature algo (2.1) RSA-SHA1
gnutls[3]: EXT[0x2f23800]: sent signature algo (2.2) DSA-SHA1
gnutls[3]: EXT[0x2f23800]: sent signature algo (2.3) ECDSA-SHA1
gnutls[3]: EXT[0x2f23800]: Sending extension SIGNATURE ALGORITHMS (28 bytes)
gnutls[3]: HSK[0x2f23800]: CLIENT HELLO was queued [239 bytes]
gnutls[4]: REC[0x2f23800]: Preparing Packet Handshake(22) with length: 239 and min pad: 0
gnutls[9]: ENC[0x2f23800]: cipher: NULL, MAC: MAC-NULL, Epoch: 0
gnutls[4]: REC[0x2f23800]: Sent Packet[1] Handshake(22) in epoch 0 and length: 244
gnutls[2]: ASSERT: gnutls_buffers.c:1075
gnutls[2]: ASSERT: gnutls_buffers.c:518
2014-06-23 15:03:30,978512000000 EEST: ganeti-luxid pid=31318 DEBUG mcode: CurlmOK, remaining: 1
gnutls[2]: ASSERT: gnutls_buffers.c:1075
gnutls[4]: REC[0x2f23800]: SSL 3.3 Handshake packet received. Epoch 0, length: 53
gnutls[4]: REC[0x2f23800]: Expected Packet Handshake(22)
gnutls[4]: REC[0x2f23800]: Received Packet Handshake(22) with length: 53
gnutls[4]: REC[0x2f23800]: Decrypted Packet[0] Handshake(22) with length: 53
gnutls[3]: HSK[0x2f23800]: SERVER HELLO (2) was received. Length 49[49], frag offset 0, frag length: 49, sequence: 0
gnutls[3]: HSK[0x2f23800]: Server's version: 3.3
gnutls[3]: HSK[0x2f23800]: SessionID length: 0
gnutls[3]: HSK[0x2f23800]: SessionID: 00
gnutls[3]: HSK[0x2f23800]: Selected cipher suite: RSA_AES_128_GCM_SHA256
gnutls[3]: HSK[0x2f23800]: Selected compression method: NULL (0)
gnutls[3]: EXT[0x2f23800]: Parsing extension 'SAFE RENEGOTIATION/65281' (1 bytes)
gnutls[3]: EXT[0x2f23800]: Parsing extension 'SESSION TICKET/35' (0 bytes)
gnutls[3]: HSK[0x2f23800]: Safe renegotiation succeeded
gnutls[2]: ASSERT: gnutls_buffers.c:1075
gnutls[4]: REC[0x2f23800]: SSL 3.3 Handshake packet received. Epoch 0, length: 700
gnutls[4]: REC[0x2f23800]: Expected Packet Handshake(22)
gnutls[4]: REC[0x2f23800]: Received Packet Handshake(22) with length: 700
gnutls[4]: REC[0x2f23800]: Decrypted Packet[1] Handshake(22) with length: 700
gnutls[3]: HSK[0x2f23800]: CERTIFICATE (11) was received. Length 696[696], frag offset 0, frag length: 696, sequence: 0
gnutls[2]: ASSERT: gnutls_buffers.c:1075
gnutls[4]: REC[0x2f23800]: SSL 3.3 Handshake packet received. Epoch 0, length: 79
gnutls[4]: REC[0x2f23800]: Expected Packet Handshake(22)
gnutls[4]: REC[0x2f23800]: Received Packet Handshake(22) with length: 79
gnutls[4]: REC[0x2f23800]: Decrypted Packet[2] Handshake(22) with length: 79
gnutls[3]: HSK[0x2f23800]: CERTIFICATE REQUEST (13) was received. Length 71[75], frag offset 0, frag length: 71, sequence: 0
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (6.1) RSA-SHA512
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (6.2) (null)
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (6.3) ECDSA-SHA512
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (5.1) RSA-SHA384
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (5.2) (null)
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (5.3) ECDSA-SHA384
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (4.1) RSA-SHA256
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (4.2) DSA-SHA256
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (4.3) ECDSA-SHA256
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (3.1) RSA-SHA224
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (3.2) DSA-SHA224
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (3.3) ECDSA-SHA224
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (2.1) RSA-SHA1
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (2.2) DSA-SHA1
gnutls[3]: EXT[0x2f23800]: rcvd signature algo (2.3) ECDSA-SHA1
gnutls[2]: ASSERT: gnutls_buffers.c:1075
gnutls[3]: HSK[0x2f23800]: SERVER HELLO DONE (14) was received. Length 0[0], frag offset 0, frag length: 1, sequence: 0
gnutls[2]: ASSERT: gnutls_buffers.c:1310
gnutls[3]: HSK[0x2f23800]: CERTIFICATE was queued [700 bytes]
gnutls[3]: HSK[0x2f23800]: CLIENT KEY EXCHANGE was queued [262 bytes]
gnutls[2]: sign handshake cert vrfy: picked RSA-SHA512 with SHA512
<segfault>

--- Backtrace

#0  0x00007fcce89d0002 in __gmpn_powm () from /usr/lib/x86_64-linux-gnu/libgmp.so.10
No symbol table info available.
#1  0x00007fcce8997a8d in __gmpz_powm () from /usr/lib/x86_64-linux-gnu/libgmp.so.10
No symbol table info available.
#2  0x00007fcce59fdbf2 in _nettle_rsa_blind (pub=pub at entry=0x7fff9b21a230, random_ctx=random_ctx at entry=0x0, random=random at entry=0x7fcce73ec5e0 <rnd_func>, c=c at entry=0x7fff9b21a220, ri=ri at entry=0x7fff9b21a1b0) at rsa-blind.c:56
        r = {{_mp_alloc = 33, _mp_size = 32, _mp_d = 0x7fcce470f9b8}}
#3  0x00007fcce59fcc39 in nettle_rsa_pkcs1_sign_tr (pub=pub at entry=0x7fff9b21a230, key=key at entry=0x7fff9b21a260, random_ctx=random_ctx at entry=0x0, random=random at entry=0x7fcce73ec5e0 <rnd_func>, length=<optimized out>, digest_info=<optimized out>, s=s at entry=0x7fff9b21a220) at rsa-pkcs1-sign-tr.c:47
        ri = {{_mp_alloc = 33, _mp_size = 32, _mp_d = 0x7fcce470fad0}}
#4  0x00007fcce73ee681 in _wrap_nettle_pk_sign (algo=<optimized out>, signature=0x7fff9b21a520, vdata=0x7fff9b21a310, pk_params=0x1807cb0) at pk.c:459
        priv = {size = 254, d = {{_mp_alloc = 32, _mp_size = 32, _mp_d = 0x7fcce472f5e8}}, p = {{_mp_alloc = 17, _mp_size = 16, _mp_d = 0x7fcce472f6f8}}, q = {{_mp_alloc = 17, _mp_size = 16, _mp_d = 0x7fcce472f790}}, a = {{_mp_alloc = 16, _mp_size = 16, _mp_d = 0x7fcce472faf8}}, b = {{_mp_alloc = 16, _mp_size = 16, _mp_d = 0x7fcce472fb88}}, c = {{_mp_alloc = 17, _mp_size = 16, _mp_d = 0x7fcce472f828}}}
        pub = {size = 254, n = {{_mp_alloc = 33, _mp_size = 32, _mp_d = 0x7fcce472f4b8}}, e = {{_mp_alloc = 1, _mp_size = 1, _mp_d = 0x7fcce472f5d0}}}
        s = {{_mp_alloc = 32, _mp_size = 32, _mp_d = 0x7fcce470f878}}
        ret = <optimized out>
        hash_len = 32767
        me = <optimized out>
#5  0x00007fcce736905b in gnutls_privkey_sign_hash (signer=signer at entry=0x18077b0, hash_algo=<optimized out>, flags=flags at entry=0, hash_data=hash_data at entry=0x7fff9b21a420, signature=0x7fff9b21a520) at gnutls_privkey.c:801
        ret = 0
        digest = {data = 0x18261c0 "0Q0\r\006\t`\206H\001e\003\004\002\003\005", size = 83}
#6  0x00007fcce735e58c in sign_tls_hash (session=0x1808d70, session at entry=0x8, hash_algo=hash_algo at entry=0x7fcce76297e0 <hash_algorithms+192>, cert=cert at entry=0x1826f40, pkey=pkey at entry=0x18077b0, hash_concat=hash_concat at entry=0x7fff9b21a420, signature=signature at entry=0x7fff9b21a520) at gnutls_sig.c:244
        key_usage = 0
#7  0x00007fcce735ef5e in _gnutls_handshake_sign_crt_vrfy12 (signature=0x7fff9b21a520, pkey=0x18077b0, cert=0x1826f40, session=0x8) at gnutls_sig.c:598
        ret = <optimized out>
        concat = "!\242\200\"\315J\367\346\215FtB\341\070\372\035/\331va%Y\017[u\255\345\000Y\225]\305\361w\337\037\317\313\036}\220y\215\025\326\350\307?\340oq\241t8w\006\241\252\224\350\065u\370\005\000\000\000\000\000\000\000\065\000\000\000\000\000\000\000\000\000"
        sign_algo = GNUTLS_SIGN_RSA_SHA512
        me = 0x7fcce76297e0 <hash_algorithms+192>
        dconcat = {data = 0x7fff9b21a460 "!\242\200\"\315J\367\346\215FtB\341\070\372\035/\331va%Y\017[u\255", <incomplete sequence \345>, size = 64}
#8  _gnutls_handshake_sign_crt_vrfy (session=session at entry=0x1808d70, cert=0x1826f40, pkey=0x18077b0, signature=signature at entry=0x7fff9b21a520) at gnutls_sig.c:635
        dconcat = {data = 0x2 <Address 0x2 out of bounds>, size = 3879062009}
        ret = <optimized out>
        concat = "!\242\200\"\315J\367\346\215FtB\341\070\372\035/\331va%Y\017[u\255\345\000Y\225]\305\361w\337\037\317\313\036}\220y\215\025\326\350\307?\340oq\241t8w\006\241\252\224\350\065u\370\005\000\000\000\000\000\000\000\065\000\000\000\000\000\000\000\000\000"
        td_md5 = {e = 0x1809fb8, hash = 0x100, output = 0x7fff9b21a570, deinit = 0x7fcce7360de6 <_gnutls_buffer_append_data+262>, key = 0x7fff00000000, keysize = 25202032, handle = 0x100}
        td_sha = {e = 0x7fff9b21a460, hash = 0x7fcc00000040, output = 0x9, deinit = 0x2, key = 0x100, keysize = -1692293776, handle = 0x0}
        pk = <optimized out>
#9  0x00007fcce73ca9e9 in _gnutls_gen_cert_client_crt_vrfy (session=0x1808d70, data=0x7fff9b21a570) at cert.c:1536
        ret = <optimized out>
        apr_cert_list = 0x1826f40
        apr_pkey = 0x18077b0
        apr_cert_list_length = 1
        signature = {data = 0x0, size = 0}
        sign_algo = <optimized out>
#10 0x00007fcce734ef71 in _gnutls_send_client_certificate_verify (session=session at entry=0x1808d70, again=0) at gnutls_kx.c:315
        data = {allocd = 0x0, data = 0x0, max_length = 0, length = 0}
        ret = 0
#11 0x00007fcce734c687 in _gnutls_handshake_client (session=0x1808d70) at gnutls_handshake.c:2777
        ret = <optimized out>
#12 gnutls_handshake (session=session at entry=0x1808d70) at gnutls_handshake.c:2535
        ret = <optimized out>
        params = 0x180b0b0
#13 0x00007fcce8c3ff1e in handshake (conn=conn at entry=0x1815c30, sockindex=sockindex at entry=0, duringconnect=duringconnect at entry=true, nonblocking=nonblocking at entry=true) at vtls/gtls.c:303
        data = 0x17fe2f0
        connssl = 0x1815e38
        session = 0x1808d70
        sockfd = 9
        timeout_ms = <optimized out>
        rc = <optimized out>
        what = <optimized out>
#14 0x00007fcce8c40df3 in gtls_connect_common (conn=conn at entry=0x1815c30, sockindex=sockindex at entry=0, nonblocking=nonblocking at entry=true, done=done at entry=0x7fff9b21a735) at vtls/gtls.c:967
        rc = <optimized out>
        connssl = 0x1815e38
#15 0x00007fcce8c411dd in Curl_gtls_connect_nonblocking (conn=conn at entry=0x1815c30, sockindex=sockindex at entry=0, done=done at entry=0x7fff9b21a735) at vtls/gtls.c:989
No locals.
#16 0x00007fcce8c418e0 in Curl_ssl_connect_nonblocking (conn=conn at entry=0x1815c30, sockindex=sockindex at entry=0, done=0x7fff9b21a735) at vtls/vtls.c:293
        res = <optimized out>
#17 0x00007fcce8c0037e in https_connecting (conn=0x1815c30, done=<optimized out>) at http.c:1383
        result = 1692294855
#18 0x00007fcce8c22c6f in multi_runsingle (multi=multi at entry=0x1813d00, now=..., data=data at entry=0x17fe2f0) at multi.c:1203
        disconnect_conn = false
        msg = 0x0
        connected = false
        async = false
        protocol_connect = false
        dophase_done = false
        done = false
        result = CURLM_OK
        k = <optimized out>
        timeout_ms = <optimized out>
        control = -390090637
#19 0x00007fcce8c237e1 in curl_multi_perform (multi_handle=0x1813d00, running_handles=0x7fcce47710d8) at multi.c:1762
        result = <optimized out>
        wc = 0x1806e98
        multi = 0x1813d00
        data = 0x17fe2f0
        returncode = CURLM_OK
        t = 0xe7dacc1d
        now = {tv_sec = 176387, tv_usec = 922306}




From nmav at gnutls.org  Mon Jun 23 15:41:53 2014
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Mon, 23 Jun 2014 15:41:53 +0200
Subject: [gnutls-help] Issues with libcurl and GnuTLS 3.2
In-Reply-To: <20140623121359.GA14827@marvin.ws.skroutz.gr>
References: <20140623121359.GA14827@marvin.ws.skroutz.gr>
Message-ID: <CAJU7zaLY6vVd0R_rE8PJ2gv_B2yVY6eHLPSAifRLKMEc5nHJGQ@mail.gmail.com>

On Mon, Jun 23, 2014 at 2:14 PM, Apollon Oikonomopoulos
<apoikos at debian.org> wrote:
> Hi,
>
> (Please Cc me on reply, I'm not subscribed to the list. Thanks.)
> I'm trying to debug a (rather painful) issue that apparently lies either in
> libcurl, or GnuTLS 3.2. It all started out with Debian bug #751886 [1],
> where the software at hand (Ganeti) uses Haskell's FFI to call libcurl, which
> in Debian is currently linked with GnuTLS 3.2.
[...]
> I managed to obtain a few meaningful backtraces from the segfaulting instances,
> all of them consistently happening during the handshake and all of them
> with the same call trace like the one at the bottom of this message. Also,
> debug output from a corrupted and a crashed handshake follow.

Hello Apollon,
 Could you get a valgrind trace of the failed instance (with crash or
not)? I suspect there is a memory corruption somewhere and valgrind
may be better than the debugger identifying it.

>  - With the same version of libcurl (7.37.0) linked against gnutls 2.12, luxid
>    works reliably. Actually the bug appeared when Debian switched to a gnutls
>    3.2-linked version of libcurl. I tried out different combinations of
>    libcurl and GnuTLS versions, the only failing combinations were with
>    GnuTLS 3.2.

Andreas mentioned a bug sometime ago that depended on having an
application linked with both gnutls28 and gnutls26. Could that be the
issue in that case too?

regards,
Nikos


From apoikos at debian.org  Mon Jun 23 16:31:43 2014
From: apoikos at debian.org (Apollon Oikonomopoulos)
Date: Mon, 23 Jun 2014 17:31:43 +0300
Subject: [gnutls-help] Issues with libcurl and GnuTLS 3.2
In-Reply-To: <CAJU7zaLY6vVd0R_rE8PJ2gv_B2yVY6eHLPSAifRLKMEc5nHJGQ@mail.gmail.com>
References: <20140623121359.GA14827@marvin.ws.skroutz.gr>
 <CAJU7zaLY6vVd0R_rE8PJ2gv_B2yVY6eHLPSAifRLKMEc5nHJGQ@mail.gmail.com>
Message-ID: <20140623143142.GB21646@marvin.ws.skroutz.gr>

Hello Niko,

On 15:41 Mon 23 Jun     , Nikos Mavrogiannopoulos wrote:
> On Mon, Jun 23, 2014 at 2:14 PM, Apollon Oikonomopoulos
> <apoikos at debian.org> wrote:
> > Hi,
> >
> > (Please Cc me on reply, I'm not subscribed to the list. Thanks.)
> > I'm trying to debug a (rather painful) issue that apparently lies either in
> > libcurl, or GnuTLS 3.2. It all started out with Debian bug #751886 [1],
> > where the software at hand (Ganeti) uses Haskell's FFI to call libcurl, which
> > in Debian is currently linked with GnuTLS 3.2.
> [...]
> > I managed to obtain a few meaningful backtraces from the segfaulting instances,
> > all of them consistently happening during the handshake and all of them
> > with the same call trace like the one at the bottom of this message. Also,
> > debug output from a corrupted and a crashed handshake follow.
> 
> Hello Apollon,
>  Could you get a valgrind trace of the failed instance (with crash or
> not)? I suspect there is a memory corruption somewhere and valgrind
> may be better than the debugger identifying it.

Right, here's the output on a run with segfault:

==7248== Memcheck, a memory error detector
==7248== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==7248== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info
==7248== Command: /usr/sbin/ganeti-luxid -f
==7248== 
2014-06-23 17:26:32,671936000000 EEST: ganeti-luxid pid=7248 NOTICE ganeti-luxid daemon startup
2014-06-23 17:26:33,097049000000 EEST: ganeti-luxid pid=7248 INFO Loaded new config, serial 30
2014-06-23 17:26:33,115620000000 EEST: ganeti-luxid pid=7248 INFO Starting up in inotify mode
==7248== Conditional jump or move depends on uninitialised value(s)
==7248==    at 0x6B0FF70: gnutls_session_get_data (in /usr/lib/x86_64-linux-gnu/libgnutls-deb0.so.28.30.6)
==7248==    by 0x52FB939: gtls_connect_step3 (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.3.0)
==7248==    by 0x52FBE49: gtls_connect_common (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.3.0)
==7248==    by 0x52FC8DF: Curl_ssl_connect_nonblocking (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.3.0)
==7248==    by 0x52BB37D: https_connecting (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.3.0)
==7248==    by 0x52DDC6E: multi_runsingle (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.3.0)
==7248==    by 0x52DE7E0: curl_multi_perform (in /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.3.0)
==7248==    by 0x725700: ??? (in /usr/lib/ganeti/2.10/usr/sbin/ganeti-luxid)
==7248== 
2014-06-23 17:26:38,584101000000 EEST: ganeti-luxid pid=7248 INFO Successfully handled Query
==7248== Invalid read of size 8
==7248==    at 0xAFCB7A: ??? (in /usr/lib/ganeti/2.10/usr/sbin/ganeti-luxid)
==7248==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==7248== 
==7248== 
==7248== Process terminating with default action of signal 11 (SIGSEGV)
==7248==  Access not within mapped region at address 0x0
==7248==    at 0xAFCB7A: ??? (in /usr/lib/ganeti/2.10/usr/sbin/ganeti-luxid)
==7248==  If you believe this happened as a result of a stack
==7248==  overflow in your program's main thread (unlikely but
==7248==  possible), you can try to increase the size of the
==7248==  main thread stack using the --main-stacksize= flag.
==7248==  The main thread stack size used in this run was 8388608.
==7248== 
==7248== HEAP SUMMARY:
==7248==     in use at exit: 397,811 bytes in 1,275 blocks
==7248==   total heap usage: 8,921 allocs, 7,646 frees, 2,577,477 bytes allocated
==7248== 
==7248== LEAK SUMMARY:
==7248==    definitely lost: 0 bytes in 0 blocks
==7248==    indirectly lost: 0 bytes in 0 blocks
==7248==      possibly lost: 0 bytes in 0 blocks
==7248==    still reachable: 397,811 bytes in 1,275 blocks
==7248==         suppressed: 0 bytes in 0 blocks
==7248== Rerun with --leak-check=full to see details of leaked memory
==7248== 
==7248== For counts of detected and suppressed errors, rerun with: -v
==7248== Use --track-origins=yes to see where uninitialised values come from
==7248== ERROR SUMMARY: 3 errors from 2 contexts (suppressed: 2 from 2)

> 
> >  - With the same version of libcurl (7.37.0) linked against gnutls 2.12, luxid
> >    works reliably. Actually the bug appeared when Debian switched to a gnutls
> >    3.2-linked version of libcurl. I tried out different combinations of
> >    libcurl and GnuTLS versions, the only failing combinations were with
> >    GnuTLS 3.2.
> 
> Andreas mentioned a bug sometime ago that depended on having an
> application linked with both gnutls28 and gnutls26. Could that be the
> issue in that case too?

IIRC, this was due to symbol clashes. The binary indeed pulls in both as 
indirect dependencies, but gnutls28 has symbol versioning, so I wouldn't 
expect this to be an issue.

Thanks!
Apollon


From nmav at gnutls.org  Tue Jun 24 11:57:02 2014
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Tue, 24 Jun 2014 11:57:02 +0200
Subject: [gnutls-help] DTLS Handshake between server and client.
In-Reply-To: <CAPtPDjYONZF8erVMjgPC1kQ7y1aBV0bhNB6mOEFuSZQB6A6o3g@mail.gmail.com>
References: <CAPtPDjYONZF8erVMjgPC1kQ7y1aBV0bhNB6mOEFuSZQB6A6o3g@mail.gmail.com>
Message-ID: <CAJU7zaL9LQtoPf-hEZ3kDGE9ozf1bnSWSRm4u5kZ6=kh7s-V_w@mail.gmail.com>

On Mon, Jun 23, 2014 at 9:14 AM, Sandeep Kumar <sandeepdas.cse at gmail.com> wrote:
> Hi,
> I've implemented a test program for server and client using the existing
> example of gnu-tls. This program emulates DTLS handshake over SCTP.
> There are several messages starting from client hello then hello verify
> request etc.
> All i want to know is that whether is it mandatory for server to verify the
> cookie for DTLS because if its case of SCTP the same is already done while
> complete SCTP Handshake.

A server doesn't need to send cookies, unless you want to protect from
a denial of service attack (possible under UDP, no idea whether
possible under SCTP). For more information see:
http://gnutls.org/manual/html_node/DTLS-sessions.html#DTLS-sessions

regards,
Nikos


From apoikos at debian.org  Tue Jun 24 15:20:54 2014
From: apoikos at debian.org (Apollon Oikonomopoulos)
Date: Tue, 24 Jun 2014 16:20:54 +0300
Subject: [gnutls-help] Issues with libcurl and GnuTLS 3.2
In-Reply-To: <20140623121359.GA14827@marvin.ws.skroutz.gr>
References: <20140623121359.GA14827@marvin.ws.skroutz.gr>
Message-ID: <20140624132054.GD2289@marvin.ws.skroutz.gr>

Hello,

On 15:13 Mon 23 Jun     , Apollon Oikonomopoulos wrote:
>  - With the same version of libcurl (7.37.0) linked against gnutls 
>    2.12, luxid
>    works reliably. Actually the bug appeared when Debian switched to a gnutls
>    3.2-linked version of libcurl. I tried out different combinations of 
>    libcurl and GnuTLS versions, the only failing combinations were with 
>    GnuTLS 3.2. 
> 

> --- Backtrace
> 
> #0  0x00007fcce89d0002 in __gmpn_powm () from /usr/lib/x86_64-linux-gnu/libgmp.so.10
> No symbol table info available.
> #1  0x00007fcce8997a8d in __gmpz_powm () from /usr/lib/x86_64-linux-gnu/libgmp.so.10
> No symbol table info available.
> #2  0x00007fcce59fdbf2 in _nettle_rsa_blind (pub=pub at entry=0x7fff9b21a230, random_ctx=random_ctx at entry=0x0, random=random at entry=0x7fcce73ec5e0 <rnd_func>, c=c at entry=0x7fff9b21a220, ri=ri at entry=0x7fff9b21a1b0) at rsa-blind.c:56
>         r = {{_mp_alloc = 33, _mp_size = 32, _mp_d = 0x7fcce470f9b8}}

The good news is that this is no GnuTLS bug.

For future reference, the problem seems to be in GMP's memory 
management: GHC by default uses GMP for its Integer and Fractional 
implementation[1]; binaries produced by GHC embed the GHC runtime and 
are linked with libgmp. GMP in turn uses a per-process memory heap, 
which in the GHC RTS case is managed by the Haskell garbage collector 
using mp_set_memory_functions((*alloc)(), (*realloc)(), 
(*dealloc)())[1]. The Garbage Collector in turn probably assumes that 
this heap should contain Haskell objects known to itself exclusively.

Now, gnutls28 pulls in and uses libgmp as well (via nettle), but this 
use is opaque to the Haskell garbage collector, since the gmp function 
calls happen way below the GHC runtime and their results are not bound 
to Haskell objects. Eventually a Haskell GC run will overwrite the data 
used by any FFI calls to libraries using GMP themselves, causing 
extensive memory corruption of the key material as seen in the 
backtrace. This explains both, the segfaults and the "Decrypt errors" 
encountered.

gnutls26 does not use nettle/GMP and that is why the problem does not 
manifest.

[1] https://ghc.haskell.org/trac/ghc/wiki/ReplacingGMPNotes

Regards,
Apollon


From nmav at gnutls.org  Thu Jun 26 20:36:42 2014
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Thu, 26 Jun 2014 20:36:42 +0200
Subject: [gnutls-help] gnutls 3.3.5
Message-ID: <1403807802.27453.4.camel@nomad.lan>

Hello,
 I've just released gnutls 3.3.5. This release adds new features, and fixes bugs on the next-stable branch.


* Version 3.3.5 (released 2014-06-26)

** libgnutls: Added gnutls_record_recv_packet() and gnutls_packet_deinit().
These functions provide a variant of gnutls_record_recv() that avoids
the final memcpy of data.

** libgnutls: gnutls_x509_crl_iter_crt_serial() was added as a
faster variant of gnutls_x509_crl_get_crt_serial() when coping with
very large structures.

** libgnutls: When the decoding of a printable DN element fails, then treat
it as unknown and print its hex value rather than failing. That works around
an issue in a TURKTRST root certificate which improperly encodes the
X520countryName element.

** libgnutls: gnutls_x509_trust_list_add_trust_file() will return the number
of certificates present in a PKCS #11 token when loading it.

** libgnutls: Allow the post client hello callback to put the handshake on
hold, by returning GNUTLS_E_AGAIN or GNUTLS_E_INTERRUPTED.

** certtool: option --to-p12 will now consider --load-ca-certificate

** certtol: Added option to specify the PKCS #12 friendly name on command
line.

** p11tool: Allow marking a certificate copied to a token as a CA.

** API and ABI modifications:
GNUTLS_PKCS11_OBJ_FLAG_MARK_CA: Added
gnutls_x509_crl_iter_deinit: Added
gnutls_x509_crl_iter_crt_serial: Added
gnutls_record_recv_packet: Added
gnutls_packet_deinit: Added
gnutls_packet_get: Added


Getting the Software
====================

GnuTLS may be downloaded directly from
<ftp://ftp.gnutls.org/gcrypt/gnutls/>.  A list of GnuTLS mirrors can be
found at <http://www.gnutls.org/download.html>.

Here are the XZ and LZIP compressed sources:

  ftp://ftp.gnutls.org/gcrypt/gnutls/v3.3/gnutls-3.3.5.tar.xz
  ftp://ftp.gnutls.org/gcrypt/gnutls/v3.3/gnutls-3.3.5.tar.lz

Here are OpenPGP detached signatures signed using key 0x96865171:

  ftp://ftp.gnutls.org/gcrypt/gnutls/v3.3/gnutls-3.3.5.tar.xz.sig
  ftp://ftp.gnutls.org/gcrypt/gnutls/v3.3/gnutls-3.3.5.tar.lz.sig

Note that it has been signed with my openpgp key:
pub   3104R/96865171 2008-05-04 [expires: 2028-04-29]
uid                  Nikos Mavrogiannopoulos <nmav <at> gnutls.org>
uid                  Nikos Mavrogiannopoulos <n.mavrogiannopoulos <at>
gmail.com>
sub   2048R/9013B842 2008-05-04 [expires: 2018-05-02]
sub   2048R/1404A91D 2008-05-04 [expires: 2018-05-02]

regards,
Nikos




From ametzler at bebt.de  Sat Jun 28 19:45:38 2014
From: ametzler at bebt.de (Andreas Metzler)
Date: Sat, 28 Jun 2014 19:45:38 +0200
Subject: [gnutls-help] 3.3.5 testsuite error on kfreebsd-amd64
Message-ID: <20140628174538.GP1461@downhill.g.la>

Hello,

./tests/long-session-id occassionally fails on kfreebsd-amd64 with
exitcode 141, sadly --verbose looks exactly the same as a succeeding
test.

cu Andreas

-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'
-------------- next part --------------
A non-text attachment was scrubbed...
Name: long-session-id.log.fail.gz
Type: application/gzip
Size: 1148 bytes
Desc: not available
URL: </pipermail/attachments/20140628/b16c3be6/attachment-0001.bin>