GPGME CVS branched

[Simon Josefsson]

Marcus Brinkmann <marcus.brinkmann at ruhr-uni-bochum.de> writes:

> At Sun, 12 Dec 2004 18:44:14 +0100,
> Simon Josefsson <jas at extundo.com> wrote:
>> Marcus Brinkmann <marcus.brinkmann at ruhr-uni-bochum.de> writes:
>> I find that requiring threaded applications to provide thread hooks
>> into libraries is problematic.  The most common situation where a this
>> causes problems is when a threaded application uses one library that
>> in turn uses gcrypt/gpgme.
>
> This is in fact exactly the situation that required us to take the
> hook approach.
>
> We have done extensive analysis of this problem, and we have seen what
> happened to our other attempts at solving this.  And we have tried
> everything we could think of: magic support via weak symbols,
> different versions of the same library a la foo-threadpackage, and the
> hooks.  We have also looked at how other projects solve this, and have
> not seen anything better elsewhere.  There is no perfect approach for
> this problem, it's impossible.  The goals are conflicting (support for
> several thread libraries, portability to many platforms, supporting
> mixed library environments).  So the best we can do is to be flexible
> and make the problem obvious in whatever ways we can.
>
> You can find an extensive defense of our approach in the gcrypt
> mailing list:
>
> http://lists.gnupg.org/pipermail/gcrypt-devel/2004-March/000566.html

Thanks.  I did not read this very carefully, but I cannot find any
discussion of WHY you actually require thread primitives in libgcrypt.

I understand that once you have a design that uses thread primitives
like mutexes, you will get into all the problems you went through.
And I agree it probably can't be solved cleanly.  So I'm trying to
explore the consequence of that conclusion, and attack the reason you
needed mutexes in the first place: why?

Compare libgcrypt with Nettle (that I use in GSASL/Shishi, via a
abstraction layer that may use libgcrypt instead).  Nettle does not
require any mutexes, and consequently doesn't suffer from the issues
you bring up.  I'm trying to understand why libgcrypt couldn't use a
similar strategy to solve this problem, instead of "infecting" the
users of libgcrypt with the hooks-approach.

>> and consequently does not set them any thread hooks in gcrypt.
>
> It should export an interface to the user to do so (assuming that the
> fact that gcrypt is used should be hidden).

The fact that gcrypt is used is not very well hidden right now,
because of the secure memory issues.  But is complex when libgcrypt is
an optional dependency of my library.  Making applications link to
libgcrypt "just in case" my library uses libgcryp seems bad.

>> Requiring the application to be notice that gnutls/gsasl uses gcrypt,
>> and have the application call the thread hooks in gcrypt, destroy
>> abstraction layers.
>
> I always found it a mistake by the gnutls developer to let the user of
> gnutls call a gcrypt function directly.  I advised him to wrap the
> interface.

I have written both libraries and applications that use GnuTLS, and I
have never had to call gcrypt functions directly because of that.
Perhaps everything is wrapped now.

> Have you actually tried mixing thread libraries?

No, it was a hypothetical example.  I recall seeing the issue come up
with some big applications, like OpenOffice or GNOME, but don't know
the details.

>> I reviewed all uses of thread hooks in libgcrypt (a lot!), and to me
>> all of the uses could be avoided by choosing a simpler design.
>> Compare Nettle.  Perhaps the only exception I could find was
>> randomness gathering, but then again I think that is such a critical
>> function that the complexity of thread interaction should be avoided
>> at all.
>
> Well, once you have mutex support, it doesn't hurt to use it when it
> comes in useful.  So, unless you can eliminate all uses, it doesn't
> make sense to talk about eliminating some of them.

Right.  But when trying to figure out what critical uses of libgcrypt
have for mutexes, it would help to get rid of the non-critical parts.

(I'm not proposing to change libgcrypt, it is probably too late for
that.  I'm talking about these issues in general, taking libgcrypt and
possibly gpgme as examples.)

> Of course there are alternative approaches possible.  But they come at
> a cost, too, and it is always a trade-off.  Personally, I would have
> designed some of the gcrypt interfaces differently myself, but that's
> a different story.  I am not sure I would opt for a design that does
> not require any mutual exclusion, but then, we have not even said
> explicitely how such a design in the context of gcrypt may look, so
> it's a moot point.

What costs do you see with a mutex-free design?  I'm looking from the
application point of view, and the Nettle API seems simpler than the
Libgcrypt API, because I don't have to worry about thread
interactions.  I'm not sure I can identify any cost that the Nettle
API brings me that the libgcrypt API doesn't also suffer from.

Thanks,
Simon