important gcrypt threading support changes

[Marcus Brinkmann]

Hi,

we took your criticism very seriously, and believe you are right.  So,
we made the effort to change the thread support in libgcrypt totally.
It's basically a 180 degree turn.

I want you to pay special attention to the following setup that we
have decided on now.  If any of this goes in the wrong direction, now
(at least before 1.2 is released) is the time to speak up.  After that
it will be extremely painful, I guess.  Of course, I can't make you
responsible for the bugs or misdesign we write.  But I assume that it
is in your own best interest to develop a common strategy between
libgcrypt and its users.

I have checked in some changes that implement the below plan into CVS.
The tags are marcus-before-thread-cbs and marcus-after-thread-cbs.
Please test them and report any problems.

First, let me tell you straight ahead that I am not particularly
familiar with gcrypt's interface.  There are things in the
initialization I didn't study, and in its interfaces, that are
certainly relevant in the whole picture as far as thread support is
concerned.  There might be design problems that are not addressed in
this mail at all.  It would be good to go over the whole interface
with a fine toothed comb and look hard at all of it.  My personal
feeling is that for a library as low-level as gcrypt, having a more
abstract execution context that is used for each and every function
where it matters is advisable.  However, that's just my gut feeling,
and this is really all I have to say about it.  The rest of this mail
is about the minimal requirements of good thread support, if all other
interfaces are right (which they may or may not be).

I added a new structure, gcry_thread_cbs, which has the following
members: An enum gcry_thread_option option value, and a series of
function pointers for functions: init, mutex_init, mutex_destroy,
mutex_lock, mutex_unlock, read, write, select, etc.  These members
split into four groups:

1. enum gcry_thread_option option: This member is a value that
   characterizes the callbacks as being for support of a particular
   thread package.  Its values can be GCRY_THREAD_OPTION_DEFAULT,
   GCRY_THREAD_OPTION_PTH, GCRY_THREAD_OPTION_PTHREAD, and
   GCRY_THREAD_OPTION_USER.  The meaning of this field will be
   explained below.

2. init: This callback initializes the thread package.  For pthread
   this can be NULL, for pth this must call pth_init() (and convert
   the return value to a system error value).  The init function is
   called once at gcrypt's own initialization time.

3. mutex_init, mutex_destroy, mutex_lock, mutex_unlock: These
   functions create, destroy, acquire and release a mutex variable.
   They can be mapped easily to the according pth/pthread function
   calls.  All except mutex_destroy must be != NULL (this is more or
   less arbitrary.  Call it a sanity check.  We can change that if
   convenient).

4. read, write, select etc: These are optional.  If they are not NULL,
   gcrypt uses them in place of the system call with the same name.
   For pth, these are mapped to the pth_* variants of these call,
   which yield to anothe ruser thread if the system call blocks.

Callbacks are installed with

err = gcry_control (GCRYCTL_SET_THREAD_CBS, struct gcry_thread_cbs *cbs);
if (err)
  ...

_before_ _any_ _other_ function in the library.

So far, so easy.  For the programmers convenience, I added two macros.
They are used in the following way:

#include <errno.h>
#include <pth.h>
GCRY_THREAD_OPTION_PTH_IMPL;

#include <errno.h>
#include <pthread.h>
GCRY_THREAD_OPTION_PTHREAD_IMPL;

These define static callback functions for the appropriate thread
package.  They also define static callback structures gcry_threads_pth
resp. gcry_threads_pthread that set the OPTION value appropriately.
You can then use them with gcry_control to install them as your
callback handlers.

This sounds still very easy.  The complexity comes in if you are not
an application developer, but a library developer.  Let's investigate
a couple of scenarios:

A. Your library requires a certain thread package.  In this case, you
can hardwire the thread package at gcrypt initialization time.  The
application will have to use the thread library of your choice.

B. Your library requires a thread library for internal locking, etc,
but you support several of them.  Then you already provide a way to
let the user of your library select which thread package to use.  In
this case, you can install the appropriate callback handlers at gcrypt
initialization time after the user made his choice.

C. Your library is thread-safe without the support of any thread
library.  This is good for you, but gcrypt still needs your help.  So
you have three alternatives here: Do A. above.  Do B. above.  Do the
following (called "C"): You export the gcrypt interface as part of the
interface of _your_ library and require that the user initializes
gcrypt for you.  This is a viable option if you don't want to add your
own wrapper interface ("B" above).  However, it will prevent your use
of gcrypt to be transparent to the user of your library (which might
be bad).

These are the basic cases I can come up with.  Serious difficulties
only arise when you have more than one library in a single
application.  You can have one of the following cases:

D. Your library uses gcrypt transparently (it's not part of the
official interface).  The application also uses gcrypt independently
of your library.

E. You library uses gcrypt transparently, and another library linked
into the same application uses gcrypt (transparently or not).

F. Your library uses gcrypt transparently, another library does the
same, and the application also uses gcrypt independently of the
libraries.

First, let us assume that all libraries and applications use
compatible versions of gcrypt (ie, they agree on the soname, so that
only one instance of the library is linked into the program).  Then
all symbols will be resolved correctly.  There is still a problem, as
to who gets to initialize gcrypt and make a selection of the features.
The dominant feature in this discussion.  I already hinted above that
there are other issues that might come into play which are not
resolved here (secure memory allocation, etc).  Let me address only
the issue of selecting the thread package.

The way we solve this is, basically, that the first one to initialize
gcrypt wins, but that if any of the other loses, an error is returned.
This means that the following happens:

The first one to initialize the gcrypt library will "burn in" the enum
gcry_thread_option OPTION value from the thread callbacks he used.  If
no callbacks where registered, the value is
GCRY_THREAD_OPTION_DEFAULT.

Any subsequent initialization attempt will make the following check:

If a subsequent initialization attempt does not involve setting the
thread callbacks, nothing happens.

If a subsequent initialization attempt involves setting the thread
callbacks, the following check is made:

	If either of the two thread options, the current "burned in"
	one or the requested one, is GCRY_THREAD_OPTION_USER, then an
	error is returned.

	If neither is GCRY_THREAD_OPTION_USER, but the two are
	different (for example, one is PTH and one is PTHREAD), then
	an error is returned.

	Otherwise the two thread options are the same, and they are
	not USER, so success is returned.

The idea is that we have no idea what happens if USER is used, but the
user will make certain strict guarantees over what happens if PTH or
PTHREAD is used as GCRY_THREAD_OPTION_ value.  Basically, he uses the
official GCRY_THREAD_OPTION_{PTH,PTHREAD}_IMPL implementations or
compatible ones.

This will result in a sanity check if different users of gcrypt
request different thread supports.

Again, this check is only affected by the choice of the thread library.
It might be that with the current code, there are other compatibility
issues to resolve before this really works.  But eventually, we might
be able to support multiple independent users of gcrypt in the same
application without conflicts.

Comments are highly appreciated.  The above design is built in a bit
of a hurry (certainly quicker than I am comfortable with), to address
your concerns for gcrypt 1.2.  Still, we have some amount of
confidence into it.  Now we would like your opinion.

Thanks,
Marcus