1.1.42 / Win32 MinGW : random number generation is very slow
Ludovic LANGE
llange@users.sourceforge.net
Fri, 29 Aug 2003 14:34:24 +0200
This is a multi-part message in MIME format.
--------------050601090308070605070702
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Hello,
I'm trying to generate MPI random numbers on a Win32 (MinGW on a
Win2000) platform. (More precisely, I'm trying to add support for an
encrypted protocol in GAIM, an Instant Messenging application)
In GAIM, I'm using the gcry_mpi_randomize function, with a level of :
GCRY_VERY_STRONG_RANDOM to generate a session key
According to my tests, the first random number generation in GAIM is
very, very slow during the first invocation (sometime it can take up to
1~2 minutes !!) which is too long for the interactive use I need it for.
However, I should note that a test program, *outside GAIM*, is very fast
to generate the first random number.
I didn't have the time to fully understand the internals of rndw32.c,
but here is the debug info with a timer ( QueryPerformanceCounter )
comparing the test program invoking gcry_mpi_randomize function, then
GAIM invoking gcry_mpi_randomize function.
The timer is the delta, in performance counter increments, between the
current log call and the precedent log call.
/* Test program */
duration: 0 rndw32#gather_random: platform=2
duration: 222 rndw32#gather_random: req=3 len=300 lvl=2
duration: 33 rndw32#slow_gatherer_95: init toolkit
duration: 46709 rndw32#slow_gatherer_95: walk heap
duration: 79843 rndw32#slow_gatherer_95: walk processes
duration: 9746 rndw32#slow_gatherer_95: walk threads
duration: 22015 rndw32#slow_gatherer_95: walk modules
duration: 2932 rndw32#slow_gatherer_nt: init toolkit
duration: 315 rndw32#slow_gatherer_nt: check product options
duration: 31493 rndw32#slow_gatherer_nt: netapi32 loaded
duration: 5816 rndw32#slow_gatherer_nt: get netstats
duration: 386 NOTE: you should run 'diskperf -y' to enable the disk
statistics
duration: 280 rndw32#slow_gatherer_nt: get perf data
duration: 2388387 rndw32#slow_gatherer_nt: get perf data
duration: 846706 rndw32#slow_gatherer_nt: get perf data
duration: 816804 rndw32#slow_gatherer_nt: get perf data
duration: 1184713 rndw32#gather_random_fast: req=1
duration: 820 rndw32#gather_random_fast: perf data
(A total of 3 heaps were walked)
heap 1, 56 blocks
heap 2, 2 blocks
heap 3, 114 blocks
/* GAIM */
duration: 0 rndw32#gather_random: platform=2
duration: 4154 rndw32#gather_random: req=3len=300 lvl=2
duration: 1869 rndw32#slow_gatherer_95: init toolkit
duration: 8215 rndw32#slow_gatherer_95: walk heap
duration: 328406929 rndw32#slow_gatherer_95: walk processes
duration: 13863 rndw32#slow_gatherer_95: walk threads
duration: 27643 rndw32#slow_gatherer_95: walk modules
duration: 35375 rndw32#slow_gatherer_nt: init toolkit
duration: 2366 rndw32#slow_gatherer_nt: check product options
duration: 3367 rndw32#slow_gatherer_nt: netapi32 loaded
duration: 5028 rndw32#slow_gatherer_nt: get netstats
duration: 2724 NOTE: you should run 'diskperf -y' to enable the disk
statistics
duration: 1762 rndw32#slow_gatherer_nt: get perf data
duration: 1353009 rndw32#slow_gatherer_nt: get perf data
duration: 872048 rndw32#slow_gatherer_nt: get perf data
duration: 838450 rndw32#slow_gatherer_nt: get perf data
duration: 1038618 rndw32#gather_random_fast: req=1
duration: 2711 rndw32#gather_random_fast: perf data
(A total of 10 heaps were walked)
heap 1, 678 blocks
heap 2, 2 blocks
heap 3, 16099 blocks
heap 4, 7 blocks
heap 5, 7 blocks
heap 6, 46 blocks
heap 7, 8 blocks
heap 8, 8 blocks
heap 9, 3 blocks
heap 10, 9 blocks
GAIM is a GTK 2.0 application, compiled with MinGW. As you can see from
the numbers, in GAIM quite all the numbers are a little higher (which
can be normal although I don't know why), but the striking difference is
the number before the 'walk processes' log call.
Considering the timer I added, this in fact means that the time spent
between the log line 'walk heap' and 'walk processes' is very important.
Considering the file 'rndw32.c', this means that the 'walk heap' loop is
very, very long indeed. I added a count of the number of heap / heap
blocks walked (nbr of calls to pHeap32ListNext / pHeap32Next) to compare.
When I define out the whole 'walk heap' block, the generation is very
fast, so the problem is really there.
Perharps we should consider limiting the loop to a given maximum number
of heaps / heap blocks walked in order to reduce the time spent there ?
Or limiting the number of calls to the inner 'add' function to a given
number of iterations ? (for my application, I limited this to 200 calls
max to the Add function inside the heap loop, and this is very fast
(total of ~2 second for generating the first key).)
As I don't really know the impact on the quality of the entropy pool, I
prefer asking advices on this subject.
Best regards,
Ludovic LANGE
--------------050601090308070605070702
Content-Type: text/plain;
name="libgcrypt-1.1.42-mingw.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="libgcrypt-1.1.42-mingw.diff"
diff -ruN libgcrypt-1.1.42/cipher/random.c libgcrypt-1.1.42.new/cipher/random.c
--- libgcrypt-1.1.42/cipher/random.c Tue Jul 15 00:36:44 2003
+++ libgcrypt-1.1.42.new/cipher/random.c Thu Aug 28 15:40:11 2003
@@ -778,6 +778,11 @@
return fnc;
#endif
+#if USE_RNDW32
+ fnc = rndw32_gather_random;
+ return fnc;
+#endif
+
log_fatal (_("no entropy gathering module detected\n"));
return NULL;
diff -ruN libgcrypt-1.1.42/cipher/rndw32.c libgcrypt-1.1.42.new/cipher/rndw32.c
--- libgcrypt-1.1.42/cipher/rndw32.c Mon Jun 9 15:50:37 2003
+++ libgcrypt-1.1.42.new/cipher/rndw32.c Thu Aug 28 15:41:58 2003
@@ -53,7 +53,7 @@
#include <string.h>
#include <windows.h>
-
+#include <winioctl.h>
#include "types.h"
#include "g10lib.h"
@@ -710,8 +710,8 @@
}
-static int
-gather_random( void (*add)(const void*, size_t, int), int requester,
+int
+rndw32_gather_random( void (*add)(const void*, size_t, int), int requester,
size_t length, int level )
{
static int is_initialized;
@@ -761,8 +761,8 @@
}
-static int
-gather_random_fast( void (*add)(const void*, size_t, int), int requester )
+int
+rndw32_gather_random_fast( void (*add)(const void*, size_t, int), int requester )
{
static int addedFixedItems = 0;
diff -ruN libgcrypt-1.1.42/src/secmem.c libgcrypt-1.1.42.new/src/secmem.c
--- libgcrypt-1.1.42/src/secmem.c Tue Jul 15 00:55:36 2003
+++ libgcrypt-1.1.42.new/src/secmem.c Thu Aug 28 14:05:40 2003
@@ -248,7 +248,9 @@
uid_t uid;
int err;
+#if 0
uid = getuid ();
+#endif
#ifdef HAVE_BROKEN_MLOCK
if (uid)
@@ -267,14 +269,15 @@
if (err && errno)
err = errno;
#endif
-
- if (uid && ! geteuid ())
+#if 0
+ if (uid && ! getuid ())
{
/* check that we really dropped the privs.
* Note: setuid(0) should always fail */
- if (setuid (uid) || getuid () != geteuid () || !setuid (0))
+ if (setuid (uid) || getuid () != getuid () || !setuid (0))
log_fatal ("failed to reset uid: %s\n", strerror (errno));
}
+#endif
if (err)
{
@@ -421,13 +424,15 @@
uid_t uid;
disable_secmem = 1;
+#if 0
uid = getuid ();
- if (uid != geteuid ())
+ if (uid != getuid ())
{
- if (setuid (uid) || getuid () != geteuid () || !setuid (0))
+ if (setuid (uid) || getuid () != getuid () || !setuid (0))
log_fatal ("failed to drop setuid\n");
}
#endif
+#endif
}
else
{
@@ -436,10 +441,14 @@
if (! pool_okay)
{
init_pool (n);
- if (! geteuid ())
+#if 0
+ if (! getuid ())
+#endif
lock_pool (pool, n);
+#if 0
else
log_info ("Secure memory is not locked into core\n");
+#endif
}
else
log_error ("Oops, secure memory pool already initialized\n");
diff -ruN libgcrypt-1.1.42/w32-dll/build-dll libgcrypt-1.1.42.new/w32-dll/build-dll
--- libgcrypt-1.1.42/w32-dll/build-dll Thu Jan 1 01:00:00 1970
+++ libgcrypt-1.1.42.new/w32-dll/build-dll Thu Aug 28 13:40:27 2003
@@ -0,0 +1,71 @@
+#!/bin/sh
+# Run this to generate the libgcrypt W32 DLL
+#
+# Copyright (C) 2001, 2002 Free Software Foundation, Inc.
+#
+# This file is free software; as a special exception the author gives
+# unlimited permission to copy and/or distribute it, with or without
+# modifications, as long as this notice is preserved.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY, to the extent permitted by law; without even the
+# implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+
+AS=as
+OBJS=
+
+grep "#define HAVE_DOSISH_SYSTEM" ../config.h
+if [ $? = 1 ]; then
+ exit
+fi
+
+for i in $(ls ../cipher/*.o); do
+ OBJS="$OBJS $i"
+done
+for i in $(ls ../mpi/*.o); do
+ OBJS="$OBJS $i"
+done
+for i in $(ls ../src/*.o); do
+ OBJS="$OBJS $i"
+done
+
+# add asm code which is postfixed with .lo
+OBJS="$OBJS ../mpi/mpih-sub1.lo ../mpi/mpih-add1.lo ../mpi/mpih-lshift.lo"
+OBJS="$OBJS ../mpi/mpih-mul2.lo ../mpi/mpih-mul1.lo ../mpi/mpih-rshift.lo"
+OBJS="$OBJS ../mpi/mpih-mul3.lo"
+
+# echo 'running dlltool the first time' >&2
+# dlltool --no-delete \
+# --as $AS \
+# --def libgcrypt.def \
+# --output-exp libgcrypt.exp \
+# --output-lib libgcrypt.imp \
+# --dllname libgcrypt.dll $OBJS
+# echo 'doing dummy link to create the base file' >&2
+# gcc -mdll -Wl,--base-file -Wl,libgcrypt.base \
+# -o libgcrypt.dll libgcrypt.exp $OBJS
+# echo 'running dlltool the second time' >&2
+# dlltool --no-delete \
+# --as $AS \
+# --def libgcrypt.def \
+# --output-exp libgcrypt.exp \
+# --output-lib libgcrypt.imp \
+# --base-file libgcrypt.base \
+# --dllname libgcrypt.dll $OBJS
+#
+# echo 'doing final link' >&2
+# gcc -mdll -o libgcrypt.dll libgcrypt.exp $OBJS
+# strip libgcrypt.dll
+#
+# dlltool --def libgcrypt.def \
+# --as $AS \
+# --dllname libgcrypt.dll \
+# --output-lib libgcrypt.lib
+#
+#echo 'clean up'
+#rm -f dh.o dh.s dt.o dt.s
+#rm -f libgcrypt.base libgcrypt.exp libgcrypt.imp tlibgcrypt.exp
+
+echo "$OBJS"
+gcc -shared -o libgcrypt.dll $OBJS -L/usr/local/lib -lgpg-error -Wl,--out-implib,libgcrypt.a -Wl,--enable-auto-image-base
+
--------------050601090308070605070702--