From misc at dstoecker.de Mon Sep 5 11:13:55 2005 From: misc at dstoecker.de (Dirk Stoecker) Date: Mon Sep 5 11:19:21 2005 Subject: libgcrypt and patches [CLEANUP_PATCH] In-Reply-To: <87vf1u2gqt.fsf@wheatstone.g10code.de> References: <87vf1u2gqt.fsf@wheatstone.g10code.de> Message-ID: Hello, > > Is there any sense in resending the patches? > > I recall a mail which tried to solve some one time allocations - we > want apply such a fix. Ok, I try it again. Attached is a minor patch against current svn, which uses GCRYCTL_FINALIZE to get a cleanup of the whole library. When this is used, all the loaded memory is freed in a way which (hopefully) brings the system to initial state. It should be possible to reuse the library afterwards as if it has never been used (except for possible bugs). I use this for 3-4 months now and never had any problems. I did not modify the examples, althought this would be a good idea. All the examples should run in valgrind without reporting any unfreed memory! Ciao -- ____ _ _ ____ _ _ _ _ ____ | | | | | | \ / | | | the cool Gremlin from Bischofswerda | __ | ____| | \/ | | | WWW: http://www.dstoecker.de/ | | | | | | | | PGP key available on www page. |____| _|_ |____| _|_ _|_ |____| I hope AMIGA never stops making fun! -------------- next part -------------- Eigenschafts?nderungen: ___________________________________________________________________ Name: svn:ignore + aclocal.m4 autom4te.cache config.h config.h.in config.log config.status configure libtool Makefile Makefile.in stamp-h1 Eigenschafts?nderungen: w32-dll ___________________________________________________________________ Name: svn:ignore + Makefile Makefile.in Eigenschafts?nderungen: mpi ___________________________________________________________________ Name: svn:ignore + .deps .libs asm-syntax.h Makefile Makefile.in mpi-asm-defs.h mpih-add1.S mpih-lshift.S mpih-mul1.S mpih-mul2.S mpih-mul3.S mpih-rshift.S mpih-sub1.S sysdep.h Eigenschafts?nderungen: src ___________________________________________________________________ Name: svn:ignore + .deps .libs libgcrypt-config Makefile Makefile.in Index: src/global.c =================================================================== --- src/global.c (Revision 1102) +++ src/global.c (Arbeitskopie) @@ -81,7 +81,18 @@ BUG (); } +static void +global_finalize (void) +{ + _gcry_pk_finalize(); + _gcry_md_finalize(); + _gcry_cipher_finalize(); +// ath_finalize(); + _gcry_random_finalize(); + any_init_done = 0; +} + static const char* parse_version_number( const char *s, int *number ) { @@ -284,6 +295,10 @@ _gcry_fast_random_poll (); break; + case GCRYCTL_FINALIZE: + global_finalize(); + break; + default: err = GPG_ERR_INV_OP; } Index: src/g10lib.h =================================================================== --- src/g10lib.h (Revision 1102) +++ src/g10lib.h (Arbeitskopie) @@ -239,6 +239,10 @@ gcry_err_code_t _gcry_pk_init (void); gcry_err_code_t _gcry_ac_init (void); +void _gcry_cipher_finalize(void); +void _gcry_md_finalize(void); +void _gcry_pk_finalize(void); + gcry_err_code_t _gcry_pk_module_lookup (int id, gcry_module_t *module); void _gcry_pk_module_release (gcry_module_t module); gcry_err_code_t _gcry_pk_get_elements (int algo, char **enc, char **sig); Eigenschafts?nderungen: tests ___________________________________________________________________ Name: svn:ignore + .deps .libs ac ac-data ac-schemes basic benchmark hmac keygen Makefile Makefile.in pkbench prime pubkey register tsexp Eigenschafts?nderungen: doc ___________________________________________________________________ Name: svn:ignore + gcrypt.info Makefile Makefile.in Eigenschafts?nderungen: cipher ___________________________________________________________________ Name: svn:ignore + .deps .libs Makefile.in Makefile Index: cipher/pubkey.c =================================================================== --- cipher/pubkey.c (Revision 1102) +++ cipher/pubkey.c (Arbeitskopie) @@ -2305,6 +2305,20 @@ return err; } +void +_gcry_pk_finalize(void) +{ + if(default_pubkeys_registered) + { + ath_mutex_lock(&pubkeys_registered_lock); + while(pubkeys_registered) + { + _gcry_module_release(pubkeys_registered); + } + default_pubkeys_registered = 0; + ath_mutex_unlock(&pubkeys_registered_lock); + } +} gcry_err_code_t _gcry_pk_module_lookup (int algorithm, gcry_module_t *module) Index: cipher/md.c =================================================================== --- cipher/md.c (Revision 1102) +++ cipher/md.c (Arbeitskopie) @@ -1212,6 +1212,20 @@ return err; } +void +_gcry_md_finalize(void) +{ + if(default_digests_registered) + { + ath_mutex_lock(&digests_registered_lock); + while(digests_registered) + { + _gcry_module_release(digests_registered); + } + default_digests_registered = 0; + ath_mutex_unlock(&digests_registered_lock); + } +} int gcry_md_is_secure (gcry_md_hd_t a) Index: cipher/cipher.c =================================================================== --- cipher/cipher.c (Revision 1102) +++ cipher/cipher.c (Arbeitskopie) @@ -1377,6 +1377,21 @@ return err; } +void +_gcry_cipher_finalize(void) +{ + if(default_ciphers_registered) + { + ath_mutex_lock(&ciphers_registered_lock); + while(ciphers_registered) + { + _gcry_module_release(ciphers_registered); + } + default_ciphers_registered = 0; + ath_mutex_unlock(&ciphers_registered_lock); + } +} + /* Get a list consisting of the IDs of the loaded cipher modules. If LIST is zero, write the number of loaded cipher modules to LIST_LENGTH and return. If LIST is non-zero, the first Index: cipher/random.c =================================================================== --- cipher/random.c (Revision 1102) +++ cipher/random.c (Arbeitskopie) @@ -168,10 +168,8 @@ keypool = secure_alloc ? gcry_xcalloc_secure(1,POOLSIZE+BLOCKLEN) : gcry_xcalloc(1,POOLSIZE+BLOCKLEN); is_initialized = 1; - } - /* Used to register a progress callback. */ void _gcry_register_random_progress (void (*cb)(void *,const char*,int,int,int), @@ -206,6 +204,17 @@ } void +_gcry_random_finalize(void) +{ + if(is_initialized) + { + gcry_free(rndpool); + gcry_free(keypool); + is_initialized = 0; + } +} + +void _gcry_random_dump_stats() { log_info ( Index: cipher/random.h =================================================================== --- cipher/random.h (Revision 1102) +++ cipher/random.h (Arbeitskopie) @@ -23,6 +23,7 @@ #include "types.h" void _gcry_random_initialize (int full); +void _gcry_random_finalize(void); void _gcry_register_random_progress (void (*cb)(void *,const char*,int,int,int), void *cb_data ); void _gcry_random_dump_stats(void); From wk at gnupg.org Mon Sep 5 18:47:42 2005 From: wk at gnupg.org (Werner Koch) Date: Mon Sep 5 18:51:25 2005 Subject: libgcrypt and patches [CLEANUP_PATCH] In-Reply-To: (Dirk Stoecker's message of "Mon, 5 Sep 2005 11:13:55 +0200 (CEST)") References: <87vf1u2gqt.fsf@wheatstone.g10code.de> Message-ID: <871x435ump.fsf@wheatstone.g10code.de> On Mon, 5 Sep 2005 11:13:55 +0200 (CEST), Dirk Stoecker said: > Ok, I try it again. Attached is a minor patch against current svn, which > uses GCRYCTL_FINALIZE to get a cleanup of the whole library. When this is > used, all the loaded memory is freed in a way which (hopefully) brings > the system to initial state. It should be possible to reuse the library I still fail to see the need for it. Why do you want to remove a non simple DSO out of your process (which is actually not possible on most systems) once it has been loaded? Shalom-Salam, Werner From gcrypt at dstoecker.de Mon Sep 5 19:36:21 2005 From: gcrypt at dstoecker.de (Dirk Stoecker) Date: Mon Sep 5 20:17:12 2005 Subject: libgcrypt and patches [CLEANUP_PATCH] In-Reply-To: <871x435ump.fsf@wheatstone.g10code.de> References: <87vf1u2gqt.fsf@wheatstone.g10code.de> <871x435ump.fsf@wheatstone.g10code.de> Message-ID: On Mon, 5 Sep 2005, Werner Koch wrote: > On Mon, 5 Sep 2005 11:13:55 +0200 (CEST), Dirk Stoecker said: > > > Ok, I try it again. Attached is a minor patch against current svn, which > > uses GCRYCTL_FINALIZE to get a cleanup of the whole library. When this is > > used, all the loaded memory is freed in a way which (hopefully) brings > > the system to initial state. It should be possible to reuse the library > > I still fail to see the need for it. Why do you want to remove a non > simple DSO out of your process (which is actually not possible on most > systems) once it has been loaded? No. I don't want to remove it. I want to free it's allocated resources. libgcrypt dynamically allocates resources. The software I need this for has really heavy memory usage including lots of allocations, ... The points are: 1) Debugging tools to track memory loss cannot and will not distinguish between memory loss on purpose and memory loss due to programming errors. 2) I want the memory back. I have enough other memory troubles. I do not need to waste memory, which is only used once in the application. 3) Libraries of whatever kind should be able to free every resource they request. It is no good style to forget such stuff. 4) Libgcrypt already has the appropriate command for it, only it is not used yet. 5) It's really straightforward and easy. It does not change the interface. It only enhances it. If you do not call gcry_control(GCRYCTL_FINALIZE, 0); nothing changes. If you call it you get the memory back, but can nevertheless start to use all the functions again. So finally: There is nothing speaking against it, but some points for it. Quoting a mail from July, 1th: >> 1. The problem is that when FN_gcry_pk_genkey is >> reached program is stopped, I get the following >> message: Detected memory leaks! and my library is >> deployed. > > I suppose this due to one-time allocation done by libgcrypt. There is > currently no way of getting rid of these one-time allocations. Well, now there would be a way to do so. > Dynamically removing libgcrypt from the process is not supported. I don't care for that. Only for the memory. But probably this gets a possibility afterwards as well, who knows? Ciao -- ____ _ _ ____ _ _ _ _ ____ | | | | | | \ / | | | the cool Gremlin from Bischofswerda | __ | ____| | \/ | | | WWW: http://www.dstoecker.de/ | | | | | | | | PGP key available on www page. |____| _|_ |____| _|_ _|_ |____| I hope AMIGA never stops making fun! From christianbiere at gmx.de Fri Sep 9 01:47:06 2005 From: christianbiere at gmx.de (Christian Biere) Date: Fri Sep 9 02:47:39 2005 Subject: libgcrypt fails to use /dev/random on NetBSD Message-ID: <20050908234706.GA23111@cyclonus> >Submitter-Id: net >Originator: Christian Biere >Confidential: no >Synopsis: libgcrypt fails to use /dev/random on NetBSD >Severity: serious >Priority: medium >Category: pkg >Class: sw-bug >Release: NetBSD 3.99.7 >Environment: System: NetBSD cyclonus 3.99.7 NetBSD 3.99.7 (STARSCREAM) #4: Sat Aug 13 17:11:37 CEST 2005 bin@cyclonus:/usr/obj/objdir/sys/arch/i386/compile/STARSCREAM i386 Architecture: i386 Machine: i386 >Description: The configure script of libgcrypt has "/dev/srandom" hardcoded for NetBSD instead of "/dev/random". This means libgcrypt will try to access a non-existing device at runtime and fall back to a replacement method to gather entropy which is likely to be (much) weaker. >How-To-Repeat: Compile libgcrypt and note that it reports that no random device was found: "checking for random device... no" It doesn't complain excessively though. configure will print a message suggesting to use EGD. However, it does not make obvious that the reason for showing this message is that the expected random device was not found. I've only noticed this incidently by tracing an application linked against libgcrypt when I my eyes caught an access attempt to "/dev/srandom" in the log output. >Fix: As a workaround, add a symbolic link /dev/srandom pointing to an existing /dev/random. This doesn't require recompiling libgcrypt. The attached patch against libgcrypt 1.2.1 should fix the bug in the configure script. diff -ur configure.orig configure --- configure.orig 2005-01-05 14:53:24.000000000 +0100 +++ configure 2005-09-09 01:13:26.000000000 +0200 @@ -2259,14 +2259,14 @@ case "${target}" in - *-openbsd* | *-netbsd*) + *-openbsd* ) # FIXME: Are these the best flags for OpenBSD? NAME_OF_DEV_RANDOM="/dev/srandom" NAME_OF_DEV_URANDOM="/dev/urandom" # DYNLINK_MOD_CFLAGS="-shared -rdynamic $CFLAGS_PIC -Wl,-Bshareable -Wl,-x" ;; - *-solaris* | *-irix* | *-dec-osf* ) + *-solaris* | *-irix* | *-dec-osf* | *-netbsd* ) NAME_OF_DEV_RANDOM="/dev/random" NAME_OF_DEV_URANDOM="/dev/random" # DYNLINK_MOD_CFLAGS="-shared $CFLAGS_PIC" diff -ur configure.ac.orig configure.ac --- configure.ac.orig 2005-01-05 14:52:33.000000000 +0100 +++ configure.ac 2005-09-09 01:10:30.000000000 +0200 @@ -187,14 +187,14 @@ case "${target}" in - *-openbsd* | *-netbsd*) + *-openbsd* ) # FIXME: Are these the best flags for OpenBSD? NAME_OF_DEV_RANDOM="/dev/srandom" NAME_OF_DEV_URANDOM="/dev/urandom" # DYNLINK_MOD_CFLAGS="-shared -rdynamic $CFLAGS_PIC -Wl,-Bshareable -Wl,-x" ;; - *-solaris* | *-irix* | *-dec-osf* ) + *-solaris* | *-irix* | *-dec-osf* | *-netbsd* ) NAME_OF_DEV_RANDOM="/dev/random" NAME_OF_DEV_URANDOM="/dev/random" # DYNLINK_MOD_CFLAGS="-shared $CFLAGS_PIC" From terrence at trainedmonkeystudios.org Fri Sep 9 19:53:30 2005 From: terrence at trainedmonkeystudios.org (terrence@trainedmonkeystudios.org) Date: Fri Sep 9 20:30:02 2005 Subject: libgcrypt, libgpg-error shared/dynamic libs under MinGW Message-ID: <46179.192.168.1.2.1126288410.squirrel@uber> Jim, Remember that in Windows there is no concept of a dynamic code library. Dll's have an entry point that must be present. This is the same error as "missing main" in a program, only for the dll entry point DllMain. See this for an explanation: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdkintro/sdkintro/calling_the_c_run_time_library_from_a_dll.asp And this for a detailed description of the callback: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/dllmain.asp For libgcrypt, this will just be a blank function, so you'll probably just want to put a block like this somewhere convenient: #ifdef WIN32 BOOL WINAPI DllMain( HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpReserved ) { } #endif -Terrence Cole www.trainedmonkeystudios.org p.s. I'll see what would be involved in supporting the native VC6 compiler at some point. From nmav at gnutls.org Sat Sep 10 11:12:42 2005 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sat Sep 10 12:17:41 2005 Subject: gnutls 1.2.6 and Mozilla Firefox compatibility problem In-Reply-To: <200509101034.23034.nmav@gnutls.org> References: <200509101034.23034.nmav@gnutls.org> Message-ID: <200509101112.42825.nmav@gnutls.org> On Saturday 10 September 2005 10:34, Nikos Mavrogiannopoulos wrote: > The problem is that in the 2nd forked session the server tries to resume > the previous connection. You can check this by looking the session ID. The > one the server selects the second time is the same as the client requested > (resume). This is totally strange since there is no communication > between the objects (lie in a different process), so the second process > shoudn't even know the session ID of the first server process. > It seems to work ok if you move the gnutls_session_t session declaration to > after the forked process has been created (after if (pid==0)). I'm still > looking at it but it really looks odd. The problem seems to be libgcrypt's random generator. As far as I understand when you fork() the random generator is on the same state for every children. That's why the server produces the same session ID in the second process. I am not really sure about it, and I don't know how to overcome this, that's why I crosspost to gcrypt-devel as well. -- Nikos Mavrogiannopoulos From christianbiere at gmx.de Sat Sep 10 17:39:57 2005 From: christianbiere at gmx.de (Christian Biere) Date: Sat Sep 10 17:40:34 2005 Subject: gnutls 1.2.6 and Mozilla Firefox compatibility problem In-Reply-To: <200509101112.42825.nmav@gnutls.org> References: <200509101034.23034.nmav@gnutls.org> <200509101112.42825.nmav@gnutls.org> Message-ID: <20050910153957.GA20524@cyclonus> Nikos Mavrogiannopoulos wrote: > On Saturday 10 September 2005 10:34, Nikos Mavrogiannopoulos wrote: > > The problem is that in the 2nd forked session the server tries to resume > > the previous connection. You can check this by looking the session ID. The > > one the server selects the second time is the same as the client requested > > (resume). This is totally strange since there is no communication > > between the objects (lie in a different process), so the second process > > shoudn't even know the session ID of the first server process. > > It seems to work ok if you move the gnutls_session_t session declaration to > > after the forked process has been created (after if (pid==0)). I'm still > > looking at it but it really looks odd. > The problem seems to be libgcrypt's random generator. As far as I understand > when you fork() the random generator is on the same state for every children. > That's why the server produces the same session ID in the second process. Either fork() much earlier or use exec*() after fork() to get a process in a clean state. -- Christian -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 186 bytes Desc: not available Url : /pipermail/attachments/20050910/1c787d6d/attachment.pgp From wk at gnupg.org Tue Sep 13 11:55:23 2005 From: wk at gnupg.org (Werner Koch) Date: Tue Sep 13 12:01:21 2005 Subject: gnutls 1.2.6 and Mozilla Firefox compatibility problem In-Reply-To: <200509101112.42825.nmav@gnutls.org> (Nikos Mavrogiannopoulos's message of "Sat, 10 Sep 2005 11:12:42 +0200") References: <200509101034.23034.nmav@gnutls.org> <200509101112.42825.nmav@gnutls.org> Message-ID: <877jdlgulw.fsf@wheatstone.g10code.de> On Sat, 10 Sep 2005 11:12:42 +0200, Nikos Mavrogiannopoulos said: > The problem seems to be libgcrypt's random generator. As far as I understand > when you fork() the random generator is on the same state for every children. I just checked it and indeed the fork detection code used to help only against other threads forking but not againts plain forks without execs. I have changed this and added fork detection to gcry_create_nonce too. Added regression tests for these two cases. Please checkout svn://cvs.gnupg.org/libgcrypt/branches/LIBGCRYPT-1-2-BRANCH this should fix the problem. If it is okay for you; I can do a new release today. Salam-Shalom, Werner