[PATCH] Rinjdael: Fix use of SSE2 outside USE_AESNI/ctx->use_aesni

[1xx]

Hi all .
I applied this patch to the libgcrypt11-1.5.0.
And I resumed tests of gnome-keyring-3.6.1 "make check" .
The tests failed again .
I investigated failure and found a leak of the patch .
I wrote further patch .
Please confirm it .

I made all "make check" tests of libgcrypt11-1.5.0
and gnome-keyring successful now on Pentium III-M .

Thanks .


2013/1/28 Jussi Kivilinna <jussi.kivilinna at mbnet.fi>:
> * cipher/rijndael.c (_gcry_aes_cbc_enc): Check if AES-NI is enabled before
> calling aesni_prepare() and aesni_cleanup().
> --
>
> aesni_cleanup() contains SSE2 instructions that are interpreted as MMX on CPUs
> without SSE2 support (Pentium-III, etc). This causes x87 register state to be
> poisoned, causing crashes later on when program tries to use floating point
> registers.
>
> Add '#ifdef USE_AESNI' and 'if (ctx->use_aesni)' for aesni_cleanup() and, while
> at it, for aesni_prepare() too.
>
> Reported-by: Mitsutoshi NAKANO <bkbin005 at rinku.zaq.ne.jp>
> Signed-off-by: Jussi Kivilinna <jussi.kivilinna at mbnet.fi>
>
> ---
>
> Note: this patch is for current development version of libgcrypt, it might not
> apply cleanly to earlier stable releases.

--
1xx <ItSANgo at gmail.com>


-------- diff -u  rijndael.c.orig2 rijndael.c --------
--- rijndael.c.orig2    2013-01-29 08:50:34.910273459 +0900
+++ rijndael.c  2013-01-29 14:46:14.304715698 +0900
@@ -1609,7 +1609,11 @@
       inbuf += BLOCKSIZE;
       outbuf += BLOCKSIZE;
     }
-  aesni_cleanup ();
+
+#ifdef USE_AESNI
+  if (ctx->use_aesni)
+    aesni_cleanup ();
+#endif /*USE_AESNI*/

   _gcry_burn_stack (48 + 2*sizeof(int) + BLOCKSIZE + 4*sizeof (char*));
 }