[PATCH] Rinjdael: Fix use of SSE2 outside USE_AESNI/ctx->use_aesni
Jussi Kivilinna
jussi.kivilinna at mbnet.fi
Mon Jan 28 10:11:33 CET 2013
* cipher/rijndael.c (_gcry_aes_cbc_enc): Check if AES-NI is enabled before
calling aesni_prepare() and aesni_cleanup().
--
aesni_cleanup() contains SSE2 instructions that are interpreted as MMX on CPUs
without SSE2 support (Pentium-III, etc). This causes x87 register state to be
poisoned, causing crashes later on when program tries to use floating point
registers.
Add '#ifdef USE_AESNI' and 'if (ctx->use_aesni)' for aesni_cleanup() and, while
at it, for aesni_prepare() too.
Reported-by: Mitsutoshi NAKANO <bkbin005 at rinku.zaq.ne.jp>
Signed-off-by: Jussi Kivilinna <jussi.kivilinna at mbnet.fi>
---
Note: this patch is for current development version of libgcrypt, it might not
apply cleanly to earlier stable releases.
---
cipher/rijndael.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/cipher/rijndael.c b/cipher/rijndael.c
index 24372d9..a3080f5 100644
--- a/cipher/rijndael.c
+++ b/cipher/rijndael.c
@@ -1399,7 +1399,11 @@ _gcry_aes_cbc_enc (void *context, unsigned char *iv,
unsigned char *outbuf = outbuf_arg;
const unsigned char *inbuf = inbuf_arg;
- aesni_prepare ();
+#ifdef USE_AESNI
+ if (ctx->use_aesni)
+ aesni_prepare ();
+#endif /*USE_AESNI*/
+
for ( ;nblocks; nblocks-- )
{
if (0)
@@ -1449,7 +1453,11 @@ _gcry_aes_cbc_enc (void *context, unsigned char *iv,
if (!cbc_mac)
outbuf += BLOCKSIZE;
}
- aesni_cleanup ();
+
+#ifdef USE_AESNI
+ if (ctx->use_aesni)
+ aesni_cleanup ();
+#endif /*USE_AESNI*/
_gcry_burn_stack (48 + 2*sizeof(int));
}
More information about the Gcrypt-devel
mailing list