gcry_cipher_encrypt() doesn't release all x87 FPU data registers .
jussi.kivilinna at mbnet.fi
Mon Jan 28 09:49:33 CET 2013
Quoting 1xx <itsango at gmail.com>:
> Hi all .
> I found a problem in aesni_cleanup() on rijndael.c .
> This macro uses MMX instruction, but doesn't call EMMS .
Those are SSE2 instructions, not MMX. So it does not touch x87
registers at all.
> Therefore, all x87 FPU data registers remains used,
> and after calling gcry_cipher_encrypt(),
> programs cannot use float data .
> So, gnome-keyring, Chromium and Chrome crash on some PCs .
> I want you to correct it .
> I wrote a patch .
> And I wrote a reappearance program .
> If you want them, please see
> https://bugs.launchpad.net/ubuntu/+source/libgcrypt11/+bug/1105758 .
Your bug report shows that you have Pentium-III. It does not support SSE2.
Since PXOR for MMX is encoded with '0F EF' and for SSE2 with '66 0F
EF', the SSE2 version runs as MMX version on CPUs without SSE2
support. The 0x66 byte at front of SSE2-opcode is operand-size
override prefix and is being ignored on your CPU.
Bug appears to be in _gcry_aes_cbc_enc(), aesni_cleanup() is called
outside '#ifdef USE_AESNI' and/or 'if (ctr->use_aesni)'. This works on
most (modern) CPUs, but not those without SSE2 support.
> Thanks .
> 1xx <ItSANgo at gmail.com>
> Gcrypt-devel mailing list
> Gcrypt-devel at gnupg.org
More information about the Gcrypt-devel