[gnutls-devel] GnuTLS | certtool --generate-privkey Program received signal SIGILL, Illegal instruction when needle is built with x86-sha-ni (#1496)

[Read-only notification of GnuTLS library development activities]

Giuseppe Foti created an issue: https://gitlab.com/gnutls/gnutls/-/issues/1496



## Description of problem:
I run the command `certtool --generate-privkey` on a LXD container with a gentoo image.
The program fails with:
`Illegal instruction (core dumped).`

Running certtool on gdb the error received is:

```
Program received signal SIGILL, Illegal instruction.
0x00007ffff784a8dd in _nettle_sha256_compress_n () from /usr/lib64/libnettle.so.8
```

libnettle.so.8 was built on the same container that built version 3.9.1 of dev-libs/nettle using this ebuild:
https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-libs/nettle/nettle-3.9.1.ebuild
and with those CPU_FLAGS_X86 enabled: aes pclmul sha
(read here for hints about what CPU_FLAGS_ are in gentoo: https://wiki.gentoo.org/wiki/CPU_FLAGS_*)

After rebuilding dev-libs/nettle-3.9.1 disabling the "sha" CPU_FLAGS_X86 certtool works as expected
**The cpu_flags_x86_sha passes the x86-sha-ni arg to the compiler**

## Version of gnutls used:
3.8.0
https://packages.gentoo.org/packages/net-libs/gnutls
https://gitweb.gentoo.org/repo/gentoo.git/tree/net-libs/gnutls/gnutls-3.8.0.ebuild

## Distributor of gnutls (e.g., Ubuntu, Fedora, RHEL)
Gentoo

## How reproducible:

Steps to Reproduce:

 * build https://git.lysator.liu.se/nettle/nettle/-/tags/nettle_3.9.1_release_20230601 with x86-sha-ni ARG on configure
 * build gnutls-3.8.0
 * run `certtool --generate-privkey`

## Actual results:
The program fails with Illegal instruction (core dumped).

## Expected results:
Private Key generated

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/gnutls/-/issues/1496
You're receiving this email because of your account on gitlab.com.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20230807/a00d1447/attachment.html>