[gnutls-devel] GnuTLS | x86(_64): CPU feature detection broken (#1282)

[Read-only notification of GnuTLS library development activities]

Gregor Jasny commented:


I hit the same bug in Ubuntu 22.04 (`3.7.3-4ubuntu1`) which is supposed to be fixed.

# environment

* Ubuntu 22.04
* libgnutls30:amd64 3.7.3-4ubuntu1

# testcase

```
gnutls-cli github.com:443
```

# dmesg

```
[20561.492164] traps: gnutls-cli[13879] trap invalid opcode ip:7f58f0bbc8b1 sp:7fffc5ac0500 error:0 in libgnutls.so.30.31.0[7f58f0a9b000+129000]
```

# backtrace (with ddebs installed)

```
#0  sha256_block_data_order_avx () at ../../../../lib/accelerated/x86/elf/sha256-ssse3-x86_64.s:3165
#1  0x00007ffff7ee9782 in x86_sha256_update (ctx=0x7fffffff77e0, length=64, data=0x7fffffff7720 '\\' <repeats 64 times>, "\300w\377\377\377\177")
    at accelerated/x86/../../../../lib/accelerated/x86/sha-x86-ssse3.c:164
#2  0x00007ffff776ecc2 in nettle_hmac_set_key () from /lib/x86_64-linux-gnu/libnettle.so.8
#3  0x00007ffff7ef1c1b in wrap_x86_hmac_fast (algo=<optimized out>, nonce=<optimized out>, nonce_size=<optimized out>, key=0x7ffff7f3f583, key_size=0, 
    text=0x7fffffff7b00, text_size=32, digest=0x555555608f18) at accelerated/x86/../../../../lib/accelerated/x86/hmac-x86-ssse3.c:294
#4  0x00007ffff7e1d355 in _gnutls_mac_fast (algorithm=<optimized out>, key=<optimized out>, keylen=<optimized out>, text=<optimized out>, textlen=<optimized out>, 
    digest=<optimized out>) at ../../lib/hash_int.c:167
#5  0x00007ffff7e58056 in _tls13_init_secret2 (prf=0x7ffff7fb21d0 <hash_algorithms.lto_priv+112>, psk=<optimized out>, psk_size=32, out=0x555555608f18)
    at ../../lib/secrets.c:59
#6  0x00007ffff7e10617 in _tls13_init_secret (psk=0x0, psk_size=0, session=0x555555607790) at ../../lib/secrets.c:35
#7  read_server_hello (datalen=<optimized out>, 
    data=0x5555558c6470 "\003\003\325\312`\342\333\361p\270\331gC~߲&\211Q\035\006\262so2=+XR\230c`\367\213 \213\306Ċ\346\064\"!\343\v^\211\254niq\261\374\225\265\252b\033m\351UA\325X\213'{\023\001", session=0x555555607790) at ../../lib/handshake.c:2097
#8  _gnutls_recv_handshake (session=0x555555607790, type=<optimized out>, optional=<optimized out>, buf=0x0) at ../../lib/handshake.c:1656
#9  0x00007ffff7e18d4a in handshake_client (session=0x555555607790) at ../../lib/handshake.c:3072
#10 gnutls_handshake (session=0x555555607790) at ../../lib/handshake.c:2871
#11 0x00005555555651e7 in ?? ()
#12 0x00005555555658d8 in ?? ()
#13 0x000055555555de2c in ?? ()
#14 0x00007ffff7addd90 in __libc_start_call_main (main=main at entry=0x55555555d5b0, argc=argc at entry=2, argv=argv at entry=0x7fffffffe458)
    at ../sysdeps/nptl/libc_start_call_main.h:58
#15 0x00007ffff7adde40 in __libc_start_main_impl (main=0x55555555d5b0, argc=2, argv=0x7fffffffe458, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7fffffffe448) at ../csu/libc-start.c:392
#16 0x000055555555ff85 in ?? ()
```

# disassembly

```
Dump of assembler code for function sha256_block_data_order_avx:
   0x00007ffff7f1c880 <+0>:	mov    %rsp,%rax
   0x00007ffff7f1c883 <+3>:	push   %rbx
   0x00007ffff7f1c884 <+4>:	push   %rbp
   0x00007ffff7f1c885 <+5>:	push   %r12
   0x00007ffff7f1c887 <+7>:	push   %r13
   0x00007ffff7f1c889 <+9>:	push   %r14
   0x00007ffff7f1c88b <+11>:	push   %r15
   0x00007ffff7f1c88d <+13>:	shl    $0x4,%rdx
   0x00007ffff7f1c891 <+17>:	sub    $0x60,%rsp
   0x00007ffff7f1c895 <+21>:	lea    (%rsi,%rdx,4),%rdx
   0x00007ffff7f1c899 <+25>:	and    $0xffffffffffffffc0,%rsp
   0x00007ffff7f1c89d <+29>:	mov    %rdi,0x40(%rsp)
   0x00007ffff7f1c8a2 <+34>:	mov    %rsi,0x48(%rsp)
   0x00007ffff7f1c8a7 <+39>:	mov    %rdx,0x50(%rsp)
   0x00007ffff7f1c8ac <+44>:	mov    %rax,0x58(%rsp)
=> 0x00007ffff7f1c8b1 <+49>:	vzeroupper 
   0x00007ffff7f1c8b4 <+52>:	mov    (%rdi),%eax
   0x00007ffff7f1c8b6 <+54>:	mov    0x4(%rdi),%ebx
   0x00007ffff7f1c8b9 <+57>:	mov    0x8(%rdi),%ecx
```

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/gnutls/-/issues/1282#note_991427169
You're receiving this email because of your account on gitlab.com.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20220614/363dbba8/attachment-0001.html>