[gnutls-devel] GnuTLS | gnutls server should not negotiate TLS 1.3 if the private key from PKCS#11 does not support RSA-PSS nor raw-RSA (#731)

[Development of GNU's TLS library]

New Issue was created.

Issue 731: https://gitlab.com/gnutls/gnutls/issues/731
Author:    Anderson Sasaki
Assignee:  

## Description of problem:
When server applications use GnuTLS for TLS with server key in a PKCS#11 device, the library should check if the PKCS#11 module supports RSA-PSS or raw-RSA mechanisms before negotiating TLS 1.3 and failing without any mechanisms to try (or blindly trying unsupported mechanisms).

Originally reported in: https://bugzilla.redhat.com/show_bug.cgi?id=1681274

## Version of gnutls used:
3.6.5

## Distributor of gnutls (e.g., Ubuntu, Fedora, RHEL)
RHEL

## How reproducible:
always

Steps to Reproduce:
 * Prerequisites:
   * PKCS#11 module without RSA-PSS nor raw-RSA mechanisms support.
 * Run GnuTLS test server (but it should be reproducible with any other server using GnuTLS) using the key in the PKCS#11 device:
```
gnutls-serv --http --x509certfile="pkcs11:token=SomeDevice;object=cert;type=cert" --x509keyfile="pkcs11:token=SomeDevice;object=key;type=private?pin-value=111111" -d 9
```
 *  Try to connect to the https port with TLS 1.3 capable client:
```
wget --no-check-certificate --tries 1 https://localhost:5556/
```

## Actual results:
TLS 1.3 connection fails without any ciphersuites that could be used:
```
|<2>| checking c0.9e (GNUTLS_DHE_RSA_AES_128_CCM) for compatibility
|<2>| checking 00.33 (GNUTLS_DHE_RSA_AES_128_CBC_SHA1) for compatibility
|<3>| ASSERT: ciphersuites.c[_gnutls_figure_common_ciphersuite]:1587
|<3>| ASSERT: handshake.c[_gnutls_server_select_suite]:1079
|<3>| ASSERT: handshake.c[read_client_hello]:800
|<3>| ASSERT: handshake.c[_gnutls_recv_handshake]:1545
|<3>| ASSERT: handshake.c[handshake_server]:3389
Error in handshake: No supported cipher suites have been found.
|<5>| REC: Sending Alert[2|40] - Handshake failed
|<5>| REC[0x55ed12486eb0]: Preparing Packet Alert(21) with length: 2 and min pad: 0
|<9>| ENC[0x55ed12486eb0]: cipher: NULL, MAC: MAC-NULL, Epoch: 0
|<5>| REC[0x55ed12486eb0]: Sent Packet[1] Alert(21) in epoch 0 and length: 7
```

## Expected results:
The library should downgrade the TLS version to 1.2 if it can not provide valid signature for TLS 1.3

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/gnutls/issues/731
You're receiving this email because of your account on gitlab.com.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20190314/13b6ede7/attachment.html>