[gnutls-devel] GnuTLS | apparent bug in _gnutls_x509_der_encode with fix/workaround that shouldn't work (#1078)

[Read-only notification of GnuTLS library development activities]

CurtisVillamizar created an issue: https://gitlab.com/gnutls/gnutls/-/issues/1078



## Description of problem:

Attached is a fix/workaround to _gnutls_x509_der_encode (in lib/x509/common.c) that should not work but does.  The change provides a buffer to asn1_der_coding rather than zero when only the size is needed.  The suspected reason that this works is without the buffer asn1_der_coding is doing some inplace modification affecting later calls.  That reason is not proven.  A test case (c++ program) to reproduce the problem and shell output giving limited debug output is (or will be shortly) provided.  A prior version of gnutls worked.

## Version of gnutls used:

3.6.14 (on FreeBSD 12.1-STABLE #0 r363326M)

## Distributor of gnutls (e.g., Ubuntu, Fedora, RHEL)

FreeBSD ports collection (revision 542586) compiled with FreeBSD clang version 10.0.0

## How reproducible:

100% of the time with provided c++ program.  Fix works 100% of the time in cases where it should work.

Steps to Reproduce:

 * one   - read instructions in first few comment lines of provided test program
 * two   - follow directions to compile and run testcase 0 to 5 (only 1 and 5 *should not* produce core dump)
 * three - apply patch to gnutls and relink - testcase 1 and 5 now produce success

## Actual results:

Testcase 0-5 produce core dump without patch to gnutls.  Testcase 1 and 5 succeed with patch.

## Expected results:

Testcase 1 and 5 should succeed.  Other testcases exist to provide information and are OK to core dump.  But it would be nice (tm) if a few didn't fail.
[patch-common.c](/uploads/bda48bf8693ed2ae1655a4306d0349a2/patch-common.c)

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/gnutls/-/issues/1078
You're receiving this email because of your account on gitlab.com.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20200829/01af9b7a/attachment.html>