[gnutls-devel] Fwd: support of stapled OCSP responses under TLS1.3

[Thomas Klute]

Am 27.11.2017 um 14:35 schrieb Nikos Mavrogiannopoulos:
>> 1) If I understand the comment in lib/tls13/certificate.c,
>> append_status_request() correctly stapling multiple certificates would
>> actually not work together with
>> gnutls_certificate_set_retrieve_function2, which mod_gnutls needs to use
>>  for virtual hosts.
> 
> The use of gnutls_certificate_set_retrieve_function2 remains
> unaffected. However, to take advantage of multiple responses, it would
> have to be modified to utilize
> gnutls_certificate_set_ocsp_status_request_function3.

Ah, I might have misunderstood the API doc for
gnutls_certificate_set_ocsp_status_request_function3 here:

>  * When the flag %GNUTLS_OCSP_CB_GLOBAL_SET is specified in @flags, this
>  * function can be used to set a callback that is used even when the
>  * certificates are provided by the application via a callback. That is,
>  * when gnutls_certificate_set_retrieve_function() and friends are used.
>  * In that case the callback will be called with the selected certificate.

What is that different from setting the callback without
GNUTLS_OCSP_CB_GLOBAL_SET? I would have expected the const
gnutls_cert_info_st *cinfo parameter for the callback to refer to the
certificate GnuTLS needs an OCSP response for either way. If it iterates
over the chain it should work for certificates set via retrieve function
just as well.

> 	assert(session->internals.selected_ocsp_func != NULL);
> 
> 	/* The global ocsp callback function can only be used to return
> 	 * a single certificate request */
> 	if (!session->internals.selected_ocsp_func && ctx->cert_index != 0)
> 		return 0;

This is the comment in append_status_request() I was referring to, but
it appears I missed the preceding assert(). On the other hand, that
should make the if condition always false, so I'm not sure what it is for.

>> 2) The certificate is passed to the callback as a gnutls_pcert_st. I'd
>> need to call gnutls_pcert_export_x509 during each run of the callback to
>> get a gnutls_x509_crt_t and access things like fingerprint (used for the
>> cache key) or OCSP responder URL. I'm not sure if that might hurt
>> performance.
> 
> If you are storing the certificate as a gnutls_pcert_st internally,
> I'd also store the OCSP response in the same structure. E.g.
> struct int_store_st {
>   gnutls_pcert_st *certs[];
>   gnutls_datum_t ocsp_responses[];
> }
> 
> where each response corresponds to each cert. When the callback from
> gnutls_certificate_set_ocsp_status_request_function3 is set, you'll
> get a gnutls_cert_info_st which has an index indicating the positioni
> n the chain the OCSP is asked for. With such a setup no additional
> parsing will be required. If I misread what you were describing,
> please let me know.

I was referring to how I might use the new API, not the current
implementation. The current mod_gnutls implementation keeps certificate
chains as both gnutls_pcert_st and gnutls_x509_crt_t arrays, and the
OCSP responses in a key/value cache.

My point was about using the fixed callback parameters instead of
devising some way to pass the right certificate chain (or a way to find
it) using the void *ptr parameter: If there is a current response in the
cache, I need only the certificate fingerprint to return it.

Another approach could be to add a certificate retrieve callback type
that can return OCSP responses along with selected certificates, but
that's also fairly complex.

>> Regarding the first issue, I don't quite understand why the limitation
>> exists: The glob_ocsp_func is still tied to a single credentials
>> structure, so the stored ptr could still contain a reference to the
>> associated vhost or something like that. The OCSP client and caching
>> implementation in mod_gnutls needs just the following:
>> * Fingerprint as cache key
>> * Certificate itself to generate an OCSP request if there is no current
>> response in the cache
>> * Issuer certificate/trust chain to verify response
>> Looking at the gnutls_cert_info_st structure, these are almost there,
>> just one question:
>>
>>> typedef struct gnutls_cert_info_st {
>>>       const struct gnutls_pcert_st *pcert;
>>
>> Is this only the certificate in question, or an array with cert_index
>> being the index? In the latter case the issuer cert is already
>> available, too, but there should be an array length parameter unless the
>> API guarantees that the callback is only called if the element
>> cert_index + 1 exists.
> 
> It is a single certificate only. cert_index is the index for this
> certificate in the whole chain.
> 
>>>       unsigned cert_index; /* position in chain - zero being the end-certificate */
>>>       unsigned flags;
>>> } gnutls_cert_info_st;
>>
>> A validity check on the response returned by the callback can't replace
>> the issuer certificate for mod_gnutls, because an invalid response
>> should not end up the the cache.
>>
>> Internal caching seems needlessly complex to me, especially regarding
>> memory consistency, and with a multi-process structure like the Apache
>> server caches will quickly differ between processes. I would prefer an
>> interface similar to the session cache (gnutls_db_set_*_function), if it
>> is built to let the application update the cache independently. I want
>> to move to regularly scheduled OCSP response updates for mod_gnutls
>> anyway, so the structure I'm thinking of could be:
>>
>> a) Try to get OCSP response via cache retrieve function, use it if valid.
>> b) Otherwise, call ocsp_func callback to get a fresh response.
>> c) Store response in cache if valid (ideally asynchronously).
>>
>> If the application regularly updates the cache on its own, b) and c)
>> would never happen. The application being able to update the cache
>> whenever is also important to have fresh responses, because I've often
>> seen OCSP responses that are valid for multiple days. To encourage
>> simple applications to use stapling, GnuTLS could provide a memory-based
>> default implementation.
> 
> The way I though of it, is that with the callback approach, gnutls
> will call 'gnutls_status_request_ocsp_func2' when an OCSP response is
> needed for a particular certificate. Application at that point could
> return a cached response or initiate/schedule (a) and (c) and use a
> cache outside gnutls' boundary. If I understand well, what you are
> suggesting is that gnutls does the caching in a "virtual" db which can
> be overriden by the application.

What you are describing is exactly the approach mod_gnutls is using
right now, and the callback API is perfectly sufficient for that.

> We could even use the same db as set by the
> gnutls_db_set_store_function, and rely on another set of keys. That
> is, gnutls calls 'gnutls_status_request_ocsp_func2', parses the
> response given and stores it into db with expiration time set to the
> OCSP response expiration time. Subsequent gnutls sessions would use
> the db retrieve function to get the OCSP responses, until none is
> available when gnutls_status_request_ocsp_func2 will be called again.
> 
> I see few advantages and disadvantages over that.
> 1. (adv) This consolidates the OCSP response usage of expiration time
> and validity across gnutls applications.
> 2. (disadv) This introduces a little more overhead for OCSP response
> obtaining by calling both the callback and the DB functions for each
> certificate in a chain.
> 3. (disadv) The semantics of renewal will be quite complex. gnutls
> should call gnutls_status_request_ocsp_func2 even before the OCSP
> response is expired to somehow indicate that it should be renewed
> without blocking the current session. Alternatively, we need a method
> to notify the caller on when to provide a fresher OCSP response.
> 
> As we have seen in the handling of OCSP responses in existing servers
> (1) is quite important, however 3 is also not very trivial to address
> either. What do you think?

Updating the response cache via the callback is something I'd like to
get rid of in mod_gnutls (details below), in particular because it
triggers an OCSP request only if there is no longer a valid response in
the cache. That way even a short OCSP responder outage prevents
stapling. Updating the response cache regularly (and well before cache
expiration) could hide such outages.

I was just considering that *if* you want to add a response caching
mechanism in GnuTLS itself for applications that do not want to
implement their own cache, applications that do manage their own cache
should be able to replace that GnuTLS cache: Basically let GnuTLS get
responses directly from the application's cache using a retrieve
function, let the application do the rest. My point is that the
application should be able to update the OCSP responses in the cache
periodically, without involving the callback. However, I personally
doubt that the complexity is worth it.

In case you're interested in the mod_gnutls approach: It currently does
the same as Apache's mod_ssl, which is to update the cache only if
there's a client request and no valid response in the cache. That's
fairly simple, but leads to the problem described above, and delays the
request that triggered the update. For servers with only occasional
activity the request may also reveal when they are serving requests.
Instead, I want to move to regularly scheduled updates outside
of request handling and well before the cached response expires. The
callback API works for that, I'd just like to have fast access to a
unique cache key (currently the fingerprint).

>> For OCSP response files, an equivalent to gnutls_ocsp_resp_import2 that
>> parses a file containing multiple responses into a list would be
>> helpful. The current mod_gnutls implementation just pushes the file
>> content into the internal cache, and a list would make it easy to do the
>> same for multiple certificates. Of course applications could do that on
>> their own, but it seems like a common use case.
> 
> Do you mean something like gnutls_x509_crt_list_import2? That is a
> function which will import the responses from a memory buffer and
> store them as gnutls_ocsp_resp_t?

Exactly. I admit I didn't look at the tool diffs too closely, but it
looked like they got support for that format, so it'd be useful if other
applications could just reuse that to support multi-response files.

Best regards,
Thomas