From n.mavrogiannopoulos at gmail.com Mon Nov 20 08:56:43 2017 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Mon, 20 Nov 2017 08:56:43 +0100 Subject: [gnutls-devel] support of stapled OCSP responses under TLS1.3 In-Reply-To: References: Message-ID: On Fri, Oct 13, 2017 at 1:50 PM, Nikos Mavrogiannopoulos wrote: > Hi, > I'm going through the support of stapled OCSP responses under TLS1.3. > The major change in TLS1.3 is that there can be an OCSP response for > each certificate sent, rather than one response for the > end-certificate, and such responses can be provided also for the > client certificate. > Supporting multiple responses when verifying the certificates seems > straightforward as we were doing that transparently without the > application intervening. [...] Hi, The merge request introducing multiple OCSP staples under TLS1.3 is at: https://gitlab.com/gnutls/gnutls/merge_requests/548 It tries hard not to require new APIs by enhancing gnutls_certificate_set_ocsp_status_request_file() to parse the response file and associate it with a certificate. On the other hand, a new callback could not be avoided to retrieve more than one responses, hence gnutls_certificate_set_ocsp_status_request_function3() is added, as well as gnutls_ocsp_status_request_get2() for application to read the responses. I'd appreciate a review on that new functionality if you are already familiar with the previous OCSP handling code, or intend to use it. After a discussion with Hubert Kario, I've also opened [0] which is about automating the retrieval of OCSP responses and association with server credentials, to reduce complexity from servers. It's currently a bit low priority in the tls1.3 plan [1], and up for grabs, but it would make an application server's code much simpler. regards, Nikos [0]. https://gitlab.com/gnutls/gnutls/issues/326 [1]. https://gitlab.com/gnutls/gnutls/milestones/8 From stbuehler at lighttpd.net Tue Nov 21 11:03:40 2017 From: stbuehler at lighttpd.net (=?UTF-8?Q?Stefan_B=c3=bchler?=) Date: Tue, 21 Nov 2017 11:03:40 +0100 Subject: [gnutls-devel] support of stapled OCSP responses under TLS1.3 In-Reply-To: References: Message-ID: Hi, On 11/20/2017 08:56 AM, Nikos Mavrogiannopoulos wrote: > On Fri, Oct 13, 2017 at 1:50 PM, Nikos Mavrogiannopoulos > wrote: >> Hi, >> I'm going through the support of stapled OCSP responses under TLS1.3. >> The major change in TLS1.3 is that there can be an OCSP response for >> each certificate sent, rather than one response for the >> end-certificate, and such responses can be provided also for the >> client certificate. >> Supporting multiple responses when verifying the certificates seems >> straightforward as we were doing that transparently without the >> application intervening. > [...] > > Hi, > The merge request introducing multiple OCSP staples under TLS1.3 is at: > https://gitlab.com/gnutls/gnutls/merge_requests/548 > > It tries hard not to require new APIs by enhancing > gnutls_certificate_set_ocsp_status_request_file() to parse the > response file and associate it with a certificate. On the other hand, > a new callback could not be avoided to retrieve more than one > responses, hence > gnutls_certificate_set_ocsp_status_request_function3() is added, as > well as gnutls_ocsp_status_request_get2() for application to read the > responses. As far as I can see gnutls_certificate_set_ocsp_status_request_function3 provides the necessary interface. > I'd appreciate a review on that new functionality if you are already > familiar with the previous OCSP handling code, or intend to use it. > > After a discussion with Hubert Kario, I've also opened [0] which is > about automating the retrieval of OCSP responses and association with > server credentials, to reduce complexity from servers. It's currently > a bit low priority in the tls1.3 plan [1], and up for grabs, but it > would make an application server's code much simpler. A first step to make usage simpler would be to add a function similar to gnutls_certificate_set_ocsp_status_request_file which accepts a gnutls_datum_t instead of a filename. And even simpler would be auto-loading these responses from "OCSP RESPONSE" pem blocks in certificate files (or separate "ocsp" files). (I made up the term "OCSP RESPONSE"; afaik there is no standard name for this). cheers, Stefan From nmav at gnutls.org Tue Nov 21 13:39:03 2017 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Tue, 21 Nov 2017 13:39:03 +0100 Subject: [gnutls-devel] support of stapled OCSP responses under TLS1.3 In-Reply-To: References: Message-ID: On Tue, Nov 21, 2017 at 11:03 AM, Stefan B?hler wrote: > Hi, > > On 11/20/2017 08:56 AM, Nikos Mavrogiannopoulos wrote: >> On Fri, Oct 13, 2017 at 1:50 PM, Nikos Mavrogiannopoulos >> wrote: >>> Hi, >>> I'm going through the support of stapled OCSP responses under TLS1.3. >>> The major change in TLS1.3 is that there can be an OCSP response for >>> each certificate sent, rather than one response for the >>> end-certificate, and such responses can be provided also for the >>> client certificate. >>> Supporting multiple responses when verifying the certificates seems >>> straightforward as we were doing that transparently without the >>> application intervening. >> [...] >> >> Hi, >> The merge request introducing multiple OCSP staples under TLS1.3 is at: >> https://gitlab.com/gnutls/gnutls/merge_requests/548 >> >> It tries hard not to require new APIs by enhancing >> gnutls_certificate_set_ocsp_status_request_file() to parse the >> response file and associate it with a certificate. On the other hand, >> a new callback could not be avoided to retrieve more than one >> responses, hence >> gnutls_certificate_set_ocsp_status_request_function3() is added, as >> well as gnutls_ocsp_status_request_get2() for application to read the >> responses. > > As far as I can see gnutls_certificate_set_ocsp_status_request_function3 > provides the necessary interface. > >> I'd appreciate a review on that new functionality if you are already >> familiar with the previous OCSP handling code, or intend to use it. >> >> After a discussion with Hubert Kario, I've also opened [0] which is >> about automating the retrieval of OCSP responses and association with >> server credentials, to reduce complexity from servers. It's currently >> a bit low priority in the tls1.3 plan [1], and up for grabs, but it >> would make an application server's code much simpler. > > A first step to make usage simpler would be to add a function similar to > gnutls_certificate_set_ocsp_status_request_file which accepts a > gnutls_datum_t instead of a filename. Thank you Stefan. If I remember well, my original intention for not bringing that function was that I assumed that OCSP responses are something volatile that will need to be updated now and then. As the credentials structure is supposed to be accessed concurrently, do you imply here having a set data buffer function which protects against concurrent access, or an "unsafe" set function which is expected to be used once, while later the credentials structure is recycled? (i.e., the server app, creates a new credentials structure for new sessions). > And even simpler would be auto-loading these responses from "OCSP > RESPONSE" pem blocks in certificate files (or separate "ocsp" files). > (I made up the term "OCSP RESPONSE"; afaik there is no standard name for > this). Is the use of such a file format something that is being used? As the OCSP response lifetime is much shorter than certificate lifecycle, I'd expect that combo to cause an inconvenience. regards, Nikos From stbuehler at lighttpd.net Tue Nov 21 14:17:55 2017 From: stbuehler at lighttpd.net (=?UTF-8?Q?Stefan_B=c3=bchler?=) Date: Tue, 21 Nov 2017 14:17:55 +0100 Subject: [gnutls-devel] support of stapled OCSP responses under TLS1.3 In-Reply-To: References: Message-ID: <7f05837b-997a-5db2-0d01-5e8f78508be7@lighttpd.net> Hi, On 11/21/2017 01:39 PM, Nikos Mavrogiannopoulos wrote: > On Tue, Nov 21, 2017 at 11:03 AM, Stefan B?hler wrote: >> Hi, >> >> On 11/20/2017 08:56 AM, Nikos Mavrogiannopoulos wrote: >>> On Fri, Oct 13, 2017 at 1:50 PM, Nikos Mavrogiannopoulos >>> wrote: >>>> Hi, >>>> I'm going through the support of stapled OCSP responses under TLS1.3. >>>> The major change in TLS1.3 is that there can be an OCSP response for >>>> each certificate sent, rather than one response for the >>>> end-certificate, and such responses can be provided also for the >>>> client certificate. >>>> Supporting multiple responses when verifying the certificates seems >>>> straightforward as we were doing that transparently without the >>>> application intervening. >>> [...] >>> >>> Hi, >>> The merge request introducing multiple OCSP staples under TLS1.3 is at: >>> https://gitlab.com/gnutls/gnutls/merge_requests/548 >>> >>> It tries hard not to require new APIs by enhancing >>> gnutls_certificate_set_ocsp_status_request_file() to parse the >>> response file and associate it with a certificate. On the other hand, >>> a new callback could not be avoided to retrieve more than one >>> responses, hence >>> gnutls_certificate_set_ocsp_status_request_function3() is added, as >>> well as gnutls_ocsp_status_request_get2() for application to read the >>> responses. >> >> As far as I can see gnutls_certificate_set_ocsp_status_request_function3 >> provides the necessary interface. >> >>> I'd appreciate a review on that new functionality if you are already >>> familiar with the previous OCSP handling code, or intend to use it. >>> >>> After a discussion with Hubert Kario, I've also opened [0] which is >>> about automating the retrieval of OCSP responses and association with >>> server credentials, to reduce complexity from servers. It's currently >>> a bit low priority in the tls1.3 plan [1], and up for grabs, but it >>> would make an application server's code much simpler. >> >> A first step to make usage simpler would be to add a function similar to >> gnutls_certificate_set_ocsp_status_request_file which accepts a >> gnutls_datum_t instead of a filename. > > Thank you Stefan. If I remember well, my original intention for not > bringing that function was that I assumed that OCSP responses are > something volatile that will need to be updated now and then. As the > credentials structure is supposed to be accessed concurrently, do you > imply here having a set data buffer function which protects against > concurrent access, or an "unsafe" set function which is expected to be > used once, while later the credentials structure is recycled? (i.e., > the server app, creates a new credentials structure for new sessions). My usecase so far is to simply restart the server after updating OCSP (we implemented an "angel" process that keeps the listening socket open, and it will wait for the new worker to be up before it stops the old one and so on, so restart doesn't cost much). So an "unsafe" set function is ok for me. Now that you need multiple files for multiple OCSP responses, gnutls_certificate_set_ocsp_status_request_file sounds even worse than before; you probably need a cronjob to update the files anyway, so if your server can handle a restart (or "reload ssl") well, it would be better to to have a "fixed response" interface. Add to this that if the file update is screwed (e.g. not atomic, or somehow fails to match the certificate it matched before), you'll have a hard time debugging it - either you won't see the error, or you'll get flooded with the same error... >> And even simpler would be auto-loading these responses from "OCSP >> RESPONSE" pem blocks in certificate files (or separate "ocsp" files). >> (I made up the term "OCSP RESPONSE"; afaik there is no standard name for >> this). > > Is the use of such a file format something that is being used? As the > OCSP response lifetime is much shorter than certificate lifecycle, I'd > expect that combo to cause an inconvenience. I used such file format (right now I'm back to a single file with DER); but mod_gnutls in lighttpd2 should support it. I think now that we might need many OCSP responses it will be easier to load multiple responses from one file, so a PEM encoding is useful imho. I also like having a way to put everything into one file; that way I can use a simple key-value lookup interface to load certs dynamically based on filenames (the backend could also use stat() for file-based lookup, or trigger ACME). But it would be sufficient if there is a generic "load_ocsp_responses_from_pem" which simply ignores other blocks, and perhaps makes sure all required responses are present and all responses match a loaded certificate. cheers, Stefan From nmav at gnutls.org Wed Nov 22 14:03:43 2017 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Wed, 22 Nov 2017 14:03:43 +0100 Subject: [gnutls-devel] support of stapled OCSP responses under TLS1.3 In-Reply-To: <7f05837b-997a-5db2-0d01-5e8f78508be7@lighttpd.net> References: <7f05837b-997a-5db2-0d01-5e8f78508be7@lighttpd.net> Message-ID: On Tue, Nov 21, 2017 at 2:17 PM, Stefan B?hler wrote: >>>>> I'm going through the support of stapled OCSP responses under TLS1.3. >>>>> The major change in TLS1.3 is that there can be an OCSP response for >>>>> each certificate sent, rather than one response for the >>>>> end-certificate, and such responses can be provided also for the >>>>> client certificate. >>>>> Supporting multiple responses when verifying the certificates seems >>>>> straightforward as we were doing that transparently without the >>>>> application intervening. >>>> [...] >>>> >>>> Hi, >>>> The merge request introducing multiple OCSP staples under TLS1.3 is at: >>>> https://gitlab.com/gnutls/gnutls/merge_requests/548 >>>> >>>> It tries hard not to require new APIs by enhancing >>>> gnutls_certificate_set_ocsp_status_request_file() to parse the >>>> response file and associate it with a certificate. On the other hand, >>>> a new callback could not be avoided to retrieve more than one >>>> responses, hence >>>> gnutls_certificate_set_ocsp_status_request_function3() is added, as >>>> well as gnutls_ocsp_status_request_get2() for application to read the >>>> responses. >>> >>> As far as I can see gnutls_certificate_set_ocsp_status_request_function3 >>> provides the necessary interface. >>> >>>> I'd appreciate a review on that new functionality if you are already >>>> familiar with the previous OCSP handling code, or intend to use it. >>>> >>>> After a discussion with Hubert Kario, I've also opened [0] which is >>>> about automating the retrieval of OCSP responses and association with >>>> server credentials, to reduce complexity from servers. It's currently >>>> a bit low priority in the tls1.3 plan [1], and up for grabs, but it >>>> would make an application server's code much simpler. >>> >>> A first step to make usage simpler would be to add a function similar to >>> gnutls_certificate_set_ocsp_status_request_file which accepts a >>> gnutls_datum_t instead of a filename. >> >> Thank you Stefan. If I remember well, my original intention for not >> bringing that function was that I assumed that OCSP responses are >> something volatile that will need to be updated now and then. As the >> credentials structure is supposed to be accessed concurrently, do you >> imply here having a set data buffer function which protects against >> concurrent access, or an "unsafe" set function which is expected to be >> used once, while later the credentials structure is recycled? (i.e., >> the server app, creates a new credentials structure for new sessions). > > My usecase so far is to simply restart the server after updating OCSP > (we implemented an "angel" process that keeps the listening socket open, > and it will wait for the new worker to be up before it stops the old one > and so on, so restart doesn't cost much). > > So an "unsafe" set function is ok for me. > > Now that you need multiple files for multiple OCSP responses, > gnutls_certificate_set_ocsp_status_request_file sounds even worse than > before; you probably need a cronjob to update the files anyway, so if > your server can handle a restart (or "reload ssl") well, it would be > better to to have a "fixed response" interface. > > Add to this that if the file update is screwed (e.g. not atomic, or > somehow fails to match the certificate it matched before), you'll have a > hard time debugging it - either you won't see the error, or you'll get > flooded with the same error... Thank you, you bring some nice points. >>> And even simpler would be auto-loading these responses from "OCSP >>> RESPONSE" pem blocks in certificate files (or separate "ocsp" files). >>> (I made up the term "OCSP RESPONSE"; afaik there is no standard name for >>> this). >> >> Is the use of such a file format something that is being used? As the >> OCSP response lifetime is much shorter than certificate lifecycle, I'd >> expect that combo to cause an inconvenience. > > I used such file format (right now I'm back to a single file with DER); > but mod_gnutls in lighttpd2 should support it. > > I think now that we might need many OCSP responses it will be easier to > load multiple responses from one file, so a PEM encoding is useful imho. > > I also like having a way to put everything into one file; that way I can > use a simple key-value lookup interface to load certs dynamically based > on filenames (the backend could also use stat() for file-based lookup, > or trigger ACME). > > But it would be sufficient if there is a generic > "load_ocsp_responses_from_pem" which simply ignores other blocks, and > perhaps makes sure all required responses are present and all responses > match a loaded certificate. Thank you Stefan, you brought some nice points. Inspired by them I generated another MR with different contents. That more aggressively changes behavior, by caching responses, but at the same time, being more "safe" in using these responses (e.g., not serving expired responses). That builds on top of the previous patch set, and is at: https://gitlab.com/gnutls/gnutls/merge_requests/551 regards, Nikos From n.mavrogiannopoulos at gmail.com Mon Nov 27 14:35:41 2017 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Mon, 27 Nov 2017 14:35:41 +0100 Subject: [gnutls-devel] Fwd: support of stapled OCSP responses under TLS1.3 In-Reply-To: References: Message-ID: [moving to mailing list - hope that's ok] On Mon, Nov 27, 2017 at 8:24 AM, Thomas Klute wrote: > Hi Nikos, > > The concept looks good to me, but from my mod_gnutls perspective I see two potential issues: > > 1) If I understand the comment in lib/tls13/certificate.c, > append_status_request() correctly stapling multiple certificates would > actually not work together with > gnutls_certificate_set_retrieve_function2, which mod_gnutls needs to use > for virtual hosts. The use of gnutls_certificate_set_retrieve_function2 remains unaffected. However, to take advantage of multiple responses, it would have to be modified to utilize gnutls_certificate_set_ocsp_status_request_function3. > 2) The certificate is passed to the callback as a gnutls_pcert_st. I'd > need to call gnutls_pcert_export_x509 during each run of the callback to > get a gnutls_x509_crt_t and access things like fingerprint (used for the > cache key) or OCSP responder URL. I'm not sure if that might hurt > performance. If you are storing the certificate as a gnutls_pcert_st internally, I'd also store the OCSP response in the same structure. E.g. struct int_store_st { gnutls_pcert_st *certs[]; gnutls_datum_t ocsp_responses[]; } where each response corresponds to each cert. When the callback from gnutls_certificate_set_ocsp_status_request_function3 is set, you'll get a gnutls_cert_info_st which has an index indicating the positioni n the chain the OCSP is asked for. With such a setup no additional parsing will be required. If I misread what you were describing, please let me know. > Regarding the first issue, I don't quite understand why the limitation > exists: The glob_ocsp_func is still tied to a single credentials > structure, so the stored ptr could still contain a reference to the > associated vhost or something like that. The OCSP client and caching > implementation in mod_gnutls needs just the following: > * Fingerprint as cache key > * Certificate itself to generate an OCSP request if there is no current > response in the cache > * Issuer certificate/trust chain to verify response > Looking at the gnutls_cert_info_st structure, these are almost there, > just one question: > >> typedef struct gnutls_cert_info_st { >> const struct gnutls_pcert_st *pcert; > > Is this only the certificate in question, or an array with cert_index > being the index? In the latter case the issuer cert is already > available, too, but there should be an array length parameter unless the > API guarantees that the callback is only called if the element > cert_index + 1 exists. It is a single certificate only. cert_index is the index for this certificate in the whole chain. >> unsigned cert_index; /* position in chain - zero being the end-certificate */ >> unsigned flags; >> } gnutls_cert_info_st; > > A validity check on the response returned by the callback can't replace > the issuer certificate for mod_gnutls, because an invalid response > should not end up the the cache. > > Internal caching seems needlessly complex to me, especially regarding > memory consistency, and with a multi-process structure like the Apache > server caches will quickly differ between processes. I would prefer an > interface similar to the session cache (gnutls_db_set_*_function), if it > is built to let the application update the cache independently. I want > to move to regularly scheduled OCSP response updates for mod_gnutls > anyway, so the structure I'm thinking of could be: > > a) Try to get OCSP response via cache retrieve function, use it if valid. > b) Otherwise, call ocsp_func callback to get a fresh response. > c) Store response in cache if valid (ideally asynchronously). > > If the application regularly updates the cache on its own, b) and c) > would never happen. The application being able to update the cache > whenever is also important to have fresh responses, because I've often > seen OCSP responses that are valid for multiple days. To encourage > simple applications to use stapling, GnuTLS could provide a memory-based > default implementation. The way I though of it, is that with the callback approach, gnutls will call 'gnutls_status_request_ocsp_func2' when an OCSP response is needed for a particular certificate. Application at that point could return a cached response or initiate/schedule (a) and (c) and use a cache outside gnutls' boundary. If I understand well, what you are suggesting is that gnutls does the caching in a "virtual" db which can be overriden by the application. We could even use the same db as set by the gnutls_db_set_store_function, and rely on another set of keys. That is, gnutls calls 'gnutls_status_request_ocsp_func2', parses the response given and stores it into db with expiration time set to the OCSP response expiration time. Subsequent gnutls sessions would use the db retrieve function to get the OCSP responses, until none is available when gnutls_status_request_ocsp_func2 will be called again. I see few advantages and disadvantages over that. 1. (adv) This consolidates the OCSP response usage of expiration time and validity across gnutls applications. 2. (disadv) This introduces a little more overhead for OCSP response obtaining by calling both the callback and the DB functions for each certificate in a chain. 3. (disadv) The semantics of renewal will be quite complex. gnutls should call gnutls_status_request_ocsp_func2 even before the OCSP response is expired to somehow indicate that it should be renewed without blocking the current session. Alternatively, we need a method to notify the caller on when to provide a fresher OCSP response. As we have seen in the handling of OCSP responses in existing servers (1) is quite important, however 3 is also not very trivial to address either. What do you think? > For OCSP response files, an equivalent to gnutls_ocsp_resp_import2 that > parses a file containing multiple responses into a list would be > helpful. The current mod_gnutls implementation just pushes the file > content into the internal cache, and a list would make it easy to do the > same for multiple certificates. Of course applications could do that on > their own, but it seems like a common use case. Do you mean something like gnutls_x509_crt_list_import2? That is a function which will import the responses from a memory buffer and store them as gnutls_ocsp_resp_t? regards, Nikos From thomas2.klute at uni-dortmund.de Tue Nov 28 07:53:16 2017 From: thomas2.klute at uni-dortmund.de (Thomas Klute) Date: Tue, 28 Nov 2017 07:53:16 +0100 Subject: [gnutls-devel] Fwd: support of stapled OCSP responses under TLS1.3 In-Reply-To: References: Message-ID: Am 27.11.2017 um 14:35 schrieb Nikos Mavrogiannopoulos: >> 1) If I understand the comment in lib/tls13/certificate.c, >> append_status_request() correctly stapling multiple certificates would >> actually not work together with >> gnutls_certificate_set_retrieve_function2, which mod_gnutls needs to use >> for virtual hosts. > > The use of gnutls_certificate_set_retrieve_function2 remains > unaffected. However, to take advantage of multiple responses, it would > have to be modified to utilize > gnutls_certificate_set_ocsp_status_request_function3. Ah, I might have misunderstood the API doc for gnutls_certificate_set_ocsp_status_request_function3 here: > * When the flag %GNUTLS_OCSP_CB_GLOBAL_SET is specified in @flags, this > * function can be used to set a callback that is used even when the > * certificates are provided by the application via a callback. That is, > * when gnutls_certificate_set_retrieve_function() and friends are used. > * In that case the callback will be called with the selected certificate. What is that different from setting the callback without GNUTLS_OCSP_CB_GLOBAL_SET? I would have expected the const gnutls_cert_info_st *cinfo parameter for the callback to refer to the certificate GnuTLS needs an OCSP response for either way. If it iterates over the chain it should work for certificates set via retrieve function just as well. > assert(session->internals.selected_ocsp_func != NULL); > > /* The global ocsp callback function can only be used to return > * a single certificate request */ > if (!session->internals.selected_ocsp_func && ctx->cert_index != 0) > return 0; This is the comment in append_status_request() I was referring to, but it appears I missed the preceding assert(). On the other hand, that should make the if condition always false, so I'm not sure what it is for. >> 2) The certificate is passed to the callback as a gnutls_pcert_st. I'd >> need to call gnutls_pcert_export_x509 during each run of the callback to >> get a gnutls_x509_crt_t and access things like fingerprint (used for the >> cache key) or OCSP responder URL. I'm not sure if that might hurt >> performance. > > If you are storing the certificate as a gnutls_pcert_st internally, > I'd also store the OCSP response in the same structure. E.g. > struct int_store_st { > gnutls_pcert_st *certs[]; > gnutls_datum_t ocsp_responses[]; > } > > where each response corresponds to each cert. When the callback from > gnutls_certificate_set_ocsp_status_request_function3 is set, you'll > get a gnutls_cert_info_st which has an index indicating the positioni > n the chain the OCSP is asked for. With such a setup no additional > parsing will be required. If I misread what you were describing, > please let me know. I was referring to how I might use the new API, not the current implementation. The current mod_gnutls implementation keeps certificate chains as both gnutls_pcert_st and gnutls_x509_crt_t arrays, and the OCSP responses in a key/value cache. My point was about using the fixed callback parameters instead of devising some way to pass the right certificate chain (or a way to find it) using the void *ptr parameter: If there is a current response in the cache, I need only the certificate fingerprint to return it. Another approach could be to add a certificate retrieve callback type that can return OCSP responses along with selected certificates, but that's also fairly complex. >> Regarding the first issue, I don't quite understand why the limitation >> exists: The glob_ocsp_func is still tied to a single credentials >> structure, so the stored ptr could still contain a reference to the >> associated vhost or something like that. The OCSP client and caching >> implementation in mod_gnutls needs just the following: >> * Fingerprint as cache key >> * Certificate itself to generate an OCSP request if there is no current >> response in the cache >> * Issuer certificate/trust chain to verify response >> Looking at the gnutls_cert_info_st structure, these are almost there, >> just one question: >> >>> typedef struct gnutls_cert_info_st { >>> const struct gnutls_pcert_st *pcert; >> >> Is this only the certificate in question, or an array with cert_index >> being the index? In the latter case the issuer cert is already >> available, too, but there should be an array length parameter unless the >> API guarantees that the callback is only called if the element >> cert_index + 1 exists. > > It is a single certificate only. cert_index is the index for this > certificate in the whole chain. > >>> unsigned cert_index; /* position in chain - zero being the end-certificate */ >>> unsigned flags; >>> } gnutls_cert_info_st; >> >> A validity check on the response returned by the callback can't replace >> the issuer certificate for mod_gnutls, because an invalid response >> should not end up the the cache. >> >> Internal caching seems needlessly complex to me, especially regarding >> memory consistency, and with a multi-process structure like the Apache >> server caches will quickly differ between processes. I would prefer an >> interface similar to the session cache (gnutls_db_set_*_function), if it >> is built to let the application update the cache independently. I want >> to move to regularly scheduled OCSP response updates for mod_gnutls >> anyway, so the structure I'm thinking of could be: >> >> a) Try to get OCSP response via cache retrieve function, use it if valid. >> b) Otherwise, call ocsp_func callback to get a fresh response. >> c) Store response in cache if valid (ideally asynchronously). >> >> If the application regularly updates the cache on its own, b) and c) >> would never happen. The application being able to update the cache >> whenever is also important to have fresh responses, because I've often >> seen OCSP responses that are valid for multiple days. To encourage >> simple applications to use stapling, GnuTLS could provide a memory-based >> default implementation. > > The way I though of it, is that with the callback approach, gnutls > will call 'gnutls_status_request_ocsp_func2' when an OCSP response is > needed for a particular certificate. Application at that point could > return a cached response or initiate/schedule (a) and (c) and use a > cache outside gnutls' boundary. If I understand well, what you are > suggesting is that gnutls does the caching in a "virtual" db which can > be overriden by the application. What you are describing is exactly the approach mod_gnutls is using right now, and the callback API is perfectly sufficient for that. > We could even use the same db as set by the > gnutls_db_set_store_function, and rely on another set of keys. That > is, gnutls calls 'gnutls_status_request_ocsp_func2', parses the > response given and stores it into db with expiration time set to the > OCSP response expiration time. Subsequent gnutls sessions would use > the db retrieve function to get the OCSP responses, until none is > available when gnutls_status_request_ocsp_func2 will be called again. > > I see few advantages and disadvantages over that. > 1. (adv) This consolidates the OCSP response usage of expiration time > and validity across gnutls applications. > 2. (disadv) This introduces a little more overhead for OCSP response > obtaining by calling both the callback and the DB functions for each > certificate in a chain. > 3. (disadv) The semantics of renewal will be quite complex. gnutls > should call gnutls_status_request_ocsp_func2 even before the OCSP > response is expired to somehow indicate that it should be renewed > without blocking the current session. Alternatively, we need a method > to notify the caller on when to provide a fresher OCSP response. > > As we have seen in the handling of OCSP responses in existing servers > (1) is quite important, however 3 is also not very trivial to address > either. What do you think? Updating the response cache via the callback is something I'd like to get rid of in mod_gnutls (details below), in particular because it triggers an OCSP request only if there is no longer a valid response in the cache. That way even a short OCSP responder outage prevents stapling. Updating the response cache regularly (and well before cache expiration) could hide such outages. I was just considering that *if* you want to add a response caching mechanism in GnuTLS itself for applications that do not want to implement their own cache, applications that do manage their own cache should be able to replace that GnuTLS cache: Basically let GnuTLS get responses directly from the application's cache using a retrieve function, let the application do the rest. My point is that the application should be able to update the OCSP responses in the cache periodically, without involving the callback. However, I personally doubt that the complexity is worth it. In case you're interested in the mod_gnutls approach: It currently does the same as Apache's mod_ssl, which is to update the cache only if there's a client request and no valid response in the cache. That's fairly simple, but leads to the problem described above, and delays the request that triggered the update. For servers with only occasional activity the request may also reveal when they are serving requests. Instead, I want to move to regularly scheduled updates outside of request handling and well before the cached response expires. The callback API works for that, I'd just like to have fast access to a unique cache key (currently the fingerprint). >> For OCSP response files, an equivalent to gnutls_ocsp_resp_import2 that >> parses a file containing multiple responses into a list would be >> helpful. The current mod_gnutls implementation just pushes the file >> content into the internal cache, and a list would make it easy to do the >> same for multiple certificates. Of course applications could do that on >> their own, but it seems like a common use case. > > Do you mean something like gnutls_x509_crt_list_import2? That is a > function which will import the responses from a memory buffer and > store them as gnutls_ocsp_resp_t? Exactly. I admit I didn't look at the tool diffs too closely, but it looked like they got support for that format, so it'd be useful if other applications could just reuse that to support multi-response files. Best regards, Thomas From n.mavrogiannopoulos at gmail.com Wed Nov 29 13:56:12 2017 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Wed, 29 Nov 2017 13:56:12 +0100 Subject: [gnutls-devel] Fwd: support of stapled OCSP responses under TLS1.3 In-Reply-To: References: Message-ID: On Tue, Nov 28, 2017 at 7:53 AM, Thomas Klute wrote: > Am 27.11.2017 um 14:35 schrieb Nikos Mavrogiannopoulos: >>> 1) If I understand the comment in lib/tls13/certificate.c, >>> append_status_request() correctly stapling multiple certificates would >>> actually not work together with >>> gnutls_certificate_set_retrieve_function2, which mod_gnutls needs to use >>> for virtual hosts. >> >> The use of gnutls_certificate_set_retrieve_function2 remains >> unaffected. However, to take advantage of multiple responses, it would >> have to be modified to utilize >> gnutls_certificate_set_ocsp_status_request_function3. > > Ah, I might have misunderstood the API doc for > gnutls_certificate_set_ocsp_status_request_function3 here: > >> * When the flag %GNUTLS_OCSP_CB_GLOBAL_SET is specified in @flags, this >> * function can be used to set a callback that is used even when the >> * certificates are provided by the application via a callback. That is, >> * when gnutls_certificate_set_retrieve_function() and friends are used. >> * In that case the callback will be called with the selected certificate. > > What is that different from setting the callback without > GNUTLS_OCSP_CB_GLOBAL_SET? I would have expected the const > gnutls_cert_info_st *cinfo parameter for the callback to refer to the > certificate GnuTLS needs an OCSP response for either way. If it iterates > over the chain it should work for certificates set via retrieve function > just as well. Without this flag this function will set a callback for the certificate chain in the credentials function specified by the index (idx). That is, without that flag it works only if gnutls_certificate_set_x509_key_file* or mem is used. >> assert(session->internals.selected_ocsp_func != NULL); >> >> /* The global ocsp callback function can only be used to return >> * a single certificate request */ >> if (!session->internals.selected_ocsp_func && ctx->cert_index != 0) >> return 0; > This is the comment in append_status_request() I was referring to, but > it appears I missed the preceding assert(). On the other hand, that > should make the if condition always false, so I'm not sure what it is for. The comment and the corresponding code seems incorrect. I'm checking it; thanks for the catch. >>> 2) The certificate is passed to the callback as a gnutls_pcert_st. I'd >>> need to call gnutls_pcert_export_x509 during each run of the callback to >>> get a gnutls_x509_crt_t and access things like fingerprint (used for the >>> cache key) or OCSP responder URL. I'm not sure if that might hurt >>> performance. >> >> If you are storing the certificate as a gnutls_pcert_st internally, >> I'd also store the OCSP response in the same structure. E.g. >> struct int_store_st { >> gnutls_pcert_st *certs[]; >> gnutls_datum_t ocsp_responses[]; >> } >> >> where each response corresponds to each cert. When the callback from >> gnutls_certificate_set_ocsp_status_request_function3 is set, you'll >> get a gnutls_cert_info_st which has an index indicating the positioni >> n the chain the OCSP is asked for. With such a setup no additional >> parsing will be required. If I misread what you were describing, >> please let me know. > > I was referring to how I might use the new API, not the current > implementation. The current mod_gnutls implementation keeps certificate > chains as both gnutls_pcert_st and gnutls_x509_crt_t arrays, and the > OCSP responses in a key/value cache. So is the existing information sufficient to figure the key, or is there something else we should add in gnutls_cert_info_st? > Another approach could be to add a certificate retrieve callback type > that can return OCSP responses along with selected certificates, but > that's also fairly complex. That could also be. I think the main reason a separate callback was used for the multiple OCSP responses is because there was already such a callback in place. Would that approach be simpler for mod_gnutls? > I was just considering that *if* you want to add a response caching > mechanism in GnuTLS itself for applications that do not want to > implement their own cache, applications that do manage their own cache > should be able to replace that GnuTLS cache: Basically let GnuTLS get > responses directly from the application's cache using a retrieve > function, let the application do the rest. My point is that the > application should be able to update the OCSP responses in the cache > periodically, without involving the callback. However, I personally > doubt that the complexity is worth it. I kind of agree here. >>> For OCSP response files, an equivalent to gnutls_ocsp_resp_import2 that >>> parses a file containing multiple responses into a list would be >>> helpful. The current mod_gnutls implementation just pushes the file >>> content into the internal cache, and a list would make it easy to do the >>> same for multiple certificates. Of course applications could do that on >>> their own, but it seems like a common use case. >> >> Do you mean something like gnutls_x509_crt_list_import2? That is a >> function which will import the responses from a memory buffer and >> store them as gnutls_ocsp_resp_t? > > Exactly. I admit I didn't look at the tool diffs too closely, but it > looked like they got support for that format, so it'd be useful if other > applications could just reuse that to support multi-response files. I'll check it. regards, Nikos