Allow an ALPN callback to pretend to not exist

RFC 7301 mandates that the server SHALL respond with a fatal
"no_application_protocol" alert when there is no overlap between
the client's supplied list and the server's list of supported protocols.
In commit 062178678f5374b09f00d70796f6e692e8775aca we changed from
ignoring non-success returns from the supplied alpn_select_cb() to
treating such non-success returns as indicative of non-overlap and
sending the fatal alert.

In effect, this is using the presence of an alpn_select_cb() as a proxy
to attempt to determine whether the application has configured a list
of supported protocols.  However, there may be cases in which an
application's architecture leads it to supply an alpn_select_cb() but
have that callback be configured to take no action on connections that
do not have ALPN configured; returning SSL_TLSEXT_ERR_NOACK from
the callback would be the natural way to do so.  Unfortunately, the
aforementioned behavior change also treated SSL_TLSEXT_ERR_NOACK as
indicative of no overlap and terminated the connection; this change
supplies special handling for SSL_TLSEXT_ERR_NOACK returns from the
callback.  In effect, it provides a way for a callback to obtain the
behavior that would have occurred if no callback was registered at
all, which was not possible prior to this change.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3158)
(cherry picked from commit 8313a787d770ac1d7ddafcbc41b13e7fb5841eae)

diff --git a/doc/ssl/SSL_CTX_set_alpn_select_cb.pod b/doc/ssl/SSL_CTX_set_alpn_select_cb.pod
index 59acbad5458df2977ede7cffdf432c2bfa5fd373..d96185c4e2cc4c5e7187697ee7293e382a718487 100644 (file)
--- a/doc/ssl/SSL_CTX_set_alpn_select_cb.pod
+++ b/doc/ssl/SSL_CTX_set_alpn_select_cb.pod
@@ -113,9 +113,15 @@ The ALPN select callback B<cb>, must return one of the following:
 
 ALPN protocol selected.
 
+=item SSL_TLSEXT_ERR_ALERT_FATAL
+
+There was no overlap between the client's supplied list and the server
+configuration.
+
 =item SSL_TLSEXT_ERR_NOACK
 
-ALPN protocol not selected.
+ALPN protocol not selected, e.g., because no ALPN protocols are configured for
+this connection.
 
 =back
 

diff --git a/ssl/t1_lib.c b/ssl/t1_lib.c
index b3855d6406a5469f1ebd1bd5f78c303cf7e591fc..3f9702b9c2ddb541276c7b63cf188ed756fe7d3a 100644 (file)
--- a/ssl/t1_lib.c
+++ b/ssl/t1_lib.c
@@ -1799,6 +1799,9 @@ static int tls1_alpn_handle_client_hello_late(SSL *s, int *al)
             /* ALPN takes precedence over NPN. */
             s->s3->next_proto_neg_seen = 0;
 #endif
+        } else if (r == SSL_TLSEXT_ERR_NOACK) {
+            /* Behave as if no callback was present. */
+            return 1;
         } else {
             *al = SSL_AD_NO_APPLICATION_PROTOCOL;
             return 0;

diff --git a/test/handshake_helper.c b/test/handshake_helper.c
index 7ef015f8a468595b2c545653cbd494a569a17929..bb042bec50ce7fd0756700e260a3afe20dd654bc 100644 (file)
--- a/test/handshake_helper.c
+++ b/test/handshake_helper.c
@@ -297,7 +297,7 @@ static int server_alpn_cb(SSL *s, const unsigned char **out,
     *out = tmp_out;
     /* Unlike NPN, we don't tolerate a mismatch. */
     return ret == OPENSSL_NPN_NEGOTIATED ? SSL_TLSEXT_ERR_OK
-        : SSL_TLSEXT_ERR_NOACK;
+        : SSL_TLSEXT_ERR_ALERT_FATAL;
 }
 
 /*