OpenSSL 1.0.2 (starting from version 1.0.2b) introduced an "error state"
mechanism. The intent was that if a fatal error occurred during a handshake
then OpenSSL would move into the error state and would immediately fail if
you attempted to continue the handshake. This works as designed for the
explicit handshake functions (SSL_do_handshake(), SSL_accept() and
SSL_connect()), however due to a bug it does not work correctly if
SSL_read() or SSL_write() is called directly. In that scenario, if the
handshake fails then a fatal error will be returned in the initial function
call. If SSL_read()/SSL_write() is subsequently called by the application
for the same SSL object then it will succeed and the data is passed without
being decrypted/encrypted directly from the SSL/TLS record layer.
In order to exploit this issue an attacker would have to trick an
application into behaving incorrectly by issuing an SSL_read()/SSL_write()
after having already received a fatal error.
Thanks to David Benjamin (Google) for reporting this issue and suggesting
this fix.
CVE-2017-3737
Reviewed-by: Rich Salz <rsalz@openssl.org>
# define SSL_ST_BEFORE 0x4000
# define SSL_ST_OK 0x03
# define SSL_ST_RENEGOTIATE (0x04|SSL_ST_INIT)
-# define SSL_ST_ERR 0x05
+# define SSL_ST_ERR (0x05|SSL_ST_INIT)
# define SSL_CB_LOOP 0x01
# define SSL_CB_EXIT 0x02