Correct alignment calculation in ssl3_setup_write

The alignment calculation in ssl3_setup_write incorrectly results in an
alignment allowance of
(-SSL3_RT_HEADER_LENGTH) & (SSL3_ALIGN_PAYLOAD - 1) bytes. This equals 3
in almost all cases. The maximum alignment actually used in do_ssl3_write
is (SSL3_ALIGN_PAYLOAD - 1). This equals 7 bytes in almost all cases. So
there is a potential to overrun the buffer by up to 4 bytes.

Fortunately, the encryption overhead allowed for is 80 bytes which
consists of 16 bytes for the cipher block size and 64 bytes for the MAC
output. However the biggest MAC that we ever produce is HMAC-384 which is
48 bytes - so we have a headroom of 16 bytes (i.e. more than the 4 bytes
of potential overrun).

Thanks to Nagesh Hegde for reporting this.

Fixes #11766

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/11768)

diff --git a/ssl/record/ssl3_buffer.c b/ssl/record/ssl3_buffer.c
index 5e2cceee27d889bd9158211ac5228329b36b9a35..2c25099e10c045324996c703018fd089d5670d2a 100644 (file)
--- a/ssl/record/ssl3_buffer.c
+++ b/ssl/record/ssl3_buffer.c
@@ -94,7 +94,7 @@ int ssl3_setup_write_buffer(SSL *s, size_t numwpipes, size_t len)
             headerlen = SSL3_RT_HEADER_LENGTH;
 
 #if defined(SSL3_ALIGN_PAYLOAD) && SSL3_ALIGN_PAYLOAD!=0
-        align = (-SSL3_RT_HEADER_LENGTH) & (SSL3_ALIGN_PAYLOAD - 1);
+        align = SSL3_ALIGN_PAYLOAD - 1;
 #endif
 
         len = ssl_get_max_send_fragment(s)