Fix a text canonicalisation bug in CMS

Where a CMS detached signature is used with text content the text goes
through a canonicalisation process first prior to signing or verifying a
signature. This process strips trailing space at the end of lines, converts
line terminators to CRLF and removes additional trailing line terminators
at the end of a file. A bug in the canonicalisation process meant that
some characters, such as form-feed, were incorrectly treated as whitespace
and removed. This is contrary to the specification (RFC5485). This fix
could mean that detached text data signed with an earlier version of
OpenSSL 1.1.0 may fail to verify using the fixed version, or text data
signed with a fixed OpenSSL may fail to verify with an earlier version of
OpenSSL 1.1.0. A workaround is to only verify the canonicalised text data
and use the "-binary" flag (for the "cms" command line application) or set
the SMIME_BINARY/PKCS7_BINARY/CMS_BINARY flags (if using CMS_verify()).

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5791)

diff --git a/CHANGES b/CHANGES
index e15a28959a67e09d355fffbeeb4184ac7089db5a..7199f3d1961738a6e8709ba1c8de277b62c0304e 100644 (file)
--- a/CHANGES
+++ b/CHANGES
@@ -9,7 +9,22 @@
 
  Changes between 1.1.0h and 1.1.0i [xx XXX xxxx]
 
-  *)
+  *) Fixed a text canonicalisation bug in CMS
+
+     Where a CMS detached signature is used with text content the text goes
+     through a canonicalisation process first prior to signing or verifying a
+     signature. This process strips trailing space at the end of lines, converts
+     line terminators to CRLF and removes additional trailing line terminators
+     at the end of a file. A bug in the canonicalisation process meant that
+     some characters, such as form-feed, were incorrectly treated as whitespace
+     and removed. This is contrary to the specification (RFC5485). This fix
+     could mean that detached text data signed with an earlier version of
+     OpenSSL 1.1.0 may fail to verify using the fixed version, or text data
+     signed with a fixed OpenSSL may fail to verify with an earlier version of
+     OpenSSL 1.1.0. A workaround is to only verify the canonicalised text data
+     and use the "-binary" flag (for the "cms" command line application) or set
+     the SMIME_BINARY/PKCS7_BINARY/CMS_BINARY flags (if using CMS_verify()).
+     [Matt Caswell]
 
  Changes between 1.1.0g and 1.1.0h [27 Mar 2018]
 

diff --git a/crypto/asn1/asn_mime.c b/crypto/asn1/asn_mime.c
index 84475e9470d95ef284062f3008e65e50d2cf0152..da0085f680cd0cef81a29b12ed2ba50eb419773f 100644 (file)
--- a/crypto/asn1/asn_mime.c
+++ b/crypto/asn1/asn_mime.c
@@ -969,12 +969,14 @@ static int strip_eol(char *linebuf, int *plen, int flags)
     p = linebuf + len - 1;
     for (p = linebuf + len - 1; len > 0; len--, p--) {
         c = *p;
-        if (c == '\n')
+        if (c == '\n') {
             is_eol = 1;
-        else if (is_eol && flags & SMIME_ASCIICRLF && c < 33)
+        } else if (is_eol && flags & SMIME_ASCIICRLF && c == 32) {
+            /* Strip trailing space on a line; 32 == ASCII for ' ' */
             continue;
-        else if (c != '\r')
+        } else if (c != '\r') {
             break;
+        }
     }
     *plen = len;
     return is_eol;