Fix undefined behaviour in X509_NAME_cmp()

If the lengths of both names is 0 then don't attempt to do a memcmp.

Issue reported by Simon Friedberger, Robert Merget and Juraj Somorovsky.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6291)

diff --git a/crypto/x509/x509_cmp.c b/crypto/x509/x509_cmp.c
index 3d4931be4e142924dc208f136209a7e563502c91..67c187229eff22fff4e197ac32d4266171eef2cd 100644 (file)
--- a/crypto/x509/x509_cmp.c
+++ b/crypto/x509/x509_cmp.c
@@ -173,7 +173,7 @@ int X509_NAME_cmp(const X509_NAME *a, const X509_NAME *b)
 
     ret = a->canon_enclen - b->canon_enclen;
 
-    if (ret)
+    if (ret != 0 || a->canon_enclen == 0)
         return ret;
 
     return memcmp(a->canon_enc, b->canon_enc, a->canon_enclen);