Fix undefined behaviour in X509_NAME_cmp()

If the lengths of both names is 0 then don't attempt to do a memcmp.

Issue reported by Simon Friedberger, Robert Merget and Juraj Somorovsky.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6291)

(cherry picked from commit 511190b691183a1fb160e7e05e2974dc73cab0c6)

diff --git a/crypto/x509/x509_cmp.c b/crypto/x509/x509_cmp.c
index 49c71b91280e52ea1a94b2bf005dbb8db5b70f1e..2d82f8fa8b2a6c3660dad6ddf8fa88f14c279ecb 100644 (file)
--- a/crypto/x509/x509_cmp.c
+++ b/crypto/x509/x509_cmp.c
@@ -219,7 +219,7 @@ int X509_NAME_cmp(const X509_NAME *a, const X509_NAME *b)
 
     ret = a->canon_enclen - b->canon_enclen;
 
-    if (ret)
+    if (ret != 0 || a->canon_enclen == 0)
         return ret;
 
     return memcmp(a->canon_enc, b->canon_enc, a->canon_enclen);