Don't treat a bare OCTETSTRING as DigestInfo in int_rsa_verify

The function int_rsa_verify is an internal function used for verifying an
RSA signature. It takes an argument |dtype| which indicates the digest type
that was used. Dependant on that digest type the processing of the
signature data will vary. In particular if |dtype == NID_mdc2| and the
signature data is a bare OCTETSTRING then it is treated differently to the
default case where the signature data is treated as a DigestInfo (X509_SIG).

Due to a missing "else" keyword the logic actually correctly processes the
OCTETSTRING format signature first, and then attempts to continue and
process it as DigestInfo. This will invariably fail because we already know
that it is a bare OCTETSTRING.

This failure doesn't actualy make a real difference because it ends up at
the |err| label regardless and still returns a "success" result. This patch
just cleans things up to make it look a bit more sane.

RT#4076

Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit dffe51091f412dcbc18f6641132f0b4f0def6bce)

diff --git a/crypto/rsa/rsa_sign.c b/crypto/rsa/rsa_sign.c
index bc91da2c1fe9cdaefee6843ea7e98583bafeb0a6..41c827f45313787a0b374e8ab1ef23ab09f7dbc2 100644 (file)
--- a/crypto/rsa/rsa_sign.c
+++ b/crypto/rsa/rsa_sign.c
@@ -218,14 +218,13 @@ int int_rsa_verify(int dtype, const unsigned char *m,
             memcpy(rm, s + 2, 16);
             *prm_len = 16;
             ret = 1;
-        } else if (memcmp(m, s + 2, 16))
+        } else if (memcmp(m, s + 2, 16)) {
             RSAerr(RSA_F_INT_RSA_VERIFY, RSA_R_BAD_SIGNATURE);
-        else
+        } else {
             ret = 1;
-    }
-
-    /* Special case: SSL signature */
-    if (dtype == NID_md5_sha1) {
+        }
+    } else if (dtype == NID_md5_sha1) {
+        /* Special case: SSL signature */
         if ((i != SSL_SIG_LENGTH) || memcmp(s, m, SSL_SIG_LENGTH))
             RSAerr(RSA_F_INT_RSA_VERIFY, RSA_R_BAD_SIGNATURE);
         else