Fix a possible crash in BN_from_montgomery_word

Thanks to Darovskikh Andrei for for reporting this issue.

Fixes: #5785
Fixes: #6302

Cherry-picked from f91e026e3832 (without test/bntest.c)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6310)

diff --git a/crypto/bn/bn_mont.c b/crypto/bn/bn_mont.c
index c0c174650cc5d3021a109cccb6d3720ad6ad6e35..dad3d07b3deb4eeb41d931ba209ce9b1d09103b0 100644 (file)
--- a/crypto/bn/bn_mont.c
+++ b/crypto/bn/bn_mont.c
@@ -95,6 +95,8 @@ static int BN_from_montgomery_word(BIGNUM *ret, BIGNUM *r, BN_MONT_CTX *mont)
 
     /* clear the top words of T */
     i = max - r->top;
+    if (i < 0)
+        return 0;
     if (i)
         memset(&rp[r->top], 0, sizeof(*rp) * i);