Return an error from BN_mod_inverse if n is 1 (or -1)
authorMatt Caswell <matt@openssl.org>
Fri, 27 Apr 2018 16:36:11 +0000 (17:36 +0100)
committerMatt Caswell <matt@openssl.org>
Thu, 3 May 2018 09:14:12 +0000 (10:14 +0100)
commitb1860d6c71733314417d053a72af66ae72e8268e
tree0dd7cac671060990c58addb98f72c29b87c7dc4c
parent4db296d9f0cf2855b358883a55b77a6b6f6848ba
Return an error from BN_mod_inverse if n is 1 (or -1)

Calculating BN_mod_inverse where n is 1 (or -1) doesn't make sense. We
should return an error in that case. Instead we were returning a valid
result with value 0.

Fixes #6004

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6119)
crypto/bn/bn_gcd.c
crypto/bn/bn_mont.c