Update control logic for BN_gcd
authorCesar Pereida Garcia <cesar.pereidagarcia@tut.fi>
Mon, 21 Oct 2019 11:41:01 +0000 (14:41 +0300)
committerNicola Tuveri <nic.tuv@gmail.com>
Wed, 23 Oct 2019 09:16:13 +0000 (12:16 +0300)
commit85728d08ae00f9b9305bee442988eb7d56ff1304
tree0979fc14bfaf318bcab21a66835f57bd5e1eb5d8
parent77f945bc9831862be7fdb35b438d494d054878c5
Update control logic for BN_gcd

PR https://github.com/openssl/openssl/pull/10122 introduced changes to
the BN_gcd function and the control logic inside it accessed `g->d[0]`
irrespective of `g->top`.

When BN_add is called, in case the result is zero, `BN_zero` is called.
The latter behaves differently depending on the API compatibility level
flag: normally `g->d[0]` is cleared but in `no-deprecated` builds only
`g->top` is set to zero.

This commit uses bitwise logic to ensure that `g` is treated as zero if
`g->top` is zero, irrespective of `g->d[0]`.

Co-authored-by: Nicola Tuveri <nic.tuv@gmail.com>
(cherry picked from commit 8aca4bfe8213402c80abc06fe25121461f79128d)

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10232)
crypto/bn/bn_gcd.c