From: Richard Levitte <levitte@openssl.org>
Date: Fri, 1 Nov 2019 19:44:14 +0000 (+0100)
Subject: Fix OSSL_PARAM_set_BN() to fill the given buffer correctly.
X-Git-Tag: openssl-3.0.0-alpha1~1045
X-Git-Url: https://git.librecmc.org/?a=commitdiff_plain;h=0f73e719c6ca6c2e955e6c08a3ab171642dc2dc0;p=oweals%2Fopenssl.git

Fix OSSL_PARAM_set_BN() to fill the given buffer correctly.

OSSL_PARAM_set_BN() filled the buffer from the left with as many bytes
as that the BIGNUM takes, regardless of buffer size or native
endianness.  This was due to BN_bn2nativepad() being given the size of
the BIGNUM rather than the size of the buffer (which meant it never
had to pad anything).

The fix is to given BN_bn2nativepad() the size of the buffer instead.
This aligns well with the corresponding _set_ functions for native
integer types work.

Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
(Merged from https://github.com/openssl/openssl/pull/10326)
---

diff --git a/crypto/params.c b/crypto/params.c
index b2ceb13278..0cd13e3b81 100644
--- a/crypto/params.c
+++ b/crypto/params.c
@@ -640,8 +640,11 @@ int OSSL_PARAM_set_BN(OSSL_PARAM *p, const BIGNUM *val)
     p->return_size = bytes;
     if (p->data == NULL)
         return 1;
-    return p->data_size >= bytes
-        && BN_bn2nativepad(val, p->data, bytes) >= 0;
+    if (p->data_size >= bytes) {
+        p->return_size = p->data_size;
+        return BN_bn2nativepad(val, p->data, p->data_size) >= 0;
+    }
+    return 0;
 }
 
 OSSL_PARAM OSSL_PARAM_construct_BN(const char *key, unsigned char *buf,