Increase rounds of Miller-Rabin testing DH_check

DH_check is used to test the validity of Diffie-Hellman parameter sets (p, q, g). Among the tests performed are primality tests on p and q, for this BN_is_prime_ex is called with the rounds of Miller-Rabin set as default. This will therefore use the average case error estimates derived from the function BN_prime_checks_for_size based on the bit size of the number tested.

However, these bounds are only accurate on testing random input. Within this testing scenario, where we are checking the validity of a DH parameter set, we can not assert that these parameters are randomly generated. Thus we must treat them as if they are adversarial in nature and increase the rounds of Miller-Rabin performed.

Generally, each round of Miller-Rabin can declare a composite number prime with probability at most (1/4), thus 64 rounds is sufficient in thwarting known generation techniques (even in safe prime settings - see https://eprint.iacr.org/2019/032 for full analysis). The choice of 64 rounds is also consistent with SRP_NUMBER_ITERATIONS_FOR_PRIME 64 as used in srp_Verify_N_and_g in openssl/apps/s_client.c.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8593)

(cherry picked from commit 2500c093aa1e9c90c11c415053c0a27a00661d0d)

diff --git a/crypto/dh/dh_check.c b/crypto/dh/dh_check.c
index 52cc0eb47609d4d79487feb1042d2024e6558ca7..67b4cec5184ddc2e160a30083e19f6fb90da5776 100644 (file)
--- a/crypto/dh/dh_check.c
+++ b/crypto/dh/dh_check.c
@@ -12,6 +12,8 @@
 #include <openssl/bn.h>
 #include "dh_locl.h"
 
+# define DH_NUMBER_ITERATIONS_FOR_PRIME 64
+
 /*-
  * Check that p and g are suitable enough
  *
@@ -125,7 +127,7 @@ int DH_check(const DH *dh, int *ret)
             if (!BN_is_one(t1))
                 *ret |= DH_NOT_SUITABLE_GENERATOR;
         }
-        r = BN_is_prime_ex(dh->q, BN_prime_checks, ctx, NULL);
+        r = BN_is_prime_ex(dh->q, DH_NUMBER_ITERATIONS_FOR_PRIME, ctx, NULL);
         if (r < 0)
             goto err;
         if (!r)
@@ -153,7 +155,7 @@ int DH_check(const DH *dh, int *ret)
     } else
         *ret |= DH_UNABLE_TO_CHECK_GENERATOR;
 
-    r = BN_is_prime_ex(dh->p, BN_prime_checks, ctx, NULL);
+    r = BN_is_prime_ex(dh->p, DH_NUMBER_ITERATIONS_FOR_PRIME, ctx, NULL);
     if (r < 0)
         goto err;
     if (!r)
@@ -161,7 +163,7 @@ int DH_check(const DH *dh, int *ret)
     else if (!dh->q) {
         if (!BN_rshift1(t1, dh->p))
             goto err;
-        r = BN_is_prime_ex(t1, BN_prime_checks, ctx, NULL);
+        r = BN_is_prime_ex(t1, DH_NUMBER_ITERATIONS_FOR_PRIME, ctx, NULL);
         if (r < 0)
             goto err;
         if (!r)