doc/man3: fix types taken by HMAC(), HMAC_Update()

HMAC() and HMAC_Update() take size_t for 'n' and 'len' respectively.

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12264)

(cherry picked from commit cc63865f336e0144f8501aa0a862ba0247a50622)

Fix wrong return value check of mmap function

The mmap function never returns NULL. If an error occurs, the function returns MAP_FAILED.

CLA: trivial

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12187)

(cherry picked from commit 1d78129dd205e3e85083a91c33540a70c51b0a23)

Refactor BN_R_NO_INVERSE logic in internal functions

Closes #12129

As described in https://github.com/openssl/openssl/issues/12129 the
readability of the internal functions providing the two alternative
implementations for `BN_mod_inverse()` is a bit lacking.

Both these functions are now completely internal, so we have the
flexibility needed to slightly improve readability and remove
unnecessary NULL checks.

The main changes here are:
- rename `BN_mod_inverse_no_branch()` as `bn_mod_inverse_no_branch()`:
  this function is `static` so it is not even visible within the rest of
  libcrypto. By convention upcase prefixes are reserved for public
  functions.
- remove `if (pnoinv == NULL)` checks in `int_bn_mod_inverse()`: this
  function is internal to the BN module and we can guarantee that all
  callers pass non-NULL arguments.
- `bn_mod_inverse_no_branch()` takes an extra `int *pnoinv` argument, so
  that it can signal if no inverse exists for the given inputs: in this
  way the caller is in charge of raising `BN_R_NO_INVERSE` as it is the
  case for the non-consttime implementation of `int_bn_mod_inverse()`.
- `BN_mod_inverse()` is a public function and must guarantee that the
  internal functions providing the actual implementation receive valid
  arguments. If the caller passes a NULL `BN_CTX` we create a temporary
  one for internal use.
- reorder function definitions in `crypto/bn/bn_gcd.c` to avoid forward
  declaration of `static` functions (in preparation for inlining)
- inline `bn_mod_inverse_no_branch()`.

(Backport to 1.1.1 from https://github.com/openssl/openssl/pull/12142)
(cherry picked from commit 5d8b3a3ef2941b8822523742a0408ca6896aa65d)

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12169)

Make it clear that you can't use all ciphers for CMAC

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12107)

Add a CMAC test

We did not have a test of the low level CMAC APIs so we add one. This is
heavily based on the HMAC test.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12107)

Correctly handle the return value from EVP_Cipher() in the CMAC code

EVP_Cipher() is a very low level routine that directly calls the
underlying cipher function. It's return value semantics are very odd.
Depending on the type of cipher 0 or -1 is returned on error. We should
just check for <=0 for a failure.

Fixes #11957

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12107)

Ensure we never use a partially initialised CMAC_CTX

If the CMAC_CTX is partially initialised then we make a note of this so
that future operations will fail if the initialisation has not been
completed.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12107)

use safe primes in ssl_get_auto_dh()

DH_get_1024_160() and DH_get_2048_224() return parameters from
RFC5114. Those parameters include primes with known small subgroups,
making them unsafe. Change the code to use parameters from
RFC 2409 and RFC 3526 instead (group 2 and 14 respectively).

This patch also adds automatic selection of 4096 bit params for 4096 bit
RSA keys

backport of 7646610

Signed-off-by: Hubert Kario <hkario@redhat.com>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12160)

CMS_get0_signers() description

CLA: trivial

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12055)

(cherry picked from commit 9ac916c7529a21cd01d1b539362abf8402719e30)

EVP: allow empty strings to EVP_Decode* functions

This is a simple check order correction.

Fixes #12143

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12144)

(cherry picked from commit 0800288e6e1d9f44d471043a970ba57743ca8f4c)

doc: Random spellchecking

A little spell checking.

Backport of commit
  af0d413654d19 ("doc: Random spellchecking")

Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
GH: #12075

Do not allow dropping Extended Master Secret extension on renegotiaton

Abort renegotiation if server receives client hello with Extended Master
Secret extension dropped in comparison to the initial session.

Fixes #9754

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12099)

Test genpkey app for EC keygen with various args

This commit adds a new recipe to test EC key generation with the
`genpkey` CLI app.

For each built-in curve, it tests key generation with text output, in
PEM and in DER format, using `explicit` and `named_curve` for parameters
encoding.

The list of built-in curves is static at the moment, as this allows to
differentiate between prime curves and binary curves to avoid failing
when ec2m is disabled.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/12085)

Silence gcc false positive warning on alpn_protos_len in test/handshake_helper.c

Fixes #12033

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/12041)

Silence gcc false positive warning on refdatalen in test/tls13encryptiontest.c

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/12041)

Fix err checking and mem leaks of BIO_set_conn_port and BIO_set_conn_address

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12036)

Replace BUF_strdup() call by OPENSSL_strdup() adding failure check in bss_acpt.c

Add OPENSSL_strdup failure check to cpt_ctrl() in bss_acpt.c

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12036)

[crypto/ec] Remove unreachable AVX2 code in NISTZ256 implementation

`crypto/ec/ecp_nistz256.c` contained code sections guarded by a
`ECP_NISTZ256_AVX2` define.

The relevant comment read:

> /*
>  * Note that by default ECP_NISTZ256_AVX2 is undefined. While it's great
>  * code processing 4 points in parallel, corresponding serial operation
>  * is several times slower, because it uses 29x29=58-bit multiplication
>  * as opposite to 64x64=128-bit in integer-only scalar case. As result
>  * it doesn't provide *significant* performance improvement. Note that
>  * just defining ECP_NISTZ256_AVX2 is not sufficient to make it work,
>  * you'd need to compile even asm/ecp_nistz256-avx.pl module.
>  */

Without diminishing the quality of the original submission, it's evident
that this code has been basically unreachable without modifications to
the library source code and is under-tested.

This commit removes these sections from the codebase.

(cherry picked from commit 00da0f69890874feaa555fafb99b967b861e9118 ,
 backported from https://github.com/openssl/openssl/pull/12019 )

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/12046)

Fix a typo in SSL_CTX_set_session_ticket_cb.pod

"SSL" takes two esses, not three.

[skip ci]

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12038)

(cherry picked from commit 09527c493596060544bda92ecd0d8ef40a366c5e)

enable DECLARE_DEPRECATED macro for Oracle Developer Studio compiler

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9575)

Fix a buffer overflow in drbg_ctr_generate

This can happen if the 32-bit counter overflows
and the last block is not a multiple of 16 bytes.

Fixes #12012

[extended tests]

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/12016)

(cherry picked from commit 42fa3e66697baa121220b4eacf03607280e4ff89)

fix a docs typo

Correct "EC_KEY_point2buf" to "EC_POINT_point2buf". The former does not exist.

CLA: trivial

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11988)

(cherry picked from commit a5a87011baeef71c86938a2bae54f89fbe99e5dc)

Prevent extended tests run unexpectedly in appveyor

Reason turns out that "git log -2" is picking up a merge
commit and a random commit message from the master branch.

Restore the expected behavior by using
git log -1 $env:APPVEYOR_PULL_REQUEST_HEAD_COMMIT

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11981)

(cherry picked from commit d805b83166538907535862372c16ff6ceb648b21)

Revert the check for NaN in %f format

Unfortunately -Ofast seems to break that check.

Fixes #11994

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12003)

(cherry picked from commit 41dccd68b9b9b7622b26d264c5fa190aa5bd4201)

Make BIO_do_connect() and friends handle multiple IP addresses

Backport of #11971

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/11989)

bio printf: Avoid using rounding errors in range check

There is a problem casting ULONG_MAX to double which clang-10 is warning about.
ULONG_MAX typically cannot be exactly represented as a double.  ULONG_MAX + 1
can be and this fix uses the latter, however since ULONG_MAX cannot be
represented exactly as a double number we subtract 65535 from this number,
and the result has at most 48 leading one bits, and can therefore be
represented as a double integer without rounding error.  By adding
65536.0 to this number we achive the correct result, which should avoid the
warning.

The addresses a symptom of the underlying problem: we print doubles via an
unsigned long integer.  Doubles have a far greater range and should be printed
better.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11955)

(cherry picked from commit 082c041b4233b17b80129d4ac6b33a28014442b0)

EVP_EncryptInit.pod: fix example

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11976)

(cherry picked from commit d561b84143f5e7956454090e15de0c5e1425ceac)

AES CTR-DRGB: performance improvement

Optimize the the AES-based implementation of the CTR_DRBG
construction, see 10.2.1 in [1].
Due to the optimizations, the code may deviate (more) from the
pseudocode in [1], but it is functional equivalence being decisive
for compliance:

"All DRBG mechanisms and algorithms are described in this document
in pseudocode, which is intended to explain functionality.
The pseudocode is not intended to constrain real-world
implementations." [9 in [1]].

The following optimizations are done:

- Replace multiple plain AES encryptions by a single AES-ECB
  encryption of a corresponding pre-initialized buffer, where
  possible.
  This allows platform-specific AES-ECB support to
  be used and reduces the overhead of multiple EVP calls.

- Replace the generate operation loop (which is a counter
  increment followed by a plain AES encryption) by a
  loop which does a plain AES encryption followed by
  a counter increment. The latter loop is just a description
  of AES-CTR, so we replace it by a single AES-CTR
  encryption.
  This allows for platform-specific AES-CTR support to be used
  and reduces the overhead of multiple EVP calls.
  This change, that is, going from a pre- to a post- counter
  increment, requires the counter in the internal state
  to be kept at "+1" (compared to the pseudocode in [1])
  such that it is in the correct state, when a generate
  operation is called.
  That in turn also requires all other operations to be
  changed from pre- to post-increment to keep functional
  equivalence.

[1] NIST SP 800-90A Revision 1

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(cherry picked from commit 28bdbe1aaa474ae8cd83e520d02e463e46ce89d9)

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/11968)

Avoid undefined behavior with unaligned accesses

Fixes: #4983

[extended tests]

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11781)

More testing for CLI usage of Ed25519 and Ed448 keys

Add testing for the `req` app and explicit conversion tests similar to
what is done for ECDSA keys.

The included test keys for Ed25519 are from the examples in RFC 8410
(Sec. 10)

The key for Ed448 is derived from the first of the test vectors in
RFC 8032 (Sec. 7.4) using OpenSSL to encode it into PEM format.

(cherry picked from commit 81722fdf2e01cfa71c46abbcc19e65aa003e083f)

This is originally a cherry-pick from
https://github.com/openssl/openssl/pull/10410, with trivial changes from
the original commit to account for the differences in 1.1.1.

Fixes #10687

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11939)

More testing for sign/verify through `dgst`

Add tests for signature generation and verification with `dgst` CLI for
common key types:
- RSA
- DSA
- ECDSA

(cherry picked from commit ef1e59ed833e8ed1d5f4de5b0c734da8561890e3)

This is a backport from https://github.com/openssl/openssl/pull/10410.
Support for testing EdDSA through `pkeyutl` was dropped as the required
`-rawin` option is not supported in 1.1.1.

Fixes #10687

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11939)

Coverity 1463830: Resource leaks (RESOURCE_LEAK)

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/11941)

(cherry picked from commit bbc3c22c0e2b3b4b6f069712dc8322a48506b775)

Fix B<..> vs. I<..> and add two remarks in OSSL_STORE_open.pod

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11912)

Allow NULL arg to OSSL_STORE_close()

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11912)

There is no -signreq option in CA.pl

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11897)

Prevent use after free of global_engine_lock

If buggy application calls engine functions after cleanup of engines
already happened the global_engine_lock will be used although
already freed.

See for example:
https://bugzilla.redhat.com/show_bug.cgi?id=1831086

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/11896)

(cherry picked from commit e12813d0d31f4f7be2ccc592d382ef3e94bdb842)

STORE: Make try_decode_PrivateKey() ENGINE aware

This function only considered the built-in and application
EVP_PKEY_ASN1_METHODs, and is now amended with a loop that goes
through all loaded engines, using whatever table of methods they each
have.

Fixes #11861

(cherry picked from commit b84439b06a1b9a7bfb47e230b70a6d3ee46e8a19)

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11887)

Fix d2i_PrivateKey() to work as documented

d2i_PrivateKey() is documented to return keys of the type given as
first argument |type|, unconditionally.  Most specifically, the manual
says this:

> An error occurs if the decoded key does not match type.

However, when faced of a PKCS#8 wrapped key, |type| was ignored, which
may lead to unexpected results.

(cherry picked from commit b2952366dd0248bf35c83e1736cd203033a22378)

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11888)

t1_trce: Fix remaining places where the 24 bit shift overflow happens

[extended tests]

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11857)

(cherry picked from commit 7486c718e54cc762edc5f1c7c526ab83d0f97ef7)

Avoid potential overflow to the sign bit when shifting left 24 places

Although there are platforms where int is 64 bit, 2GiB large BIGNUMs
instead of 4GiB should be "big enough for everybody".

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11857)

(cherry picked from commit 1d05eb55caa8965a151360c2469c463ecd990987)

Cast the unsigned char to unsigned int before shifting left

This is needed to avoid automatic promotion to signed int.

Fixes #11853

[extended tests]

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11857)

(cherry picked from commit cbeb0bfa961412eebfbdf1e72900f05527e81e15)

Fix egd and devrandom source configs

./config --with-rand-seed=egd

need to defines OPENSSL_RAND_SEED_EGD and OPENSSL_NO_EGD
so get rid of OPENSSL_NO_EGD (compiles but I did not really test EGD)

./config --with-rand-seed=devrandom

does not work since wait_random_seeded works under the assumption
that OPENSSL_RAND_SEED_GETRANDOM is supposed to be enabled as well,
that is usually the case, but not when only devrandom is enabled.
Skip the wait code in this special case.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11848)

(cherry picked from commit ddec332f329a432a45c0131d83f3bfb46114532b)

Update early data exchange scenarios in doc

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/11816)

(cherry picked from commit b2a5001d954e81e2a582f2a935212ab554a3cbbe)

Update limitation of psk_client_cb and psk_server_cb in usage with TLSv1.3

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/11816)

(cherry picked from commit e0bcb4f97f7496af032013ead15b7472b60e85fa)

Fix some places where X509_up_ref is used
without error handling.

This takes up the ball from #11278
without trying to solve everything at once.

[extended tests]

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11852)

TTY_get() in crypto/ui/ui_openssl.c open_console() can also return errno 1 (EPERM, Linux)

Signed-off-by: Maxim Zakharov <5158255+Maxime2@users.noreply.github.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11767)

(cherry picked from commit 082394839ea32386abc7ee33aaa9da864287064c)

Test TLSv1.3 out-of-band PSK with all 5 ciphersuites

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11809)

Fix crash in early data send with out-of-band PSK using AES CCM

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11809)

If SOURCE_DATE_EPOCH is defined, use it for copyright year

Using the date from SOURCE_DATE_EPOCH instead of the current date makes
it possible to reproduce a build that was built on a different year:
https://reproducible-builds.org/specs/source-date-epoch/

This is fixing an issue we had while building Tor Browser:
https://trac.torproject.org/projects/tor/ticket/33535

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/11296)

(cherry picked from commit 11d7d903447ab866d037fb8bba4ceb49c7d89191)

Correct alignment calculation in ssl3_setup_write

The alignment calculation in ssl3_setup_write incorrectly results in an
alignment allowance of
(-SSL3_RT_HEADER_LENGTH) & (SSL3_ALIGN_PAYLOAD - 1) bytes. This equals 3
in almost all cases. The maximum alignment actually used in do_ssl3_write
is (SSL3_ALIGN_PAYLOAD - 1). This equals 7 bytes in almost all cases. So
there is a potential to overrun the buffer by up to 4 bytes.

Fortunately, the encryption overhead allowed for is 80 bytes which
consists of 16 bytes for the cipher block size and 64 bytes for the MAC
output. However the biggest MAC that we ever produce is HMAC-384 which is
48 bytes - so we have a headroom of 16 bytes (i.e. more than the 4 bytes
of potential overrun).

Thanks to Nagesh Hegde for reporting this.

Fixes #11766

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/11768)

(cherry picked from commit d30ef639647ad263d09740c931a5bfb5a8b6a5f6)

Configure: Avoid SIXTY_FOUR_BIT for linux-mips64

This is a 32-bit ABI build (as opposed to linux64-mips64).
Setting SIXTY_FOUR_BIT breaks hardware optimizations, at least on
octeon processors.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11154)

Fix rsa8192.pem

Q: How did I do that?

A: That's a long story.

Precondition: I used sage 8.1 for the math, it could probably
done with simple python as well but I did not try.

First I extract numbers from rsa8192.pem:

openssl rsa -in rsa8192.pem -noout -text | sed "s/://g; s/ //g;"

cut&paste the numbers into sage:

modulus
00890d9fd57e81b5ed43283d0ea020
4a1229333d6fb9c37a179375b09c4f
7b5b1cf2eb025979b6d90b709928a0
6725e04caf2b0f7fe94afbdf9f3fa5
66f1ba75c2f6dc488039f410eb5fa8
ab152b8cfdb76791bb853059438edf
ae56bc70a32a9f3e2d883e8b751d08
3797999dc81a9c4d6bdb3a75362fd1
d9c497cf5028dfcdd4cc3eb318e79f
c0db45cbeed955da8a447f0872dee5
65bde4013340e767731441fae4fa54
51356bfbc84e1271b39f111f5f8ef3
a6c8973765b39addef80306194f4ea
89fdfc8e9744866323f6936de89b2f
e2741578b8eb3c41676702fabc50ec
c376e6b7b6e7f94e7d7b5c1bab3c9f
23bb0c8f04d8aca64c309fc063c406
553e1c1421cc45060df7f48c49f5c5
b459d572e273402d6a3ff008657fe9
1936714d1823c5cad53d80630b3216
9bf70feb2ebc1af6a35ee0bf059aed
49c4e367d567e130e2846859b271fd
a8949b182e050819866b8e762ed29f
fb3f7ca14cebfc2488662be4b3980f
c8d31890a05f38ae9690cc7d9d3efc
4808e03da104a8c28bb480bb814995
a6e8b8978ab8350d90b3894e3abf7d
c4ad0956335752c8d6944b38a1715e
7d9950f49e6cdba171fbe651a2ca26
65a7c70b6e8cf3a02c2f93dad8aa95
06481cdb032d04082a5a6c6a733b65
20fa80e2ef57b9cf858ca5ea11e084
bc31a386fc6b099f069786207f80d6
1f2bef294400d59394ad1006431366
a54ae09b0ecd3377dcd8af8fde9b94
fd559b0b7adc5113ba66fc4b3dc842
ee562cfcfd39b4ffc31576635873fc
59535b7aa98605772436c251834e23
4fb2347cc970a49818cac2a9ee95eb
b55fa2da66edd53e11245c6732140a
ae41491288cbf462eef8a807b46d0d
affa38d9ccfe8033d2d4a3cf5c5b82
9df12183f7a05d3650153cd317a017
083ac641c2c3ad11305de0a032be45
c439bd7bbbe3cb97850f9d2c66f72a
4a66e9d434544fc6d294ca3c92627b
e518bfa44e3017ac8ad9c0a26a227d
2e8677da0a4de8edb53ac9530adb63
83c72dbf562dc4d0fea4e492f09eb1
74548381a8686db3aeaaa3a9960cff
25e8c64701115da54fa7a1fb2c566a
fcb4b2a63268d818c3391a62885d13
41b3492c4f0167291b3d026a44e68c
02f2d4d255d4c0906b92a2ced0c0bb
f2bcdceaec1189895af4232dc386c9
75bf3477e5a70d3ab0ac0e5dc37024
0e34a276b155d5e290f77416a1986d
ec47f8c78236ac7df249df9ba21a80
2e6bd75b4fb1c6ffe0f4cf548761a5
6a1fcccee156523a718987f3fdaedc
7171c9050db89a83f24c5a283695b9
c28de6d3b69fc1714b0add335a0ce6
fbbdbd0bbdb01e44969d775105bba3
d2947dca2f291250f9b851e76f514d
dc5a3aa4498e6521314991568860eb
ff1258d8b4aee9ee4159153684c0c0
16c60b17537a50b53cd59aad60678b
d73f0714ab4ccae7416bab417b4907
36d59b2e9f

I used echo `echo "<paste>" ` | sed "s/ //g"
to get everything in one line, put that again
into the clipboard and
then start sage, type N=0x<paste><CR>

sage: N=0x00890d9fd57e81b5ed43283d0ea0204a1229333d6fb9c37a179375b09c4f7b5b1cf2eb025979b6d90b709928a06725e04caf2b0f7fe94afbdf9f3fa566f1ba75c2f6dc488039f410eb5fa8ab152b8cfdb76791bb853059438edfae56bc70a32a9f3e2d883e8b751d083797999dc81a9c4d6bdb3a75362fd1d9c497cf5028dfcdd4cc3eb318e79fc0db45cbeed955da8a447f0872dee565bde4013340e767731441fae4fa5451356bfbc84e1271b39f111f5f8ef3a6c8973765b39addef80306194f4ea89fdfc8e9744866323f6936de89b2fe2741578b8eb3c41676702fabc50ecc376e6b7b6e7f94e7d7b5c1bab3c9f23bb0c8f04d8aca64c309fc063c406553e1c1421cc45060df7f48c49f5c5b459d572e273402d6a3ff008657fe91936714d1823c5cad53d80630b32169bf70feb2ebc1af6a35ee0bf059aed49c4e367d567e130e2846859b271fda8949b182e050819866b8e762ed29ffb3f7ca14cebfc2488662be4b3980fc8d31890a05f38ae9690cc7d9d3efc4808e03da104a8c28bb480bb814995a6e8b8978ab8350d90b3894e3abf7dc4ad0956335752c8d6944b38a1715e7d9950f49e6cdba171fbe651a2ca2665a7c70b6e8cf3a02c2f93dad8aa9506481cdb032d04082a5a6c6a733b6520fa80e2ef57b9cf858ca5ea11e084bc31a386fc6b099f069786207f80d61f2bef294400d59394ad1006431366a54ae09b0ecd3377dcd8af8fde9b94fd559b0b7adc5113ba66fc4b3dc842ee562cfcfd39b4ffc31576635873fc59535b7aa98605772436c251834e234fb2347cc970a49818cac2a9ee95ebb55fa2da66edd53e11245c6732140aae41491288cbf462eef8a807b46d0daffa38d9ccfe8033d2d4a3cf5c5b829df12183f7a05d3650153cd317a017083ac641c2c3ad11305de0a032be45c439bd7bbbe3cb97850f9d2c66f72a4a66e9d434544fc6d294ca3c92627be518bfa44e3017ac8ad9c0a26a227d2e8677da0a4de8edb53ac9530adb6383c72dbf562dc4d0fea4e492f09eb174548381a8686db3aeaaa3a9960cff25e8c64701115da54fa7a1fb2c566afcb4b2a63268d818c3391a62885d1341b3492c4f0167291b3d026a44e68c02f2d4d255d4c0906b92a2ced0c0bbf2bcdceaec1189895af4232dc386c975bf3477e5a70d3ab0ac0e5dc370240e34a276b155d5e290f77416a1986dec47f8c78236ac7df249df9ba21a802e6bd75b4fb1c6ffe0f4cf548761a56a1fcccee156523a718987f3fdaedc7171c9050db89a83f24c5a283695b9c28de6d3b69fc1714b0add335a0ce6fbbdbd0bbdb01e44969d775105bba3d2947dca2f291250f9b851e76f514ddc5a3aa4498e6521314991568860ebff1258d8b4aee9ee4159153684c0c016c60b17537a50b53cd59aad60678bd73f0714ab4ccae7416bab417b490736d59b2e9f

likewise for prime1 (P), prime2 (Q) and
privateExponent (D) and publicExponent (E)

sage: P=0x00c6d6eb9c0f93f8b8de74b2799798c1c1ed47549426a0d76af61f04f0b455184acfeec354c1703709e8727ea0213001738e395b5ca5ed4deec30f8b1ae0e54a516777c5ae897a5562e77dfe378dbba32ee30bcdb275ab7f87f8b9af02712d371a6aa671b44e0f667a6f2d406e5099d942f592ff5c517d452d58cf3f51120062a878fb77066650275b897caf078a5353f1d714963ff90818b6b87c6db438681b4b315c4fd256c1bd383fcf8e67b49a3ee077373a66d77e4c02eebe9350478a51cdfa395d60d905c1189ed7041373f1609075faf9d9f39d3d7b0e7fc8b1d36cb8c5c48731ef5cd87e11f480cdfa4b3ee6cf7eebd3e13f4244647396d52030b82c39d69a3a4075ee90531d1ddb5f1284dc7ab09a3892de5c0ddeed115c7c9ee5fc008658830b0e6f7f366a72f215192498474ade046fc10e92591dfa133e4beff98758e0e92c05f999504a486e3f71c15b73adf7fe0d311f0d0dc1a8e2aff92572475a92e60f970174c80c4948a0cb86635cb438519b18c260ff9f212925e684ef151cc91fa1c4c8dbbc337c9b46f9a93c88cce49e49a670762186e8858051f93ebff7b0b50caadd9b32f93e396b5a6d1cd7f77b2e50e83fae5f3928433268f6cc28810071ce334025a24c7ebf92e631df0d26e99923018946de4ea5e30e868c33ea3d408d8c58815ecf9892fa3afa5f6fbef9a1dfda2da51b6f01f3dae588bdf17b
sage: Q=0x00b073b35a3bba0954b242961630e8e0af8d3ea8aad80d55669e9e79c388983363d96fe471205743001b4798b81de1be3fb06353fd0c9c4fd43f91f9cb2162b00583cc94ff791a615348d01977ca05f4796d5bff41d94dda1cb3ef2433ff4f6500d15ab290a14e651efa91b7df5d5e50f8356188d9842dcdd67c89eb7d40baa78f045969b804f7a985d35a8a8ac32715cb212a34fb89e85e538275e53663fe675944d2b5dba77b7b7abfacd8fbddee4427db6460b728952af882992750c2542d2122247d7f431ec2fe64c0311b0de5290eb619c6ab476308d59f3efff7255d07ebb51fe64ac5967c0b79bae07d36283cf768c8eee2054349cefe90876da66a52e9e0ff20cc00a40fb0c69524e7a01e5c2f07239831a0fa2f56ad405b3a25537907d990e8f5a3e4f3df99dda6afc30d4565aa5881f64adcc6f6be4077893c7e7710623e6158359c3facfef5036779903065fd0d0a2ed8bf2e53a46758dbff06284670bf10bd2e13cba4323ce50fa43df373bd0940011c424f41c5035a10b8095f420ddb67189f6dad62f1257b0f46e353a90eacc145c7db74998a5d0d8772ba1286eb582504465290171e4db87f2e89564214a2317953a7307b1b4c8dad5920866087d7ae80845478813bdf5f6740d0f3b975a344272804b036e4fef09e91d60d1a730fda2f5c222f7636bbe252f9a6caa86d5e8be7e7db2cf912817a0898a4742d
sage: D=0x3b900240c4a416aeb09b123e02f5457bb31023c9249081c5313edaac741686388492020964ce4471a653c9c63c4dc7b74c0188d0ec50bc3a29797da6c9b3616e83dea45ba5d41e6e4cba7eefca6791f45d3c86a491a899a5c42c7e61930a3681d281f34e4b49707e9bba74f68f7a91274c92904b546b5fe6267c5b8ad8d8bb199a523d7fbb5a40748b56dfccf074f3d664e705153dd903b7cd95bac556be3ab59a165d7cbb765e21a4a1d97e34b412baf1caacb57543d2bd8e5ae5cd2a86dc41e256f3f5c0073052859d1c8a12f4973caa88de2e5eb2eef8d6ebdb66fe154d8e383cba74693753affaf4dbc8e2988c08f947b1f8473a7163775656448572c325250ad2cd75c9f5d42721b91ae8fb427773605afa2f4297daf7ab34f5d71144185f3f5cbc94041081fae19fd5d47fc49421080edc5d658b5f223fb1e9b172f4da1a92263fa922225d4c0231e35d94d276fcb0e0999f5f26068528c83f49b0dd79fb157c49fe8b3d80e7cd2b3da76478ebd2ae2c8164583bee2f96591e2f41ad799ae0e2855a5699996fb2c7efb69f86874bca58628e512d579b247e43ee8db04b424e84e44cf753f86e12dd8d2e0ea0b800e6c313317c14658993b8e04c7fb5de1cba0829123dc518957be2a46f76f8ff305fee17b2310bfdd66a93c8b050451f8ceb26c518b7abb72faf08fca0bf6df8a80de511ffc00dc350cd87e52c9cf5771892300e420929da698b6b973da849c89410f089e9b39de79fc1a01a27d2f879ce5cd80ca0a3899d9f480c68d7a5f8c8b3c74936b19f0c7174987df437658046adaffdaddc3540be7fc06a1d2290f58ad9a2992d32e9ecbadf2eed961b4e68c8a89a5709a334082ad297348c4c31c54a3dadccb93afaa9f1786c4167021d4a16dd78afb41131bdd651357bd44f42cfee5d03fdfae087255d18a6a41c03aa6408b6097d8a6848cbbf05a7f20207d5673ea5e6dc849d5d3009c6e8a6c41285cd64f71b536d8345d61f404079278facde0d6e2264cbe655e877468bd3e76380d91f282f26ae17e41a1bac76c148c4c75dd22656816ab259e2c79920e27e6bcfa83732ac6e0a245dcd9cc82f69e45019cf53ac39bebca3ddea0f55ef97b6d8f7b7eadaae792fb1b55c73cf2ed644e651a22c60ed9c0bb063c50fadda6beccbf6c88c41ef1546bab5f21dbe4fbeeb11f3c5c40fa1cc3df2c11bfe7910d1d36a5ec6e66462c7e216481008931039299d23f2be4d838fb010b661a8541a8b7f7bf7d6c8032895e82133e24c835e5a3491249ec69f28e22cf9b0fab9be9ea17026fbadd470eee4676d7ac79976a1c6807e89b5dabab815ffa076caeaeb53f505a31129dac1e9f0b5d919f17aced63574c8524000022b6bc6cb9c8d6a06e44c72e055a1e2706e736af241ab3084fa56cf942aa139440f74e230be31cd8dd4bf0cbdd657f1
sage: E=0x10001

check:
sage: is_pseudoprime(P)
True
sage: gcd(N,P)
811194519730394220204949383061971492284209477134487451053533919242408334468793875483685418435472924384137737409878754330061341487239404629370463160720071782806016579636145456953095810661706004899017496722730291178259805745059054744795252171022091469940626116746608128441399036310378334222880519662696558703165249434265697658704322903051581598088400258377253583825209022558177374913570364047051007093402547387492492645729748176160840842076964161794363721255756097675823463557162877865622894488049720201680509519072521257128596878592149455958732762099800396648453225220977153025222265023206761554302369499402146842619059859650958489842850140873473393484632985863967898676228674751576699965523367097641503814266418957281198265955430221973482931544501209059788536033857660452959160612655542331433647351037413298986228798018950712662579341162832440884265576141868775326408627532047094505284395403786932363148262901839514736964209136867574532808481484592060405175685831168554790879720280778881035860464184791941816702480873202940903024652495084770128062224279875598826600084633389722629461385386069921483006677287847102371176994910369378323222717613076771700378608286670543729473076010314569999636269167049088093674649352610884381826740603
sage: N%P
0

>> P seems to be a prime, and is indeed a factor of N.

sage: is_pseudoprime(Q)
False
sage: gcd(N,Q)
1
sage: ecm(Q)
Found composite factor of 3 digits: 675
Composite cofactor ... has 1231 digits.
Q has a small factor.  The large cofactor
is way too large to be factorized (today).

>> Q must be wrong.

sage: pow(pow(2,E,N),D,N)
2
sage: pow(pow(3,E,N),D,N)
3
sage: pow(pow(5,E,N),D,N)
5
sage: pow(pow(7,E,N),D,N)
7
sage: pow(pow(11,E,N),D,N)
11
sage: pow(pow(1000,E,N),D,N)
1000

>> x^D mod N is indeed the inverse of x^E mod N
>> D seems to be correct.

>> now compute
sage: Qcorrect = N/P
sage: is_prime(Qcorrect)
False
sage: is_pseudoprime(Qcorrect)
True

>> surprise, this is a sage artefact.
>> is_prime is supposed to tell if Qcorrect
>> is a provable prime, but these numbers are
>> too large for a proof.

sage: help(Qcorrect)
class Rational
...

>> oops, it is of course not a rational number.

sage: Qcorrect = Integer(N/P)
class Integer
...

>> okay now it is an integer.

sage: is_prime(Qcorrect)
>> takes way too long: press CTRL-C
sage: is_pseudoprime(Qcorrect)
True

>> so the correct Q seems to be a prime.

sage: Q-Qcorrect
4468358315186607582623830645994123175323958284313904132666602205502546750542721902065776801908141680869902222733839989940221831332787838985874881107673910358472026239723185949529735314601712865712198736991916521419325287976337589177915143787138292689484229106140251936135768934015263941567159094923493376
sage: hex(Q-Qcorrect)
'1a10400000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000'

>> interesting, now figure out the bits that are flipped in Q:

Q         ...20ddb67189f6dad...
Qcorrect  ...20dd9c6149f6dad...

$ openssl rsa -in rsa8192.pem -outform der -out rsa8192.der
writing RSA key
$ xxd -ps < rsa8192.der > rsa8192.hex
$ sed "s/20ddb67189f6dad/20dd9c6149f6dad/" < rsa8192.hex > rsa8192.out
$ diff rsa8192.hex rsa8192.out
100c100
< 10b8095f420ddb67189f6dad62f1257b0f46e353a90eacc145c7db74998a
---
> 10b8095f420dd9c6149f6dad62f1257b0f46e353a90eacc145c7db74998a

>> et voila

$ xxd -ps -r < rsa8192.out > rsa8192.der
$ openssl rsa -inform der -in rsa8192.der -out rsa8192.pem
writing RSA key
$ openssl rsa -check -noout -in rsa8192.pem
RSA key ok
$ git diff
diff --git a/apps/rsa8192.pem b/apps/rsa8192.pem
index 946a6e5..83d962f 100644
--- a/apps/rsa8192.pem
+++ b/apps/rsa8192.pem
@@ -1,5 +1,4 @@
 -----BEGIN RSA PRIVATE KEY-----
-
 MIISKAIBAAKCBAEAiQ2f1X6Bte1DKD0OoCBKEikzPW+5w3oXk3WwnE97Wxzy6wJZ
 ebbZC3CZKKBnJeBMrysPf+lK+9+fP6Vm8bp1wvbcSIA59BDrX6irFSuM/bdnkbuF
 MFlDjt+uVrxwoyqfPi2IPot1HQg3l5mdyBqcTWvbOnU2L9HZxJfPUCjfzdTMPrMY
@@ -62,7 +61,7 @@ JH1/Qx7C/mTAMRsN5SkOthnGq0djCNWfPv/3JV0H67Uf5krFlnwLebrgfTYoPPdo
 yO7iBUNJzv6Qh22malLp4P8gzACkD7DGlSTnoB5cLwcjmDGg+i9WrUBbOiVTeQfZ
 kOj1o+Tz35ndpq/DDUVlqliB9krcxva+QHeJPH53EGI+YVg1nD+s/vUDZ3mQMGX9
 DQou2L8uU6RnWNv/BihGcL8QvS4Ty6QyPOUPpD3zc70JQAEcQk9BxQNaELgJX0IN
-22cYn22tYvElew9G41OpDqzBRcfbdJmKXQ2HcroShutYJQRGUpAXHk24fy6JVkIU
+2cYUn22tYvElew9G41OpDqzBRcfbdJmKXQ2HcroShutYJQRGUpAXHk24fy6JVkIU
 ojF5U6cwextMja1ZIIZgh9eugIRUeIE7319nQNDzuXWjRCcoBLA25P7wnpHWDRpz
 D9ovXCIvdja74lL5psqobV6L5+fbLPkSgXoImKR0LQKCAgAIC9Jk8kxumCyIVGCP
 PeM5Uby9M3GMuKrfYsn0Y5e97+kSJF1dpojTodBgR2KQar6eVrvXt+8uZCcIjfx8
@@ -98,4 +97,3 @@ TwEgE67iOb2iIoUpon/NyP4LesMzvdpsu2JFlfz13PmmQ34mFI7tWvOb3NA5DP3c
 rMlMLtKfp2w8HlMZpsUlToNCx6CI+tJrohzcs3BAVAbjFAXRKWGijB1rxwyDdHPv
 I+/wJTNaRNPQ1M0SwtEL/zJd21y3KSPn4eL+GP3efhlDSjtlDvZqkdAUsU8=
 -----END RSA PRIVATE KEY-----
-

>> DONE.

Fixes #11776

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/11783)

(cherry picked from commit 7ef43790617cb08b4bb4141df716dfb37385fe5c)

Replace misleading error message when loading PEM

The error message "short header" when the end line
of PEM data cannot be identified is misleading.
Replace it with already existing "bad end line" error.

Fixes #8815

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/11793)

(cherry picked from commit f523ca66612e6712f287aa4b4ed722a5f2d4d960)

Fix PEM certificate loading that sometimes fails

As described in https://github.com/openssl/openssl/issues/9187, the
loading of PEM certificates sometimes fails if a line of base64
content has the length of a multiple of 254.
The problem is in get_header_and_data(). When such a line with a
length of 254 (or a multiple) has been read, the next read will
only read a newline. Due to this get_header_and_data() expects to be
in the header not in the data area. This commit fixes that by checking
if lines have been read completely or only partially. In case of a
previous partial read, a newline will be ignored.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/11741)

(cherry picked from commit 0324ffc5d5d393111288eca2c9d67f2141ed65f5)

Remove AES bitsliced S-box implementation from Boyar and Peralta

[extended tests]

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11626)

Fix use-after-free in BIO_C_SET_SSL callback

Since the BIO_SSL structure was renewed by `ssl_free(b)/ssl_new(b)`,
the `bs` pointer needs to be updated before assigning to `bs->ssl`.

Thanks to @suishixingkong for reporting the issue and providing a fix.

Closes #10539

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11746)

(cherry picked from commit 73d6b4efe6835a6c97ce61df6bf339b0903e5b7a)

Improve SSL_shutdown documentation.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11531)

(cherry picked from commit e307e616f25a6b7b0f343fc1e62a35b2cba888f3)

rand_unix.c: Ensure requests to KERN_ARND don't exceed 256 bytes.

Requests for more than 256 bytes will fail.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11689)

(cherry picked from commit 7421f085005e0d7a1dd2fe61b991ff23cef91c22)

rand_unix.c: Only enable hack for old FreeBSD versions on FreeBSD

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11689)

(cherry picked from commit 0c27ce7322e15a7e0733d48aaf9c8e1d1249541f)

rand_unix.c: Include correct headers for sysctl() on NetBSD

This allows sysctl(KERN_ARND) to be detected properly.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11689)

(cherry picked from commit e2e4b784e65eaafb133a7db3d344446c43112d41)

Update dgst.pod

Removed "Use the B<pkeyutl> command instead for this." from lines 97-98
Fixes #11633

CLA: trivial

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11636)

sslapitest: only compile test when it will be used

The test_ccs_change_cipher() test routine is used only when TLS 1.2
is enabled; to fix the strict-warnings build we should not try to
compile it when TLS 1.2 is disabled, either.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11458)

(cherry picked from commit fb121631e702352d47049ac8e2e709b11c3318e4)

fuzz/asn1.c: Add missing #include

<openssl/dsa.h> gets included via ts.h...  except when 'no-ts' has been
configured.

Fixes #11597

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11640)

(cherry picked from commit 60ebc0ca5a829e2ae939a9ab13658af202b6dfc7)

i2b_PVK_bio: don't set PEM_R_BIO_WRITE_FAILURE in case of success

but in case of an error

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11380)

(cherry picked from commit 80b94a5adb461f94629d36db351a051b0a890856)

Fix type cast in SSL_CTX_set1_groups macro

The macro casts the glist parameter to char*
instead of (int *) like the documentation of the function suggest.

Also the function tls1_set_groups that is called from SSL_CTX_ctrl
takes an int * argument. This looks like a copy&paste error from
SSL_CTX_set1_groups_list function.

CLA: trivial
Signed-off-by: Arne Schwabe <arne@rfc2549.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11614)

(cherry picked from commit 7ffce852372799b6cd856b711db21332f0048314)

Fix typo from #10631

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11595)

(cherry picked from commit 4692e98bdbaebb6f078e89a75c54395839e68b24)

[BN] harden `BN_copy()` against leaks from memory accesses

`BN_copy()` (and indirectly `BN_dup()`) do not propagate the
`BN_FLG_CONSTTIME` flag: the propagation has been turned on and off a
few times in the past years, because in some conditions it has shown
unintended consequences in some code paths.

Without turning the propagation on once more, we can still improve
`BN_copy()` by avoiding to leak `src->top` in case `src` is flagged with
`BN_FLG_CONSTTIME`.
In this case we can instead use `src->dmax` as the number of words
allocated for `dst` and for the `memcpy` operation.

Barring compiler or runtime optimizations, if the caller provides `src`
flagged as const time and preallocated to a public size, no leak should
happen due to the copy operation.

(cherry picked from commit 2d9167ed0b588dacbdd0303fb6041ffe1d8b3a92)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11127)

[EC] harden EC_KEY against leaks from memory accesses

We should never leak the bit length of the secret scalar in the key,
so we always set the `BN_FLG_CONSTTIME` flag on the internal `BIGNUM`
holding the secret scalar.

This is important also because `BN_dup()` (and `BN_copy()`) do not
propagate the `BN_FLG_CONSTTIME` flag from the source `BIGNUM`, and
this brings an extra risk of inadvertently losing the flag, even when
the called specifically set it.

The propagation has been turned on and off a few times in the past
years because in some conditions has shown unintended consequences in
some code paths, so at the moment we can't fix this in the BN layer.

In `EC_KEY_set_private_key()` we can work around the propagation by
manually setting the flag after `BN_dup()` as we know for sure that
inside the EC module the `BN_FLG_CONSTTIME` is always treated
correctly and should not generate unintended consequences.

Setting the `BN_FLG_CONSTTIME` flag alone is never enough, we also have
to preallocate the `BIGNUM` internal buffer to a fixed public size big
enough that operations performed during the processing never trigger
a realloc which would leak the size of the scalar through memory
accesses.

Fixed Length
------------

The order of the large prime subgroup of the curve is our choice for
a fixed public size, as that is generally the upper bound for
generating a private key in EC cryptosystems and should fit all valid
secret scalars.

For preallocating the `BIGNUM` storage we look at the number of "words"
required for the internal representation of the order, and we
preallocate 2 extra "words" in case any of the subsequent processing
might temporarily overflow the order length.

Future work
-----------

A separate commit addresses further hardening of `BN_copy()` (and
indirectly `BN_dup()`).

(cherry picked from commit 0401d766afcd022748763f5614188301c9856c6e)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11127)

[EC] Constify internal EC_KEY pointer usage

A pair of internal functions related to EC_KEY handling could benefit
from declaring `EC_KEY *` variables as `const`, providing clarity for
callers and readers of the code, in addition to enlisting the compiler
in preventing some mistakes.

(cherry picked from commit cd701de96a147260c2290d85af8a0656120a8ff8)

In master `id2_ECParameters` and most of the ASN1 public functions have
been properly constified in their signature.

Unfortunately this has been deemed not doable in a patch release for
1.1.1 as, in subtle ways, this would break API compatibility.
See the discussion at https://github.com/openssl/openssl/pull/9347 for
more details about this.

This constification commit should still be portable w.r.t. our criteria,
as the constification happens only on internal functions.

The fix here is to explicitly discard the const qualifier before the
call to `i2d_ECParameters`, which should be safe anyway because we can
expect `i2d_ECParameters()` to treat the first argument as if it was
const.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11127)

Add setter equivalents to X509_REQ_get0_signature

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10563)

(cherry picked from commit c72e59349f50ee00a1bf8605ada17dfccb8b3b1a)

Prepare for 1.1.1h-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.1g release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update copyright year

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix NULL dereference in SSL_check_chain() for TLS 1.3

In the tls1_check_sig_alg() helper function, we loop through the list of
"signature_algorithms_cert" values received from the client and attempt
to look up each one in turn in our internal table that maps wire
codepoint to string-form name, digest and/or signature NID, etc., in
order to compare the signature scheme from the peer's list against what
is used to sign the certificates in the certificate chain we're
checking.  Unfortunately, when the peer sends a value that we don't
support, the lookup returns NULL, but we unconditionally dereference the
lookup result for the comparison, leading to an application crash
triggerable by an unauthenticated client.

Since we will not be able to say anything about algorithms we don't
recognize, treat NULL return from lookup as "does not match".

We currently only apply the "signature_algorithm_cert" checks on TLS 1.3
connections, so previous TLS versions are unaffected.  SSL_check_chain()
is not called directly from libssl, but may be used by the application
inside a callback (e.g., client_hello or cert callback) to verify that a
candidate certificate chain will be acceptable to the client.

CVE-2020-1967

Reviewed-by: Matt Caswell <matt@openssl.org>

Add test for CVE-2020-1967

Add to test_sslsigalgs a TLSProxy test that injects a
"signature_algorithms_cert" extension that contains an unallocated
codepoint.

The test currently fails, since s_server segfaults instead of
ignoring the unrecognized value.

Since "signature_algorithms" and "signature_algorithms_cert" are very
similar, also add the analogous test for "signature_algorithms".

Reviewed-by: Matt Caswell <matt@openssl.org>

Update CHANGES and NEWS for release of 1.1.1g

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add a CHANGES entry for AES constant time

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11411)

Add AES consttime code for no-asm configurations

This adds optional constant time support for AES
when building openssl for no-asm.

Enable with: ./config no-asm -DOPENSSL_AES_CONST_TIME

[extended tests]

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11411)

s_time: Allow using -CAfile option as in other commands

The s_time command in difference from all the other similar
commands supported -cafile option instead of -CAfile.
Add the -CAfile option and keep -cafile only for backwards
compatibility.

Fixes #11552

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11555)

INSTALL: document 'no-ui-console' rather than 'no-ui'

The UI interface itself is never disabled, but the console backend may
be. 'no-ui' is a deprecated backward compatibility alias for
'no-ui-console'.

Fixes #11551

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11554)

Windows: Add type casting in CRYPTO_atomic_add to remove warning

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11337)

(cherry picked from commit 7da7b27eec58d1efc7012f002c45ddbdd61a5e79)

BIO_do_accept: correct error return value

`BIO_do_accept` was returning incorrect values when unable to bind to a port.

Fixes #7717
CLA: trivial

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11505)

(cherry picked from commit 0437435a960123be1ced766d18d715f939698345)

Don't compile commands if disabled

Rather than wrapping whole files in "ifndef OPENSSL_NO_xxx" we handle
the changes in build.info

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/11518)

Add manpage entry for X509_check_purpose()

Fixes #10263

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10301)

(cherry picked from commit eacd30a703b52f5b3e8e70b58f99bd1128458c7c)

Update RAND_METHOD definition in man page

The `add` and `seed` callbacks were changed to return `int` instead of
`void` in b6dcdbfc94c482f6c15ba725754fc9e827e41851 (first included in
tag OpenSSL_1_1_0-pre1).

The `add` callback was changed to take a `double` instead of an `int`
in 853f757ecea74a271a7c5cdee3f3b5fe0d3ae863.

CLA: trivial
Fixes: #10199

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11486)

(cherry picked from commit ae3254287ff87e484c7fd8f757cad1440ee8f5ff)

Fix AES-CTR_DRBG on 1.1.1.

The backport of the timing information leak fix uses u32 which is defined
in crypto/modes/modes_local.h in 1.1.1 and include/crypto/modes.h for 3.0.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11489)

AES CTR-DRGB: do not leak timing information

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11147)

(cherry picked from commit 069165d10646a22000c596095cc04d43bbf1f807)

Integer overflow in ASN1_STRING_set.

Addressing a potential integer overflow condition.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11473)

(cherry picked from commit 96218269f4c2da82f143727fb7697d572c190bc5)

[crypto/ec] blind coordinates in ec_wNAF_mul for robustness

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Nicola Tuveri <nicola.tuveri@ibm.com>
(Merged from https://github.com/openssl/openssl/pull/11439)

(cherry picked from commit c61ced5ec50fc68707c7cea79f7df1d170f03f13)

Fix the error handling in EC_POINTs_mul

This was pointed out by a false-positive
-fsanitizer warning ;-)

However from the cryptographical POV the
code is wrong:
A point R^0 on the wrong curve
is infinity on the wrong curve.

[extended tests]

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11475)

(cherry picked from commit 1eb9b54af7e00fa12196411964ce742ea8677766)

Ensure ECDSA_size always returns >= 0

Fixes #10484

Signed-off-by: William Brawner <me@wbrawner.com>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11472)

Add test cases for the non CA certificate with pathlen:0

Accept verification without -x509_strict and reject it with it.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11463)

(cherry picked from commit 3cb55fe47c3398b81956e4fe20c4004524d47519)

Set X509_V_ERR_INVALID_EXTENSION error for invalid basic constraints

If we encounter certificate with basic constraints CA:false,
pathlen present and X509_V_FLAG_X509_STRICT is set we set
X509_V_ERR_INVALID_EXTENSION error.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11463)

(cherry picked from commit fa86e2ee3533bb7fa9f3c62c38920cf960e9fec0)

Allow certificates with Basic Constraints CA:false, pathlen:0

Do not mark such certificates with EXFLAG_INVALID although they
violate the RFC 5280, they are syntactically correct and
openssl itself can produce such certificates without any errors
with command such as:

openssl x509 -req -signkey private.pem -in csr.pem -out cert.pem \
  -extfile <(echo "basicConstraints=CA:FALSE,pathlen:0")

With the commit ba4356ae4002a04e28642da60c551877eea804f7 the
EXFLAG_INVALID causes openssl to not consider such certificate
even as leaf self-signed certificate which is breaking existing
installations.

Fixes: #11456

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11463)

(cherry picked from commit 428cf5ff83a48d0b51c97476586b2cbd053b6302)

Expand the XTS documentation

Explain that XTS does not support streaming, and that the IV value is the
tweak.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11461)

(cherry picked from commit 70d80ef9898ddbe03841efd2df2f526b71829d7f)

[crypto/ec] Ladder tweaks

- Convert to affine coords on ladder entry. This lets us use more efficient
  ladder step formulae.

- Convert to affine coords on ladder exit. This prevents the current code
  awkwardness where conversion happens twice during serialization: first to
  fetch the buffer size, then again to fetch the coords.

- Instead of projectively blinding the input point, blind both accumulators
  independently.

(cherry picked from commit a4a93bbfb0e679eaa249f77c7c4e7e823ca870ef)

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/11435)

[test] Make sm2_internal_test less fragile to changes in the ec module

Since these are KATs, the trailing randomness consumed by the ec module
does not really matter. So make the fake random buffer circular.

(cherry picked from commit 09736245b174a37abb87fb7ceb55462d940ff2bb)

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/11435)

Prepare for 1.1.1g-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.1f release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update copyright year

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11445)