[crypto/ec] for ECC parameters with NULL or zero cofactor, compute it

The cofactor argument to EC_GROUP_set_generator is optional, and SCA
mitigations for ECC currently use it. So the library currently falls
back to very old SCA-vulnerable code if the cofactor is not present.

This PR allows EC_GROUP_set_generator to compute the cofactor for all
curves of cryptographic interest. Steering scalar multiplication to more
SCA-robust code.

This issue affects persisted private keys in explicit parameter form,
where the (optional) cofactor field is zero or absent.

It also affects curves not built-in to the library, but constructed
programatically with explicit parameters, then calling
EC_GROUP_set_generator with a nonsensical value (NULL, zero).

The very old scalar multiplication code is known to be vulnerable to
local uarch attacks, outside of the OpenSSL threat model. New results
suggest the code path is also vulnerable to traditional wall clock
timing attacks.

CVE-2019-1547

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/9781)

[ec/ecp_nistp*.c] restyle: use {} around `else` too

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9511)

(cherry picked from commit 4fe2ee3a449a8ca2886584e221f34ff0ef5de119)

[ec/ecp_nistp*.c] remove flip_endian()

Replace flip_endian() by using the little endian specific
BN_bn2lebinpad() and BN_lebin2bn().

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9511)

(cherry picked from commit e0b660c27d8d97b4ad9e2098cc957de26872c0ef)

Uniform BN_bn2binpad() and BN_bn2lebinpad() implementations

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9511)

(cherry picked from commit 1b338abe3abb8c73f004c34d4b8a9272b89dfd5d)

Make BN_num_bits() consttime upon BN_FLG_CONSTTIME

This issue was partially addressed by commit
972c87dfc7e765bd28a4964519c362f0d3a58ca4, which hardened its callee
BN_num_bits_word() to avoid leaking the most-significant word of its
argument via branching and memory access pattern.
The commit message also reported:
> There are a few places where BN_num_bits is called on an input where
> the bit length is also secret. This does *not* fully resolve those
> cases as we still only look at the top word.

BN_num_bits() is called directly or indirectly (e.g., through
BN_num_bytes() or BN_bn2binpad() ) in various parts of the `crypto/ec`
code, notably in all the currently supported implementations of scalar
multiplication (in the generic path through ec_scalar_mul_ladder() as
well as in dedicated methods like ecp_nistp{224,256,521}.c and
ecp_nistz256.c).

Under the right conditions, a motivated SCA attacker could retrieve the
secret bitlength of a secret nonce through this vulnerability,
potentially leading, ultimately, to recover a long-term secret key.

With this commit, exclusively for BIGNUMs that are flagged with
BN_FLG_CONSTTIME, instead of accessing only bn->top, all the limbs of
the BIGNUM are accessed up to bn->dmax and bitwise masking is used to
avoid branching.

Memory access pattern still leaks bn->dmax, the size of the lazily
allocated buffer for representing the BIGNUM, which is inevitable with
the current BIGNUM architecture: reading past bn->dmax would be an
out-of-bound read.
As such, it's the caller responsibility to ensure that bn->dmax does not
leak secret information, by explicitly expanding the internal BIGNUM
buffer to a public value sufficient to avoid any lazy reallocation
while manipulating it: this should be already done at the top level
alongside setting the BN_FLG_CONSTTIME.

Thanks to David Schrammel and Samuel Weiser for reporting this issue
through responsible disclosure.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9511)

(cherry picked from commit 8b44198b916015f77bef1befa26edb48ad8a0238)

Fix a SCA leak using BN_bn2bin()

BN_bn2bin() is not constant-time and leaks the number of bits in the
processed BIGNUM.

The specialized methods in ecp_nistp224.c, ecp_nistp256.c and
ecp_nistp521.c internally used BN_bn2bin() to convert scalars into the
internal fixed length representation.

This can leak during ECDSA/ECDH key generation or handling the nonce
while generating an ECDSA signature, when using these implementations.
The amount and risk of leaked information useful for a SCA attack
varies for each of the three curves, as it depends mainly on the
ratio between the bitlength of the curve subgroup order (governing the
size of the secret nonce/key) and the limb size for the internal BIGNUM
representation (which depends on the compilation target architecture).

To fix this, we replace BN_bn2bin() with BN_bn2binpad(), bounding the
output length to the width of the internal representation buffer: this
length is public.

Internally the final implementation of both BN_bn2binpad() and
BN_bn2bin() already has masking in place to avoid leaking bn->top
through memory access patterns.
Memory access pattern still leaks bn->dmax, the size of the lazily
allocated buffer for representing the BIGNUM, which is inevitable with
the current BIGNUM architecture: reading past bn->dmax would be an
out-of-bound read.
As such, it's the caller responsibility to ensure that bn->dmax does not
leak secret information, by explicitly expanding the internal BIGNUM
buffer to a public value sufficient to avoid any lazy reallocation
while manipulating it: this is already done at the top level alongside
setting the BN_FLG_CONSTTIME.

Finally, the internal implementation of BN_bn2binpad() indirectly calls
BN_num_bits() via BN_num_bytes(): the current implementation of
BN_num_bits() can leak information to a SCA attacker, and is addressed
in the next commit.

Thanks to David Schrammel and Samuel Weiser for reporting this issue
through responsible disclosure.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9511)

(cherry picked from commit 805315d3a20f7274195eed75b06c391dacf3b197)

Fix a SCA leak in BN_generate_dsa_nonce

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/9782)

(cherry picked from commit 31ca19403d56ad71d823cf62990518dfc6905bb4)

[crypto/rsa] Set the constant-time flag in multi-prime RSA too

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9779)

[crypto/asn1] Fix multiple SCA vulnerabilities during RSA key validation.

This commit addresses multiple side-channel vulnerabilities present
during RSA key validation.
Private key parameters are re-computed using variable-time functions.

This issue was discovered and reported by the NISEC group at TAU Finland.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9779)

Teach TLSProxy how to parse CertificateRequest messages

We also use this in test_tls13messages to check that the extensions we
expect to see in a CertificateRequest are there.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9780)

(cherry picked from commit dc5bcb88d819de55eb37460c122e02fec91c6d86)

Don't send a status_request extension in a CertificateRequest message

If a TLSv1.3 server configured to respond to the status_request extension
also attempted to send a CertificateRequest then it was incorrectly
inserting a non zero length status_request extension into that message.

The TLSv1.3 RFC does allow that extension in that message but it must
always be zero length.

In fact we should not be sending the extension at all in that message
because we don't support it.

Fixes #9767

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9780)

(cherry picked from commit debb64a0ca43969eb3f043aa8895a4faa7f12b6e)

Fix error handling in x509_lu.c

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9639)

(cherry picked from commit c70e2ec33943d3bd46d3d9950f774307feda832b)

Cleanup includes in rand_unix.c

Fixes #9757

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9761)

(cherry picked from commit 41ffd2ab09d24692c71850ccd7d5ff154196fe01)

Remove ifndef FIPS_MODE from rand_unix.c

This will never be the case for 1.1.1 so removed.

Fixes: comment 1 of #9757

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9762)

Suppress 'No server certificate CA names sent' message

Fixes #9080

Signed-off-by: Billy Brawner <billy@wbrawner.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9710)

(cherry picked from commit 1e8e75d18be8856e753a57771754b9926c3f4264)

Test SSL_set_ciphersuites

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9621)

Fix SSL_set_ciphersuites to set even if no call to SSL_set_cipher_list

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9621)

Fix bogus check for EVP_PKEY mandatory digest in check_cert_usable()

In commit 6aca8d1a5 ("Honour mandatory digest on private key in
has_usable_cert()") I added two checks for the capabilities of the
EVP_PKEY being used. One of them was wrong, as it should only be
checking the signature of the X.509 cert (by its issuer) against the
sigalgs given in a TLS v1.3 signature_algorithms_cert extension.

Remove it.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9705)

Remove duplicate CHANGES text.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9738)

Don't include the DEVRANDOM being seeded logic on Android.

It lacks exposure of the `shm*` functions and should prefer the GETRANDOM
source.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9735)

(cherry picked from commit 280cc0180862ae6664b88d5ea12cb5f599000d36)

Fix NITs in comments and CHANGES for DEVRANDOM seeded check.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9734)

(cherry picked from commit 46a9cc9451213039fd53f62733b2ccd04e853bb2)

Fix SCA vulnerability when using PVK and MSBLOB key formats

This commit addresses a side-channel vulnerability present when
PVK and MSBLOB key formats are loaded into OpenSSL.
The public key was not computed using a constant-time exponentiation
function.

This issue was discovered and reported by the NISEC group at TAU Finland.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9587)

(cherry picked from commit 724339ff44235149c4e8ddae614e1dda6863e23e)

Avoid overflowing FDSET when using select(2).

There is a problem in the rand_unix.c code when the random seed fd is greater
than or equal to FD_SETSIZE and the FDSET overruns its limit and walks the
stack.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9686)

(cherry picked from commit e1f8584d47a499301fba781086af6885fcf21fec)

test/evp_test.c: distinguish parsing errors from processing errors

Parsing functions are at liberty to return:

1:  when parsing on processing of the parsed value succeeded
0:  when the parsed keyword is unknown
-1: when the parsed value processing failed

Some parsing functions didn't do this quite right, they returned 0
when they should have returned -1, causing a message like this:

    Line 123: unknown keyword PeerKey

When this message (which is displayed when the parsing function
returns -1) would have been more appropriate:

    Line 123: error processing keyword PeerKey = ffdhe2048-2-pub

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9682)

(cherry picked from commit f42c225d7f9a0bce0bf46103343402d3f0ad742f)

openssl dgst, openssl enc: check for end of input

The input reading loop in 'openssl dgst' and 'openssl enc' doesn't
check for end of input, and because of the way BIO works, it thereby
won't detect that the end is reached before the read is an error.
With the FILE BIO, an error occurs when trying to read past EOF, which
is fairly much ok, except when the command is used interactively, at
least on Unix.  The result in that case is that the user has to press
Ctrl-D twice for the command to terminate.

The issue is further complicated because both these commands use
filter BIOs on top of the FILE BIO, so a naïve attempt to check
BIO_eof() doesn't quite solve it, since that only checks the state of
the source/sink BIO, and the filter BIO may have some buffered data
that still needs to be read.  Fortunately, there's BIO_pending() that
checks exactly that, if any filter BIO has pending data that needs to
be processed.

We end up having to check both BIO_pending() and BIO_eof().

Thanks to Zsigmond Lőrinczy for the initial effort and inspiration.

Fixes #9355

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9668)

(cherry picked from commit 8ed7bbb411d2a9e0edef928958ad955e0be3d6dd)

Correct documented return value for BIO_get_mem_data()

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9643)

(cherry picked from commit 797a5b7af9d9bbfbcbff4607c10ad5c5595ac785)

Start up DEVRANDOM entropy improvement for older Linux devices.

Improve handling of low entropy at start up from /dev/urandom by waiting for
a read(2) call on /dev/random to succeed.  Once one such call has succeeded,
a shared memory segment is created and persisted as an indicator to other
processes that /dev/urandom is properly seeded.

This does not fully prevent against attacks weakening the entropy source.
An attacker who has control of the machine early in its boot sequence
could create the shared memory segment preventing detection of low entropy
conditions.  However, this is no worse than the current situation.

An attacker would also be capable of removing the shared memory segment
and causing seeding to reoccur resulting in a denial of service attack.
This is partially mitigated by keeping the shared memory alive for the
duration of the process's existence.  Thus, an attacker would not only need
to have called call shmctl(2) with the IPC_RMID command but the system
must subsequently enter a state where no instances of libcrypto exist in
any process.  Even one long running process will prevent this attack.

The System V shared memory calls used here go back at least as far as
Linux kernel 2.0.  Linux kernels 4.8 and later, don't have a reliable way
to detect that /dev/urandom has been properly seeded, so a failure is raised
for this case (i.e. the getentropy(2) call has already failed).

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9595)

[manual merge]

Add a fallback definition for __NR_getrandom for x86 linux

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9639)

(cherry picked from commit 038b381ecf2a988eee4c7bb21074ed0603303bd1)

Fix some pod-page ordering nits

Backport of https://github.com/openssl/openssl/pull/9602

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9632)

doc: fix link in BN_new.pod

Fixes #9622

CLA: trivial

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9627)

(cherry picked from commit faee6b21048623a422d537cdbad24f50c5c21937)

Add a fallback definition for __NR_getrandom for ARM linux

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9619)

(cherry picked from commit 24d932ec842bc5fdbd5e9b519cecf15a56bc74e6)

Test for out-of-bounds write when requesting zero bytes from shake

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9433)

(cherry picked from commit 3ce46435e6ebed69bec0fa3454cc195ced426d42)

Directly return from final sha3/keccak_final if no bytes are requested

Requesting zero bytes from shake previously led to out-of-bounds write
on some platforms.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9433)

(cherry picked from commit a890ef833d114da3430c2f2efd95e01714704d34)

.travis.yml: Use travis_terminate on failure

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9620)

(cherry picked from commit 5be78a88aa922a6c43a83a18dbe252c6a358b8e9)

Fix error handling in X509_chain_up_ref

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9614)

(cherry picked from commit cae665dfa6ccec743a7f39cf80676d7d2d787e56)

INSTALL: clarify documentation of the --api=x.y.z deprecation option

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9599)

(cherry picked from commit bf9d6bb83d009923ceb65753c6dd9fa880e8ba92)

Fix syntax error for the armv4 assembler

The error was from the alignment syntax of the code.
More details:
https://stackoverflow.com/questions/57316823/arm-assembly-syntax-in-vst-vld-commands?noredirect=1#comment101133590_57316823

CLA: trivial

Fixes: #9518

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9518)

(cherry picked from commit 2a17758940657cc3a97b032104a92f0aa304f863)

crypto/engine/eng_openssl.c: define TEST_ENG_OPENSSL_RC4_P_INIT conditionally

When OpenSSL is configured with 'no-stdio', TEST_ENG_OPENSSL_RC4_P_INIT
shouldn't be defined, as that test uses stdio.

Fixes #9597

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9598)

(cherry picked from commit 9f643f54236d6cf0d0d24327acd3b858883f0686)

Extend tests of SSL_check_chain()

Actually supply a chain and then test:
1) A successful check of both the ee and chain certs
2) A failure to check the ee cert
3) A failure to check a chain cert

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9443)

Fix SSL_check_chain()

The function SSL_check_chain() can be used by applications to check that
a cert and chain is compatible with the negotiated parameters. This could
be useful (for example) from the certificate callback. Unfortunately this
function was applying TLSv1.2 sig algs rules and did not work correctly if
TLSv1.3 was negotiated.

We refactor tls_choose_sigalg to split it up and create a new function
find_sig_alg which can (optionally) take a certificate and key as
parameters and find an appropriate sig alg if one exists. If the cert and
key are not supplied then we try to find a cert and key from the ones we
have available that matches the shared sig algs.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9443)

Add missing EBCDIC strings

Fix a few places where calling ossl_isdigit does the wrong thing on
EBCDIC based systems.
Replaced with ascii_isdigit.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9556)

(cherry picked from commit 48102247ff513d4c57b40b19c1d432f37b9e4b02)

Add description in X509_STORE manipulation

Add memory management description in X509_STORE_add_cert, otherwise
users will not be aware that they are leaking memory...

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9484)

(cherry picked from commit 86333b6e0c5c488130ab237e95b8520891b81bf6)

BIO_lookup_ex: Do not retry on EAI_MEMORY

We should not retry on EAI_MEMORY as that error is most probably
fatal and not depending on AI_ADDRCONFIG hint.

Also report the error from the first call if the second call fails
as that one would be most probably the more interesting one.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9535)

(cherry picked from commit 91cb81d40a8102c3d8667629661be8d6937db82b)

BIO_lookup_ex: Always retry the lookup on failure with AI_NUMERICHOST set

Do not try to discern the error return value on
getaddrinfo() failure but when retrying set the AI_NUMERICHOST
to avoid DNS lookups.

Fixes: #9053

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9535)

(cherry picked from commit 7f616a00e9232392927099dca1eca70d0d058665)

Configurations/unit-Makefile.tmpl: Don't clean away dotted files

A local 'make clean' did some sweeping removals of files execpt for
the .git directory.  This is a little too sweeping, as other dotted
files might be cleaned away if they happen to match the pattern that's
searched for.

An example is a symlink .dir-locals.el that would keep disappearing if
you build in the source tree and do a make clean...

So we change this to leave all dotted files alone.  Our builds do not
produce such files anyway, so this is a harmless (or rather, less
harmful) change.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9573)

(cherry picked from commit 853094dbe15a49b334f3488fc99a557abf021c09)

engines/build.info: if the padlock engine is disabled, don't build it!

Fixes #9244

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9572)

Remove some duplicate words from the documentation

Fixup INSTALL and a couple man pages to get rid of "the the" and "in the
in the".

CLA: trivial

Reviewed-by: Paul Yang <kaishen.yy@antfin.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9563)

(cherry picked from commit 3c74e77bd89e3d5400ab4d640149e27863756579)

Add TLS tests for RSA-PSS Restricted certificates

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9553)

(cherry picked from commit 20946b94658416d2fed0b9d9c7adfbe4b7d70515)

Add Restricted PSS certificate and key

Create a PSS certificate with parameter restrictions

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9553)

(cherry picked from commit 39d9ea5e502114a204750f641ca76ff5b4912401)

Ensure RSA PSS correctly returns the right default digest

A default digest of SHA256 was being returned for RSA PSS even if the
PSS parameters indicated a different digest must be used. We change this
so that the correct default digest is returned and additionally mark this
as mandatory for PSS.

This bug had an impact on sig alg selection in libssl. Due to this issue
an incorrect sig alg might be selected in the event that a server is
configured with an RSA-PSS cert with parameter restrictions.

Fixes #9545

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9553)

(cherry picked from commit 9bcc9f973b2a216461dd6f140e47ef647eb733b4)

mention what happens if OPENSSL_NO_RC2 is defined

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9415)

(cherry picked from commit 8c47e55ee69500e31e80458682c6e022294cd0be)

make ecp_nistz256_point_add_vis3() local

fixes #8936

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9132)

(cherry picked from commit 861335001b99cfd665030c50ad37a663536a5c0f)

Fix reversed meaning of error codes

The meaning of the X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY and X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT error codes were still reversed in the X509_STORE_CTX_get_error function documentation.
This used to be the problem also in the verify application documentation, but was fixed on 2010-02-23 in 7d3d178.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9529)

(cherry picked from commit 6d5aa88d74b67b1bc108e121dea687a4ca67e329)

Clarify the INSTALL instructions

Ensure users understand that they need to have appropriate permissions
to write to the install location.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9268)

(cherry picked from commit 7c03bb9fff02b7f08d4654f51f8667584a92cf72)

Change EVP_CIPHER_CTX_iv_length() to return current ivlen for some modes

Note a flag needed to be added since some ssl tests fail if they output any error
(even if the error is ignored). Only ciphers that handle the GET_IV_LEN control set this flag.

Fixes #8330

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9499)

Fix ECDSA_SIG docs

They incorrectly said that i2d_ECDSA_SIG returns 0 on error. In fact it
returns a negative value on error.

We fix this by moving the i2d_ECDSA_SIG/d2i_ECDSA_SIG docs onto the same
page as all the other d2i/i2d docs.

Fixes #9517

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/9533)

(cherry picked from commit bbda79976b5c5095c5e6557311c86c623ba335f1)

Correct the Extended Master Secret string for EBCDIC

The macro TLS_MD_MASTER_SECRET_CONST is supposed to hold the ascii string
"extended master secret". On EBCDIC machines it actually contained the
value "extecded master secret"

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9430)

(cherry picked from commit c1a3f16f735057b45df1803d58f40e4e17b233e5)

Fix SSL_MODE_RELEASE_BUFFERS functionality

At some point in the past do_ssl3_write() used to return the number of
bytes written, or a value <= 0 on error. It now just returns a success/
error code and writes the number of bytes written to |tmpwrit|.

The SSL_MODE_RELEASE_BUFFERS code was still looking at the return code
for the number of bytes written rather than |tmpwrit|. This has the effect
that the buffers are not released when they are supposed to be.

Fixes #9490

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9505)

(cherry picked from commit 8bbf63e48f27c5edaa03e6d87d969c9b6a207f3c)

Add missing accessors for X509 AuthorityKeyIdentifier

Complements commit b383aa208146, which added X509_get0_authority_key_id().

 const ASN1_OCTET_STRING *X509_get0_authority_key_id(X509 *x);
 const GENERAL_NAMES *X509_get0_authority_issuer(X509 *x);      [NEW]
 const ASN1_INTEGER *X509_get0_authority_serial(X509 *x);       [NEW]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9494)

Use allow_early_data_cb from SSL instead of SSL_CTX

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9471)

(cherry picked from commit 59b2cb2638dda3e07385ad36a41f0e141b36987b)

Fix Typos

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9295)

Fix warning C4164 in MSVC.

Fix: crypto\whrlpool\wp_block.c(90) : warning C4164: '_rotl64' : intrinsic function not declared.
Fixes #9487

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9488)

(cherry picked from commit 0c789f59f117ccbb30ffc621216ba776117c7c61)

make RSA and DSA operations throw MISSING_PRIVATE_KEY if needed, adapt ECDSA

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/9466)

(cherry picked from commit 7408f6759f1b0100438ca236ea8f549454aaf2d5)

Use OPENSSL_strlcpy instead of strncpy in e_afalg.c

This avoids a spurious gcc warning:
./config enable-asan --strict-warnings
=>
In function 'afalg_create_sk',
    inlined from 'afalg_cipher_init' at engines/e_afalg.c:545:11:
engines/e_afalg.c:376:5: error: '__builtin_strncpy' output may be
    truncated copying 63 bytes from a string of length 63 [-Werror=stringop-truncation]
  376 |     strncpy((char *) sa.salg_name, ciphername, ALG_MAX_SALG_NAME);
      |     ^~~~~~~

[extended tests]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9478)

(cherry picked from commit 62cc845fc955c8d4de7b703f57bfd8e5854f00f4)

Add weak platform independent PRNG to test framework.

Implement the GNU C library's random(3) pseudorandom number generator.
The algorithm is described: https://www.mscs.dal.ca/~selinger/random/

The rationale is to make the tests repeatable across differing platforms with
different underlying implementations of the random(3) library call.

More specifically: when executing tests with random ordering.

[extended tests]

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9463)

(cherry picked from commit e9a5932d04f6b7dd25b39a8ff9dc162d64a78c22)

Don't generate an unnecessary Diffie-Hellman key in TLS 1.3 clients.

tls_parse_stoc_key_share was generating a new EVP_PKEY public/private
keypair and then overrides it with the server public key, so the
generation was a waste anyway. Instead, it should create a
parameters-only EVP_PKEY.

(This is a consequence of OpenSSL using the same type for empty key,
empty key with key type, empty key with key type + parameters, public
key, and private key. As a result, it's easy to mistakenly mix such
things up, as happened here.)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/9445)

(cherry picked from commit 166c0b98fd6e8b1bb341397642527a9396468f6c)

Fix default installation paths on mingw

Mingw config targets assumed that resulting programs and libraries are
installed in a Unix-like environment and the default installation
prefix was therefore set to '/usr/local'.

However, mingw programs are installed in a Windows environment, and
the installation directories should therefore have Windows defaults,
i.e. the same kind of defaults as the VC config targets.

A difficulty is, however, that a "cross compiled" build can't figure
out the system defaults from environment the same way it's done when
building "natively", so we have to fall back to hard coded defaults in
that case.

Tests can still be performed when cross compiled on a non-Windows
platform, since all tests only depend on the source and build
directory, and otherwise relies on normal local paths.

CVE-2019-1552

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9400)

Make rand_pool buffers more dynamic in their sizing.

The rand pool support allocates maximal sized buffers -- this is typically
12288 bytes in size.  These pools are allocated in secure memory which is a
scarse resource.  They are also allocated per DRBG of which there are up to two
per thread.

This change allocates 64 byte pools and grows them dynamically if required.
64 is chosen to be sufficiently large so that pools do not normally need to
grow.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9428)

(cherry picked from commit a6a66e4511eec0f4ecc2943117a42b3723eb2222)

Allocate DRBG additional data pool from non-secure memory

The additional data allocates 12K per DRBG instance in the
secure memory, which is not necessary. Also nonces are not
considered secret.

[extended tests]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9424)

Remove HEADER_X509_H include detector from apps

The HEADER_X509_H check is redundant, because <openssl/x509.h>
is already included.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9365)

Remove OPENSSL_X509V3_H include detector from openssl/cms.h

The check is redundant, because <openssl/x509v3.h> is included.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9365)

Remove HEADER_BSS_FILE_C module include guard

This include guard inside an object file comes as a surprise and
serves no purpose anymore. It seems like this object file was
included by crypto/threads/mttest.c at some time, but the include
directive was removed in commit bb8abd6.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9365)

Remove external HEADER_SYMHACKS_H include guard

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9365)

Change DH parameters to generate the order q subgroup instead of 2q

This avoids leaking bit 0 of the private key.

Backport-of: #9363

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/9435)

Avoid double clearing some BIGNUMs

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9438)

(cherry picked from commit 82925f9dd0492f2e5f1d80ff46fd59f0704c8fe6)

Cygwin: enable the use of Dl_info and dladdr()

These weren't available in Cygwin at the time our DSO code was
written, but things have changed since.

Fixes #9385

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9402)

(cherry picked from commit 38f6f99cdf0a87345d646d30a764c089c38627ad)

test/enginetest.c: Make sure no config file is loaded

If a config file gets loaded, the tests get disturbed.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9393)

(cherry picked from commit 5800ba761052894145abe7a74a1159df007b6875)

Add value_barriers in constant time select functions

The barriers prevent the compiler from narrowing down the
possible value range of the mask and ~mask in the select
statements, which avoids the recognition of the select
and turning it into a conditional load or branch.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/9217)

(cherry picked from commit 04edd688b3727835f9b2c7cca7e4c963bf3ed2ba)

Cleanup use of X509 STORE locks

Cosmetic changes to use the X509_STORE_lock/unlock functions.
Renamed some ctx variables to store.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9366)

(cherry picked from commit 7a9abccde7b7a5e36efe42d89246f6cfd4d59f44)

s390x assembly pack: fix restoring of SIGILL action

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9381)

(cherry picked from commit c17d60ea293746d7cd06a910ced446edbb6c1eba)

Fix a C++ comment in the refcount.h

Although in a false-conditional code section gcc-4.8.4 flagged this with
a C90 warning :-(

include/internal/refcount.h:108:7: error: C++ style comments are not allowed in ISO C90 [-Werror]
       // under Windows CE we still have old-style Interlocked* functions

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9388)

Fix the return value for SSL_get0_chain_certs()

This function was always returning 0. It should return 1 on success.

Fixes #9374

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/9395)

(cherry picked from commit 7bc82358ae930cfbd353602bc1fd25bfad107350)

Fix SSL_CTX_set_session_id_context() docs

Also, use define rather than sizeof

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9377)

(cherry picked from commit fe9edc9d39c96c965efc4fde12ddf7fa8a852025)

Actually silently ignore GET / OCSP requests

Reviewed-by: Matt Caswell <matt@openssl.org>

Remove DRBG from SSL structure.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9390)

(cherry picked from commit 3d9b33b5e48d82d098a1f8c37dbf616a0d84621c)

Fix wrong lock claimed in x509 dir lookup.

x509 store's objects cache can get corrupted when using dir lookup
method in multithreaded application. Claim x509 store's lock when
accessing objects cache.

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9326)

(cherry picked from commit a161738a708b5e284a4714edc0c976606ea7cb26)

issue-9316: Update return documentation for RAND_set_rand_engine

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9351)

(cherry picked from commit 5fe6e2311df9bbbe347cdc7b3c22ce06e20a0ef9)

Avoid NULL pointer dereference. Fixes #9043.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9059)

(cherry picked from commit 9fd6f7d1cd2a3c8e2bc69dcb8bde8406eb6c2623)

iOS build: Replace %20 with space in config script

CLA: trivial

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9297)

(cherry picked from commit 53fd220c8fc953b603dd13257d6b2e2d1e7eb864)

Fix build error for aarch64 big endian.

Modified rev to rev64, because rev only takes integer registers.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90827
Otherwise, the following error will occur.

Error: operand 1 must be an integer register -- `rev v31.16b,v31.16b'

CLA: trivial

Signed-off-by: Lei Maohui <leimaohui@cn.fujitsu.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9151)

(cherry picked from commit 7b0fceed21c8929e0c6694f57018aa1dbba03e15)

man: fix typo in OPENSSL_fork_prepare.pod

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/9318)

(cherry picked from commit 933a73b9144397a5690a75c69694123a00d3590d)

Fix an endless loop in BN_generate_prime_ex

Happens when trying to generate 4 or 5 bit safe primes.

[extended tests]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9311)

(cherry picked from commit 291f616ced45c924d639d97fc9ca2cbeaad096cf)

Clarify documentation of SSL_CTX_set_verify client side behavior

Fixes #9259

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9291)

(cherry picked from commit e6716f2bb4d9588044820f29a7ced0f06789d6ef)

Adding Test.pm with workaround for Perl abs2rel bug

If SRCTOP != BLDTOP, and SRCTOP is given in relative form, e.g.
"./config ../openssl", then a bug in Perl's abs2rel may trigger that directory-
rewriting in __cwd results in wrong entries in %directories under certain
circumstances, e.g. when a test executes run(app(["openssl"]) after indir.

There should not be any need to go to a higher directory from BLDDIR or SRCDIR,
so it should be OK to use them in their absolute form, also resolving all
possible symlinks, right from the start.

Following the File::Spec::Functions bug description (reported to perl.org):

When abs2rel gets a path argument with ..s that are crossing over the ..s
trailing the base argument, the result is wrong.

Example
PATH: /home/goal/test/..
BASE: /home/goal/test/../../base
Good result: ../goal
Bad  result: ../..

Bug verified with File::Spec versions
- 3.6301
- 3.74 (latest)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7031)

(cherry picked from commit 7a2027240e1d01f7f5b209998d1de36af221b34b)

Check for V_ASN1_BOOLEAN/V_ASN1_NULL in X509_ATTRIBUTE_get0_data

The member value.ptr is undefined for those ASN1 types.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9278)

(cherry picked from commit 94f4d58a87eac9c6fe4cb46b998656bd6d6f03a5)

Fix ASN1_TYPE_get/set with type=V_ASN1_BOOLEAN

BOOLEAN does not have valid data in the value.ptr member,
thus don't use it here.

Fixes #9276

[extended tests]

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9278)

(cherry picked from commit 6335f837cfa7eaf1202f2557bf2ba148987226e7)

Add regenerated header files

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/9281)

util/mkerr.pl: Add an inclusion of symhacks.h in all error files

This does no harm, and ensures that the inclusion isn't mistakenly
removed in the generated *err.h where it's actually needed.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit b53c4fe3f92e3d2c5bd9fca1a171cd24f66ef14d)

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9281)

Ensure that rc5 doesn't try to use a key longer than 2040 bits

The maximum key length for rc5 is 2040 bits so we should not attempt to
use keys longer than this.

Issue found by OSS-Fuzz and Guido Vranken.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8834)

(cherry picked from commit 792cb4ee8d82e4b063f707fc9f4992271ffd65ab)

Fix Typos

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9275)