[crypto/asn1] Fix multiple SCA vulnerabilities during RSA key validation.

This commit addresses multiple side-channel vulnerabilities present
during RSA key validation.
Private key parameters are re-computed using variable-time functions.

This issue was discovered and reported by the NISEC group at TAU Finland.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9779)

Teach TLSProxy how to parse CertificateRequest messages

We also use this in test_tls13messages to check that the extensions we
expect to see in a CertificateRequest are there.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9780)

(cherry picked from commit dc5bcb88d819de55eb37460c122e02fec91c6d86)

Don't send a status_request extension in a CertificateRequest message

If a TLSv1.3 server configured to respond to the status_request extension
also attempted to send a CertificateRequest then it was incorrectly
inserting a non zero length status_request extension into that message.

The TLSv1.3 RFC does allow that extension in that message but it must
always be zero length.

In fact we should not be sending the extension at all in that message
because we don't support it.

Fixes #9767

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9780)

(cherry picked from commit debb64a0ca43969eb3f043aa8895a4faa7f12b6e)

Fix error handling in x509_lu.c

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9639)

(cherry picked from commit c70e2ec33943d3bd46d3d9950f774307feda832b)

Cleanup includes in rand_unix.c

Fixes #9757

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9761)

(cherry picked from commit 41ffd2ab09d24692c71850ccd7d5ff154196fe01)

Remove ifndef FIPS_MODE from rand_unix.c

This will never be the case for 1.1.1 so removed.

Fixes: comment 1 of #9757

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9762)

Suppress 'No server certificate CA names sent' message

Fixes #9080

Signed-off-by: Billy Brawner <billy@wbrawner.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9710)

(cherry picked from commit 1e8e75d18be8856e753a57771754b9926c3f4264)

Test SSL_set_ciphersuites

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9621)

Fix SSL_set_ciphersuites to set even if no call to SSL_set_cipher_list

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9621)

Fix bogus check for EVP_PKEY mandatory digest in check_cert_usable()

In commit 6aca8d1a5 ("Honour mandatory digest on private key in
has_usable_cert()") I added two checks for the capabilities of the
EVP_PKEY being used. One of them was wrong, as it should only be
checking the signature of the X.509 cert (by its issuer) against the
sigalgs given in a TLS v1.3 signature_algorithms_cert extension.

Remove it.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9705)

Remove duplicate CHANGES text.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9738)

Don't include the DEVRANDOM being seeded logic on Android.

It lacks exposure of the `shm*` functions and should prefer the GETRANDOM
source.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9735)

(cherry picked from commit 280cc0180862ae6664b88d5ea12cb5f599000d36)

Fix NITs in comments and CHANGES for DEVRANDOM seeded check.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9734)

(cherry picked from commit 46a9cc9451213039fd53f62733b2ccd04e853bb2)

Fix SCA vulnerability when using PVK and MSBLOB key formats

This commit addresses a side-channel vulnerability present when
PVK and MSBLOB key formats are loaded into OpenSSL.
The public key was not computed using a constant-time exponentiation
function.

This issue was discovered and reported by the NISEC group at TAU Finland.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9587)

(cherry picked from commit 724339ff44235149c4e8ddae614e1dda6863e23e)

Avoid overflowing FDSET when using select(2).

There is a problem in the rand_unix.c code when the random seed fd is greater
than or equal to FD_SETSIZE and the FDSET overruns its limit and walks the
stack.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9686)

(cherry picked from commit e1f8584d47a499301fba781086af6885fcf21fec)

test/evp_test.c: distinguish parsing errors from processing errors

Parsing functions are at liberty to return:

1:  when parsing on processing of the parsed value succeeded
0:  when the parsed keyword is unknown
-1: when the parsed value processing failed

Some parsing functions didn't do this quite right, they returned 0
when they should have returned -1, causing a message like this:

    Line 123: unknown keyword PeerKey

When this message (which is displayed when the parsing function
returns -1) would have been more appropriate:

    Line 123: error processing keyword PeerKey = ffdhe2048-2-pub

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9682)

(cherry picked from commit f42c225d7f9a0bce0bf46103343402d3f0ad742f)

openssl dgst, openssl enc: check for end of input

The input reading loop in 'openssl dgst' and 'openssl enc' doesn't
check for end of input, and because of the way BIO works, it thereby
won't detect that the end is reached before the read is an error.
With the FILE BIO, an error occurs when trying to read past EOF, which
is fairly much ok, except when the command is used interactively, at
least on Unix.  The result in that case is that the user has to press
Ctrl-D twice for the command to terminate.

The issue is further complicated because both these commands use
filter BIOs on top of the FILE BIO, so a naïve attempt to check
BIO_eof() doesn't quite solve it, since that only checks the state of
the source/sink BIO, and the filter BIO may have some buffered data
that still needs to be read.  Fortunately, there's BIO_pending() that
checks exactly that, if any filter BIO has pending data that needs to
be processed.

We end up having to check both BIO_pending() and BIO_eof().

Thanks to Zsigmond Lőrinczy for the initial effort and inspiration.

Fixes #9355

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9668)

(cherry picked from commit 8ed7bbb411d2a9e0edef928958ad955e0be3d6dd)

Correct documented return value for BIO_get_mem_data()

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9643)

(cherry picked from commit 797a5b7af9d9bbfbcbff4607c10ad5c5595ac785)

Start up DEVRANDOM entropy improvement for older Linux devices.

Improve handling of low entropy at start up from /dev/urandom by waiting for
a read(2) call on /dev/random to succeed.  Once one such call has succeeded,
a shared memory segment is created and persisted as an indicator to other
processes that /dev/urandom is properly seeded.

This does not fully prevent against attacks weakening the entropy source.
An attacker who has control of the machine early in its boot sequence
could create the shared memory segment preventing detection of low entropy
conditions.  However, this is no worse than the current situation.

An attacker would also be capable of removing the shared memory segment
and causing seeding to reoccur resulting in a denial of service attack.
This is partially mitigated by keeping the shared memory alive for the
duration of the process's existence.  Thus, an attacker would not only need
to have called call shmctl(2) with the IPC_RMID command but the system
must subsequently enter a state where no instances of libcrypto exist in
any process.  Even one long running process will prevent this attack.

The System V shared memory calls used here go back at least as far as
Linux kernel 2.0.  Linux kernels 4.8 and later, don't have a reliable way
to detect that /dev/urandom has been properly seeded, so a failure is raised
for this case (i.e. the getentropy(2) call has already failed).

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9595)

[manual merge]

Add a fallback definition for __NR_getrandom for x86 linux

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9639)

(cherry picked from commit 038b381ecf2a988eee4c7bb21074ed0603303bd1)

Fix some pod-page ordering nits

Backport of https://github.com/openssl/openssl/pull/9602

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9632)

doc: fix link in BN_new.pod

Fixes #9622

CLA: trivial

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9627)

(cherry picked from commit faee6b21048623a422d537cdbad24f50c5c21937)

Add a fallback definition for __NR_getrandom for ARM linux

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9619)

(cherry picked from commit 24d932ec842bc5fdbd5e9b519cecf15a56bc74e6)

Test for out-of-bounds write when requesting zero bytes from shake

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9433)

(cherry picked from commit 3ce46435e6ebed69bec0fa3454cc195ced426d42)

Directly return from final sha3/keccak_final if no bytes are requested

Requesting zero bytes from shake previously led to out-of-bounds write
on some platforms.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9433)

(cherry picked from commit a890ef833d114da3430c2f2efd95e01714704d34)

.travis.yml: Use travis_terminate on failure

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9620)

(cherry picked from commit 5be78a88aa922a6c43a83a18dbe252c6a358b8e9)

Fix error handling in X509_chain_up_ref

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9614)

(cherry picked from commit cae665dfa6ccec743a7f39cf80676d7d2d787e56)

INSTALL: clarify documentation of the --api=x.y.z deprecation option

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9599)

(cherry picked from commit bf9d6bb83d009923ceb65753c6dd9fa880e8ba92)

Fix syntax error for the armv4 assembler

The error was from the alignment syntax of the code.
More details:
https://stackoverflow.com/questions/57316823/arm-assembly-syntax-in-vst-vld-commands?noredirect=1#comment101133590_57316823

CLA: trivial

Fixes: #9518

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9518)

(cherry picked from commit 2a17758940657cc3a97b032104a92f0aa304f863)

crypto/engine/eng_openssl.c: define TEST_ENG_OPENSSL_RC4_P_INIT conditionally

When OpenSSL is configured with 'no-stdio', TEST_ENG_OPENSSL_RC4_P_INIT
shouldn't be defined, as that test uses stdio.

Fixes #9597

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9598)

(cherry picked from commit 9f643f54236d6cf0d0d24327acd3b858883f0686)

Extend tests of SSL_check_chain()

Actually supply a chain and then test:
1) A successful check of both the ee and chain certs
2) A failure to check the ee cert
3) A failure to check a chain cert

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9443)

Fix SSL_check_chain()

The function SSL_check_chain() can be used by applications to check that
a cert and chain is compatible with the negotiated parameters. This could
be useful (for example) from the certificate callback. Unfortunately this
function was applying TLSv1.2 sig algs rules and did not work correctly if
TLSv1.3 was negotiated.

We refactor tls_choose_sigalg to split it up and create a new function
find_sig_alg which can (optionally) take a certificate and key as
parameters and find an appropriate sig alg if one exists. If the cert and
key are not supplied then we try to find a cert and key from the ones we
have available that matches the shared sig algs.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9443)

Add missing EBCDIC strings

Fix a few places where calling ossl_isdigit does the wrong thing on
EBCDIC based systems.
Replaced with ascii_isdigit.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9556)

(cherry picked from commit 48102247ff513d4c57b40b19c1d432f37b9e4b02)

Add description in X509_STORE manipulation

Add memory management description in X509_STORE_add_cert, otherwise
users will not be aware that they are leaking memory...

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9484)

(cherry picked from commit 86333b6e0c5c488130ab237e95b8520891b81bf6)

BIO_lookup_ex: Do not retry on EAI_MEMORY

We should not retry on EAI_MEMORY as that error is most probably
fatal and not depending on AI_ADDRCONFIG hint.

Also report the error from the first call if the second call fails
as that one would be most probably the more interesting one.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9535)

(cherry picked from commit 91cb81d40a8102c3d8667629661be8d6937db82b)

BIO_lookup_ex: Always retry the lookup on failure with AI_NUMERICHOST set

Do not try to discern the error return value on
getaddrinfo() failure but when retrying set the AI_NUMERICHOST
to avoid DNS lookups.

Fixes: #9053

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9535)

(cherry picked from commit 7f616a00e9232392927099dca1eca70d0d058665)

Configurations/unit-Makefile.tmpl: Don't clean away dotted files

A local 'make clean' did some sweeping removals of files execpt for
the .git directory.  This is a little too sweeping, as other dotted
files might be cleaned away if they happen to match the pattern that's
searched for.

An example is a symlink .dir-locals.el that would keep disappearing if
you build in the source tree and do a make clean...

So we change this to leave all dotted files alone.  Our builds do not
produce such files anyway, so this is a harmless (or rather, less
harmful) change.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9573)

(cherry picked from commit 853094dbe15a49b334f3488fc99a557abf021c09)

engines/build.info: if the padlock engine is disabled, don't build it!

Fixes #9244

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9572)

Remove some duplicate words from the documentation

Fixup INSTALL and a couple man pages to get rid of "the the" and "in the
in the".

CLA: trivial

Reviewed-by: Paul Yang <kaishen.yy@antfin.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9563)

(cherry picked from commit 3c74e77bd89e3d5400ab4d640149e27863756579)

Add TLS tests for RSA-PSS Restricted certificates

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9553)

(cherry picked from commit 20946b94658416d2fed0b9d9c7adfbe4b7d70515)

Add Restricted PSS certificate and key

Create a PSS certificate with parameter restrictions

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9553)

(cherry picked from commit 39d9ea5e502114a204750f641ca76ff5b4912401)

Ensure RSA PSS correctly returns the right default digest

A default digest of SHA256 was being returned for RSA PSS even if the
PSS parameters indicated a different digest must be used. We change this
so that the correct default digest is returned and additionally mark this
as mandatory for PSS.

This bug had an impact on sig alg selection in libssl. Due to this issue
an incorrect sig alg might be selected in the event that a server is
configured with an RSA-PSS cert with parameter restrictions.

Fixes #9545

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9553)

(cherry picked from commit 9bcc9f973b2a216461dd6f140e47ef647eb733b4)

mention what happens if OPENSSL_NO_RC2 is defined

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9415)

(cherry picked from commit 8c47e55ee69500e31e80458682c6e022294cd0be)

make ecp_nistz256_point_add_vis3() local

fixes #8936

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9132)

(cherry picked from commit 861335001b99cfd665030c50ad37a663536a5c0f)

Fix reversed meaning of error codes

The meaning of the X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY and X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT error codes were still reversed in the X509_STORE_CTX_get_error function documentation.
This used to be the problem also in the verify application documentation, but was fixed on 2010-02-23 in 7d3d178.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9529)

(cherry picked from commit 6d5aa88d74b67b1bc108e121dea687a4ca67e329)

Clarify the INSTALL instructions

Ensure users understand that they need to have appropriate permissions
to write to the install location.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9268)

(cherry picked from commit 7c03bb9fff02b7f08d4654f51f8667584a92cf72)

Change EVP_CIPHER_CTX_iv_length() to return current ivlen for some modes

Note a flag needed to be added since some ssl tests fail if they output any error
(even if the error is ignored). Only ciphers that handle the GET_IV_LEN control set this flag.

Fixes #8330

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9499)

Fix ECDSA_SIG docs

They incorrectly said that i2d_ECDSA_SIG returns 0 on error. In fact it
returns a negative value on error.

We fix this by moving the i2d_ECDSA_SIG/d2i_ECDSA_SIG docs onto the same
page as all the other d2i/i2d docs.

Fixes #9517

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/9533)

(cherry picked from commit bbda79976b5c5095c5e6557311c86c623ba335f1)

Correct the Extended Master Secret string for EBCDIC

The macro TLS_MD_MASTER_SECRET_CONST is supposed to hold the ascii string
"extended master secret". On EBCDIC machines it actually contained the
value "extecded master secret"

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9430)

(cherry picked from commit c1a3f16f735057b45df1803d58f40e4e17b233e5)

Fix SSL_MODE_RELEASE_BUFFERS functionality

At some point in the past do_ssl3_write() used to return the number of
bytes written, or a value <= 0 on error. It now just returns a success/
error code and writes the number of bytes written to |tmpwrit|.

The SSL_MODE_RELEASE_BUFFERS code was still looking at the return code
for the number of bytes written rather than |tmpwrit|. This has the effect
that the buffers are not released when they are supposed to be.

Fixes #9490

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9505)

(cherry picked from commit 8bbf63e48f27c5edaa03e6d87d969c9b6a207f3c)

Add missing accessors for X509 AuthorityKeyIdentifier

Complements commit b383aa208146, which added X509_get0_authority_key_id().

 const ASN1_OCTET_STRING *X509_get0_authority_key_id(X509 *x);
 const GENERAL_NAMES *X509_get0_authority_issuer(X509 *x);      [NEW]
 const ASN1_INTEGER *X509_get0_authority_serial(X509 *x);       [NEW]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9494)

Use allow_early_data_cb from SSL instead of SSL_CTX

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9471)

(cherry picked from commit 59b2cb2638dda3e07385ad36a41f0e141b36987b)

Fix Typos

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9295)

Fix warning C4164 in MSVC.

Fix: crypto\whrlpool\wp_block.c(90) : warning C4164: '_rotl64' : intrinsic function not declared.
Fixes #9487

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9488)

(cherry picked from commit 0c789f59f117ccbb30ffc621216ba776117c7c61)

make RSA and DSA operations throw MISSING_PRIVATE_KEY if needed, adapt ECDSA

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/9466)

(cherry picked from commit 7408f6759f1b0100438ca236ea8f549454aaf2d5)

Use OPENSSL_strlcpy instead of strncpy in e_afalg.c

This avoids a spurious gcc warning:
./config enable-asan --strict-warnings
=>
In function 'afalg_create_sk',
    inlined from 'afalg_cipher_init' at engines/e_afalg.c:545:11:
engines/e_afalg.c:376:5: error: '__builtin_strncpy' output may be
    truncated copying 63 bytes from a string of length 63 [-Werror=stringop-truncation]
  376 |     strncpy((char *) sa.salg_name, ciphername, ALG_MAX_SALG_NAME);
      |     ^~~~~~~

[extended tests]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9478)

(cherry picked from commit 62cc845fc955c8d4de7b703f57bfd8e5854f00f4)

Add weak platform independent PRNG to test framework.

Implement the GNU C library's random(3) pseudorandom number generator.
The algorithm is described: https://www.mscs.dal.ca/~selinger/random/

The rationale is to make the tests repeatable across differing platforms with
different underlying implementations of the random(3) library call.

More specifically: when executing tests with random ordering.

[extended tests]

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9463)

(cherry picked from commit e9a5932d04f6b7dd25b39a8ff9dc162d64a78c22)

Don't generate an unnecessary Diffie-Hellman key in TLS 1.3 clients.

tls_parse_stoc_key_share was generating a new EVP_PKEY public/private
keypair and then overrides it with the server public key, so the
generation was a waste anyway. Instead, it should create a
parameters-only EVP_PKEY.

(This is a consequence of OpenSSL using the same type for empty key,
empty key with key type, empty key with key type + parameters, public
key, and private key. As a result, it's easy to mistakenly mix such
things up, as happened here.)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/9445)

(cherry picked from commit 166c0b98fd6e8b1bb341397642527a9396468f6c)

Fix default installation paths on mingw

Mingw config targets assumed that resulting programs and libraries are
installed in a Unix-like environment and the default installation
prefix was therefore set to '/usr/local'.

However, mingw programs are installed in a Windows environment, and
the installation directories should therefore have Windows defaults,
i.e. the same kind of defaults as the VC config targets.

A difficulty is, however, that a "cross compiled" build can't figure
out the system defaults from environment the same way it's done when
building "natively", so we have to fall back to hard coded defaults in
that case.

Tests can still be performed when cross compiled on a non-Windows
platform, since all tests only depend on the source and build
directory, and otherwise relies on normal local paths.

CVE-2019-1552

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9400)

Make rand_pool buffers more dynamic in their sizing.

The rand pool support allocates maximal sized buffers -- this is typically
12288 bytes in size.  These pools are allocated in secure memory which is a
scarse resource.  They are also allocated per DRBG of which there are up to two
per thread.

This change allocates 64 byte pools and grows them dynamically if required.
64 is chosen to be sufficiently large so that pools do not normally need to
grow.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9428)

(cherry picked from commit a6a66e4511eec0f4ecc2943117a42b3723eb2222)

Allocate DRBG additional data pool from non-secure memory

The additional data allocates 12K per DRBG instance in the
secure memory, which is not necessary. Also nonces are not
considered secret.

[extended tests]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9424)

Remove HEADER_X509_H include detector from apps

The HEADER_X509_H check is redundant, because <openssl/x509.h>
is already included.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9365)

Remove OPENSSL_X509V3_H include detector from openssl/cms.h

The check is redundant, because <openssl/x509v3.h> is included.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9365)

Remove HEADER_BSS_FILE_C module include guard

This include guard inside an object file comes as a surprise and
serves no purpose anymore. It seems like this object file was
included by crypto/threads/mttest.c at some time, but the include
directive was removed in commit bb8abd6.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9365)

Remove external HEADER_SYMHACKS_H include guard

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9365)

Change DH parameters to generate the order q subgroup instead of 2q

This avoids leaking bit 0 of the private key.

Backport-of: #9363

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/9435)

Avoid double clearing some BIGNUMs

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9438)

(cherry picked from commit 82925f9dd0492f2e5f1d80ff46fd59f0704c8fe6)

Cygwin: enable the use of Dl_info and dladdr()

These weren't available in Cygwin at the time our DSO code was
written, but things have changed since.

Fixes #9385

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9402)

(cherry picked from commit 38f6f99cdf0a87345d646d30a764c089c38627ad)

test/enginetest.c: Make sure no config file is loaded

If a config file gets loaded, the tests get disturbed.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9393)

(cherry picked from commit 5800ba761052894145abe7a74a1159df007b6875)

Add value_barriers in constant time select functions

The barriers prevent the compiler from narrowing down the
possible value range of the mask and ~mask in the select
statements, which avoids the recognition of the select
and turning it into a conditional load or branch.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/9217)

(cherry picked from commit 04edd688b3727835f9b2c7cca7e4c963bf3ed2ba)

Cleanup use of X509 STORE locks

Cosmetic changes to use the X509_STORE_lock/unlock functions.
Renamed some ctx variables to store.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9366)

(cherry picked from commit 7a9abccde7b7a5e36efe42d89246f6cfd4d59f44)

s390x assembly pack: fix restoring of SIGILL action

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9381)

(cherry picked from commit c17d60ea293746d7cd06a910ced446edbb6c1eba)

Fix a C++ comment in the refcount.h

Although in a false-conditional code section gcc-4.8.4 flagged this with
a C90 warning :-(

include/internal/refcount.h:108:7: error: C++ style comments are not allowed in ISO C90 [-Werror]
       // under Windows CE we still have old-style Interlocked* functions

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9388)

Fix the return value for SSL_get0_chain_certs()

This function was always returning 0. It should return 1 on success.

Fixes #9374

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/9395)

(cherry picked from commit 7bc82358ae930cfbd353602bc1fd25bfad107350)

Fix SSL_CTX_set_session_id_context() docs

Also, use define rather than sizeof

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9377)

(cherry picked from commit fe9edc9d39c96c965efc4fde12ddf7fa8a852025)

Actually silently ignore GET / OCSP requests

Reviewed-by: Matt Caswell <matt@openssl.org>

Remove DRBG from SSL structure.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9390)

(cherry picked from commit 3d9b33b5e48d82d098a1f8c37dbf616a0d84621c)

Fix wrong lock claimed in x509 dir lookup.

x509 store's objects cache can get corrupted when using dir lookup
method in multithreaded application. Claim x509 store's lock when
accessing objects cache.

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9326)

(cherry picked from commit a161738a708b5e284a4714edc0c976606ea7cb26)

issue-9316: Update return documentation for RAND_set_rand_engine

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9351)

(cherry picked from commit 5fe6e2311df9bbbe347cdc7b3c22ce06e20a0ef9)

Avoid NULL pointer dereference. Fixes #9043.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9059)

(cherry picked from commit 9fd6f7d1cd2a3c8e2bc69dcb8bde8406eb6c2623)

iOS build: Replace %20 with space in config script

CLA: trivial

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9297)

(cherry picked from commit 53fd220c8fc953b603dd13257d6b2e2d1e7eb864)

Fix build error for aarch64 big endian.

Modified rev to rev64, because rev only takes integer registers.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90827
Otherwise, the following error will occur.

Error: operand 1 must be an integer register -- `rev v31.16b,v31.16b'

CLA: trivial

Signed-off-by: Lei Maohui <leimaohui@cn.fujitsu.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9151)

(cherry picked from commit 7b0fceed21c8929e0c6694f57018aa1dbba03e15)

man: fix typo in OPENSSL_fork_prepare.pod

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/9318)

(cherry picked from commit 933a73b9144397a5690a75c69694123a00d3590d)

Fix an endless loop in BN_generate_prime_ex

Happens when trying to generate 4 or 5 bit safe primes.

[extended tests]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9311)

(cherry picked from commit 291f616ced45c924d639d97fc9ca2cbeaad096cf)

Clarify documentation of SSL_CTX_set_verify client side behavior

Fixes #9259

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9291)

(cherry picked from commit e6716f2bb4d9588044820f29a7ced0f06789d6ef)

Adding Test.pm with workaround for Perl abs2rel bug

If SRCTOP != BLDTOP, and SRCTOP is given in relative form, e.g.
"./config ../openssl", then a bug in Perl's abs2rel may trigger that directory-
rewriting in __cwd results in wrong entries in %directories under certain
circumstances, e.g. when a test executes run(app(["openssl"]) after indir.

There should not be any need to go to a higher directory from BLDDIR or SRCDIR,
so it should be OK to use them in their absolute form, also resolving all
possible symlinks, right from the start.

Following the File::Spec::Functions bug description (reported to perl.org):

When abs2rel gets a path argument with ..s that are crossing over the ..s
trailing the base argument, the result is wrong.

Example
PATH: /home/goal/test/..
BASE: /home/goal/test/../../base
Good result: ../goal
Bad  result: ../..

Bug verified with File::Spec versions
- 3.6301
- 3.74 (latest)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7031)

(cherry picked from commit 7a2027240e1d01f7f5b209998d1de36af221b34b)

Check for V_ASN1_BOOLEAN/V_ASN1_NULL in X509_ATTRIBUTE_get0_data

The member value.ptr is undefined for those ASN1 types.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9278)

(cherry picked from commit 94f4d58a87eac9c6fe4cb46b998656bd6d6f03a5)

Fix ASN1_TYPE_get/set with type=V_ASN1_BOOLEAN

BOOLEAN does not have valid data in the value.ptr member,
thus don't use it here.

Fixes #9276

[extended tests]

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9278)

(cherry picked from commit 6335f837cfa7eaf1202f2557bf2ba148987226e7)

Add regenerated header files

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/9281)

util/mkerr.pl: Add an inclusion of symhacks.h in all error files

This does no harm, and ensures that the inclusion isn't mistakenly
removed in the generated *err.h where it's actually needed.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit b53c4fe3f92e3d2c5bd9fca1a171cd24f66ef14d)

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9281)

Ensure that rc5 doesn't try to use a key longer than 2040 bits

The maximum key length for rc5 is 2040 bits so we should not attempt to
use keys longer than this.

Issue found by OSS-Fuzz and Guido Vranken.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8834)

(cherry picked from commit 792cb4ee8d82e4b063f707fc9f4992271ffd65ab)

Fix Typos

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9275)

man: clarify the 'random number generator must be seeded' requirement

The manual pages require for some API functions that the 'random number
generator must be seeded' before calling the function.  Initially, this
was meant literally, i.e. the OpenSSL CSPRNG had to be seeded manually
before calling these functions.

Since version 1.1.1, the CSPRNG is seeded automatically on first use,
so it's not the responsibility of the programmer anymore.  Still, he
needs to be aware that the seeding might fail.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9257)

(cherry picked from commit 262c00882a2fd7cf16672bf467a86f75b4098a7c)

man: fix documentation for RSA_generate_key()

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9260)

(cherry picked from commit 0588be2e01c244c41b0215689f411a6223063fd7)

Excise AES-XTS FIPS check.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9255)

Move 'shared_sigalgs' from cert_st to ssl_st

It was only ever in cert_st because ssl_st was a public structure
and could not be modified without breaking the API.  However, both
structures are now opaque, and thus we can freely change their layout
without breaking applications.  In this case, keeping the shared
sigalgs in the SSL object prevents complications wherein they would
inadvertently get cleared during SSL_set_SSL_CTX() (e.g., as run
during a cert_cb).

Fixes #9099

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9157)

(cherry picked from commit 29948ac80c1388cfeb0bd64539ac1fa6e0bb8990)

Revert "Delay setting the sig algs until after the cert_cb has been called"

This reverts commit 524006dd1b80c1a86a20119ad988666a80d8d8f5.

While this change did prevent the sigalgs from getting inadvertently
clobbered by SSL_set_SSL_CTX(), it also caused the sigalgs to not be
set when the cert_cb runs.  This, in turn, caused significant breakage,
such as SSL_check_chain() failing to find any valid chain.  An alternate
approach to fixing the issue from #7244 will follow.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9157)

(cherry picked from commit 6f34d7bc7d0c7fcd86c6f2772f26e42c925d8505)

Add regression test for #9099

Augment the cert_cb sslapitest to include a run that uses
SSL_check_chain() to inspect the certificate prior to installing
it on the SSL object.  If the check shows the certificate as not
valid in that context, we do not install a certificate at all, so
the handshake will fail later on in processing (tls_choose_sigalg()),
exposing the indicated regression.

Currently it fails, since we have not yet set the shared sigalgs
by the time the cert_cb runs.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9157)

(cherry picked from commit 7cb8fb07e8b71dc1fdcb0de10af7fed4347f6ea4)

Allow AES XTS decryption using duplicate keys.

This feature is enabled by default outside of FIPS builds
which ban such actions completely.

Encryption is always disallowed and will generate an error.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9112)

(cherry picked from commit 2c840201e57e27fa9f1b26a970270a91813e32fe)

Add documentation for CRYPTO_memcmp.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9225)

(cherry picked from commit 0ccff7a7ea82a921f780a483fc91c6b90a1378d9)