Determine the number of output columns for the list and help commands using
the command names rather than hard coding it (conditionally).

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4162)

Fix OCSP_basic_verify() cert chain construction in case bs->certs is NULL

Now the certs arg is not any more neglected when building the signer cert chain.
Added case to test/recipes/80-test_ocsp.t proving fix for 3-level CA hierarchy.

See also http://rt.openssl.org/Ticket/Display.html?id=4620

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4124)

sha/asm/keccak1600-armv4.pl: optimize for Thumb-2.

Reduce per-round instruction count in Thumb-2 case by 16%. This is
achieved by folding ldr/str pairs to their double-word counterparts.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix some documentation typos.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4173)

fix some typos

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4172)

Fix two MSVC warnings in apps.c

warning C4996: 'fileno': The POSIX name for this item is deprecated.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4172)

[Win] Fix some test method signatures ...

to halves MSVC warnings.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4172)

Copy dlls into fuzz directory

This should fix the recent AppVeyor failures.

[extended tests]

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4171)

bndiv fuzzer: limit the size of the input to avoid timeout

CLA: trivial

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4119)

Fix coding style of EVP_PKEY_CTX_ctrl_uint64

Code review of @dot-asm pointed out style guide violation; this patch
fixes it.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4166)

Print pathnames for 'version -r'

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4168)

STORE: Add documentation on the expectations for returned names

Returned OSSL_STORE_INFO_NAME typed infos are supposed to be a
canonical URI for the corresponding object.  For example, when using
the 'file' scheme loader, the file name is returned, possibly prefixed
with 'file://'

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3856)

STORE: Add info on the expected post_process callback behavior

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3856)

Fix overzealous cleanup command

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4167)

Add SSL_get_pending_cipher()

The existing function SSL_get_current_cipher() queries the
current session for the ciphersuite in use, but there is no way
for application code to determine what ciphersuite has been
negotiated and will be used in the future, prior to ChangeCipherState
(or the TLS 1.3 equivalent) causing the new cipher to take effect and
become visible in the session information.  Expose this information
to appropriate application callbacks to use during the handshake.

The name SSL_get_pending_cipher() was chosen for compatibility with
BoringSSL's routine of that name.

Improve the note on macro implementations in SSL_get_current_cipher.pod
while here.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4070)

Move ALPN handling from finalizer to delayed call

Commit 02f0274e8c0596dcf7e2d104250232a42c650b96 moved ALPN processing
into an extension finalization function, as the only documented ordering
requirement from previous commits was that ALPN processing occur after
SNI processing, and SNI processing is performed before the extension
finalization step.  However, it is useful for applications'
alpn_select callbacks to run after ciphersuite selection as well -- at
least one application protocol specification (HTTP/2) imposes restrictions
on which ciphersuites are usable with that protocol.  Since it is generally
more preferrable to have a successful TLS connection with a default application
protocol than to fail the TLS connection and not be able to have the preferred
application protocol, it is good to give the alpn_select callback information
about the ciphersuite to be used, so that appropriate restrctions can be
enforced in application code.

Accordingly, split the ALPN handling out into a separate tls_handl_alpn()
function akin to tls_handle_status_request(), called from
tls_post_process_client_hello().  This is an alternative to resuscitating
ssl_check_clienthello_tlsext_late(), something of an awkwward name itself.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4070)

Revert "Add some casts for %j"

This reverts commit c4d2e483a39176a476c56d35879423fe6e33c0cd.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4160)

Use new setup_tests in code of rsa_test

Although this piece of code will not be compiled at current stage, but
there seems a plan to re-open the 'no-rsa' option in the future so this
should be fixed.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4161)

Clear error stack on successful OSSL_STORE_open()

Since OSSL_STORE_open() tries with the 'file' scheme loader first, and
then on the loader implied by the URI if the former fails, the former
leaves an error on the error stack.  This is confusing, so let's clear
the error stack on success.  The implementation uses ERR_set_mark,
ERR_pop_to_mark and ERR_clear_last_mark to make sure caller errors are
preserved as much as possible.

Fixes #4089

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4094)

Add ERR_clear_last_mark()

This allows callers to set a mark, and then clear it without removing
the errors.  Useful in case an error is encountered that should be
returned up the call stack.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4094)

Rename crypto/evp/scrypt.c to crypto/evp/pbe_scrypt.c

There already is a scrypt.c in crypto/kdf/, both becoming script.o or
script.obj.  With some linkers, the same object files name more than
once means one of them is dropped, either when building shared
libraries or when building executables from static libraries.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4164)

File::Glob option ':bsd_glob' doesn't work everywhere, replace w/ a wrapper

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4069)

Consolidate the locations where we have our internal perl modules

Instead of having perl modules under test/testlib, util and util/perl,
consolidate them all to be inside util/perl.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4069)

Add some casts for %j

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4160)

Doc fixes

Write missing prime.pod and srp.pod
Implement -c in find-doc-nits (for command options)
Other fixes to some manpages
Use B<-I<digest|cipher>> notation
Split up multiple flags into a single entry in the synopsis.
Add -1 and missing-help to list command.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4144)

Instantiate when RAND_status() checks

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4150)

Fix some typo and comments
[skip ci]

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4149)

sha/asm/keccak1600-avx512.pl: fix buglet in SHA3_squeeze tail.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Wire SHAKE to EVP.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4137)

Add EVP_DigestFinalXOF, interface to extendable-output functions, XOFs.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4137)

Clarify CLI OCSP documentation

This fixes issue #3043, which ultimately was reported because
documentation was not clear on the meaning of the "-ignore_err" option.
Update both command line documentation and add this option to manpage.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4143)

Fix some Typos and indents

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4108)

Move FuzzerSetRand to separate file.

Use an inline rand.inc; this fixes Google's OSS-Fuzz builds.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4141)

[extended tests] Add steps to update an external test suite

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4139)

Update pyca-cryptography to latest commit

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4139)

no-ec2m fixes

Fix warning and don't use binary field certificate for ECDH CMS
key only test.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4134)

Add alternative CMS P-256 cert

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4134)

Fix no-ec

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4134)

Fix minor type warnings and risk of memory leak in testutil/driver.c

Discussion is in https://github.com/openssl/openssl/issues/4127

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4131)

Don't modify resumed session objects

If s->hit is set, s->session corresponds to a session created on
a previous connection, and is a data structure that is potentially
shared across other SSL objects.  As such, there are thread-safety
issues with modifying the structure without taking its lock (and
of course all corresponding read accesses would also need to take
the lock as well), which have been observed to cause double-frees.

Regardless of thread-safety, the resumed session object is intended
to reflect parameters of the connection that created the session,
and modifying it to reflect the parameters from the current connection
is confusing.  So, modifications to the session object during
ClientHello processing should only be performed on new connections,
i.e., those where s->hit is not set.

The code mostly got this right, providing such checks when processing
SNI and EC point formats, but the supported groups (formerly
supported curves) extension was missing it, which is fixed by this commit.

However, TLS 1.3 makes the suppported_groups extension mandatory
(when using (EC)DHE, which is the normal case), checking for the group
list in the key_share extension processing.  But, TLS 1.3 only [0] supports
session tickets for session resumption, so the session object in question
is the output of d2i_SSL_SESSION(), and will not be shared across SSL
objects.  Thus, it is safe to modify s->session for TLS 1.3 connections.

[0] A psk_find_session callback can also be used, but the restriction that
each callback execution must produce a distinct SSL_SESSION structure
can be documented when the psk_find_session callback documentation is
completed.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4123)

Add missing documentation of the default format for commands.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4122)

Fix memory leak in session cache test

When we are using the internal cache we have to make a copy of the
session before removing it from the parent context's cache, since
we want our copy to still be resumable.  However, SSL_CTX_remove_session()
just detaches the session from the SSL_CTX; it does not free the session.
So, we must call SSL_SESSION_free() ourselves before overwriting the
variable that we dup'd from.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4126)

Add -d flag to list -u details (now normally off)

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4125)

Add missing HTML tag in www_body in s_server.c

In the generated HTML document, the `<pre>` tag is not closed. This patch also has a trivial code-style improvement, unrelated to the bug fix.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4088)

Fix trivial nits in documentaion

Code Health (Tuesday?): Parameters' names are not correct.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4117)

Add XXX_security_bits documentation

This is a 'code health' commit to respond to this round of code health
Tuesday...

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4099)

Test server side session caching

In particular this covers the scenario mentioned in #4014

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4072)

Add an SSL_SESSION_dup() function

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4072)

Fix building without scrypt

Building without the scrypt KDF is now possible, the OPENSSL_NO_SCRYPT
define is honored in code. Previous this lead to undefined references.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4116)

Add test for ECDH CMS key only

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4115)

Support CMS decrypt without a certificate for all key types

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4115)

Add documentation for the scrypt PKEY_METHOD

Added manpage for the new scrypt EVP_PKEY_METHOD KDF interface.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4026)

Add PKEY_METHOD macro tests

Added the pkey_meth_kdf_test tests which test the PKEY_METHOD macros (at
the moment, of HKDF and scrypt).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4026)

Add interface to the scrypt KDF by means of PKEY_METHOD

Add an interface that allows accessing the scrypt KDF as a PKEY_METHOD.
This fixes #4021 (at least for the scrypt portion of the issue).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4026)

Various RAND improvements

Try to put DRBG and rand_bytes buffers in secure heap
Read the TSC fewer times (but it's still not enabled).
Short-circuit return in win RAND_poll_ex; other minor tweaks and
format-fixes.
Use the _bytes version of rdrand/rdseed
Fix ia32cap checks.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4100)

Avoid surpising password dialog in X509 file lookup.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4111)

Make RAND_DRBG fork-safe

Use atfork to count child forks, and reseed DRBG when the counts don't
match.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4101)

Change SETUP_TEST_FIXTURE so that the fixture structure is passed by
reference not by value.  This allows an error return from the setup function.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4083)

Add missing include of cryptlib.h

Also use "" not <> for all include cryptlib

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4082)

Add predicatable RAND_METHOD to test ENGINE

The test ENGINE effectively used a predictable PRNG because it supplied
a bogus implementation of SHA256 which the old version of OpenSSL's PRNG
used. The new DRBG does not use SHA256 so it is no longer predictable
if the SHA256 implementation is replaced. Use an explicit predictable
PRNG instead.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4098)

Use passed drbg, not global one

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4097)

Small typo in manpage of x509(1)

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Andy Polyakov <appro@openssl.org>
GH: #4090

Fix typo in files in crypto folder

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Andy Polyakov <appro@openssl.org>
GH: #4093

Fix typo in HKDF example documentation

Out-of-bounds array access in the example documentation of
EVP_PKEY_CTX_set_hkdf_md fixed.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4081)

Add entropy sanity check

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4092)

Set randomness buffer pointer in get_entropy calls.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4092)

Consolidate to a single asn1_time_from_tm() function

Add missing ASN1_TIME functions

Do some cleanup of the ASN1_TIME code.
Add ASN1_TIME_normalize() to normalize ASN1_TIME structures.
Add ASN1_TIME_compare() to compare two ASN1_TIME structures.
Add ASN1_TIME_cmp_time_t() to compare an ASN1_TIME to time_t
(generic version of ASN1_UTCTIME_cmp_time_t()).

Replace '0' .. '9' compares with isdigit()

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2753)

Test fixtures changed to pointers.

Change the fixture types to pointers to structures that are heap allocated in the tests that use SETUP_TEST_FIXTURE.  This will permit error returns from the setup function and allow for future running tests in parallel.

Also removed a call of `exit(2)` which allows the remaining tests to run if one fails to initialise.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4071)

recipes/80-test_ca.t: make it work with spaces in pathnames.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add RAND_priv_bytes() for private keys

Add a new global DRBG for private keys used by RAND_priv_bytes.

Add BN_priv_rand() and BN_priv_rand_range() which use RAND_priv_bytes().
Change callers to use the appropriate BN_priv... function.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4076)

Add a DRBG to each SSL object

Give each SSL object it's own DRBG, chained to the parent global
DRBG which is used only as a source of randomness into the per-SSL
DRBG.  This is used for all session, ticket, and pre-master secret keys.
It is NOT used for ECDH key generation which use only the global
DRBG. (Doing that without changing the API is tricky, if not impossible.)

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4050)

Switch from ossl_rand to DRBG rand

If RAND_add wraps around, XOR with existing. Add test to drbgtest that
does the wrap-around.

Re-order seeding and stop after first success.

Add RAND_poll_ex()

Use the DF and therefore lower RANDOMNESS_NEEDED.  Also, for child DRBG's,
mix in the address as the personalization bits.

Centralize the entropy callbacks, from drbg_lib to rand_lib.
(Conceptually, entropy is part of the enclosing application.)
Thanks to Dr. Matthias St Pierre for the suggestion.

Various code cleanups:
    -Make state an enum; inline RANDerr calls.
    -Add RAND_POLL_RETRIES (thanks Pauli for the idea)
    -Remove most RAND_seed calls from rest of library
    -Rename DRBG_CTX to RAND_DRBG, etc.
    -Move some code from drbg_lib to drbg_rand; drbg_lib is now only the
     implementation of NIST DRBG.
    -Remove blocklength

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4019)

Move ossl_assert

Move the definition of ossl_assert() out of e_os.h which is intended for OS
specific things. Instead it is moved into internal/cryptlib.h.

This also changes the definition to remove the (int) cast.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4073)

remove horrible pragma macro and remove __owur from SSL_CTX_add_session() declaration

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4014)

Remove the obsolete misleading comment and code related to it.

The comment "The following should not return 1, otherwise, things
are very strange" is from the very first commit of OpenSSL. The
really meaning of the comment is if the identical session can be
found from internal cache after calling get_session_cb but not
found before calling get_session_cb, it is just strange.

The value 1 was originated from the old doc of SSLeay, reversed
from the actual return value of SSL_CTX_add_session().

Anyway either return value of SSL_CTX_add_session() should not
interrupt the session resumption process. So the checking of
return value of SSL_CTX_add_session() is not necessary.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4014)

Allow use of long name for KDFs

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4079)

Fix indentation

Conform to coding guidelines.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

Added differentiation between missing secret and missing seed

This was previously mistakenly handled as a single error code.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

Changed use of EVP_PKEY_CTX_md() and more specific error codes

Changed HKDF to use EVP_PKEY_CTX_md() (review comment of @snhenson) and
introduced more specific error codes (not only indicating *that* some
parameter is missing, but actually *which* one it is).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

More error handling to HKDF and one more case in TLS1-PRF

HKDF now handles an invalid digest like TLS1-PRF does (i.e., returns
KDF_R_INVALID_DIGEST if the passed digest is not known). Both KDFs now
set the error code KDF_R_UNKNOWN_PARAMETER_TYPE if a type was passed
that is not recognized. This will have the effect of improving debugging
output in case a user uses "openssl pkeyutl -kdf ..." in a wrong way and
result in an actual error code (instead of just "failure" and an empty
error stack).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

Set error when HKDF used without parameters

Introduce KDF_F_PKEY_HKDF_DERIVE and return the KDF_R_MISSING_PARAMETER
error code when required parameters have not been set. This will make
"openssl pkeyutl -kdf HKDF" return a meaningful error message instead of
simply "Public Key operation error".

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

sha/asm/keccak1600-armv4.pl: improve non-NEON performance by ~10%.

This is achieved mostly by ~10% reduction of amount of instructions
per round thanks to a) switch to KECCAK_2X variant; b) merge of
almost 1/2 rotations with logical instructions. Performance is
improved on all observed processors except on Cortex-A15. This is
because it's capable of exploiting more parallelism and can execute
original code for same amount of time.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4057)

Simplify some of the sslapitest code.

Removing the use of SETUP_TEST_FIXTURE reduces complxity in those tests that
used it.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4066)

Adding NID_hmac_sha1 and _md5 to builtin_pbe[]

The OID for {1 3 6 1 5 5 8 1 2} HMAC-SHA1 (NID_hmac_sha1) is explicitly
referenced by RFC 2510, RFC 3370, and RFC 4210. This is essential for the
common implementations of CMP (Certificate Managing Protocol, RFC4210).

HMAC-MD5's OID {1 3 6 1 5 5 8 1 1} (NID_hmac_md5) is in the same branch and
it seems to generally exist (-> Internet search), but it is unclear where it is
actually defined as it appears not to be referenced by RFCs and practically
rather unused.

Those OIDs are both duplicates to OIDs from an RSA OID branch, which are already
included in builtin_pbe[]:

HMAC-SHA1 also has another OID defined in PKCS#5/RFC2898 (NID_hmacWithSHA1).

It is also unclear where the other OID for HMAC-MD5 (NID_hmacWithMD5) from the
RSA branch is officially specified, as only HMAC-SHA1 from PKCS#5 was found to be
defined. Anyway, HMAC-MD5 likely only plays a neglectable role in the future.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/3811)

Remove EXECUTE_TEST_NO_TEARDOWN.

Simplify the only test that uses this macro so it doesn't need it anymore.
Clean up the formatting a little.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4034)

sha/keccak1600.c: choose more sensible default parameters.

"More" refers to the fact that we make active BIT_INTERLEAVE choice
in some specific cases. Update commentary correspondingly.

Reviewed-by: Rich Salz <rsalz@openssl.org>

asn1/a_time.c: make handling of 'fractional point' formally correct.

Even though tm->length >= 15 && v[14] == '.' works in practice,
[because "YYYYMMDDHHMMSS." would be rejected as invalid by
asn1_time_to_tm,] formal correctness with respect to buffer
overstep in few lines vicinity improves readability.

[Also fold one if condition and improve expression readability.]

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4058)

Add test case for ASN1_TIME_print

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4061)

Add EC key generation paragraph in doc/HOWTO/keys.txt

Seems this documentation is not dead, so add this missing part

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4037)

RSA_get0_ functions permit NULL parameters

Document that the RSA_get0_ functions permit a NULL BIGNUM **. Those output parameters are ignored.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4064)

Add a test to check we get a new session even if s->hit is true in TLSv1.3

In TLSv1.3 we can resume, but still get a new session. This adds a test to
make sure that is happening.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4068)

Fix new_session_cb calls in TLSv1.3

If a new_session_cb is set then it was only ever getting invoked if !s->hit
is true. This is sensible for <=TLSv1.2 but does not work for TLSv1.3.

Fixes #4045

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4068)

Add '-ext' option to display extensions in 'x509'

This is to address issue #3932. Support comma-separated string
to specify what extensions to be displayed.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4016)

Remove redundant variable

[to be squashed]

Fix typo in documents

I scanned all files in the doc folder with a spell checker (https://github.com/EWSoftware/VSSpellChecker).
This patch (hopefully) corrected all spell errors that it found.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4065)

Fix an information leak in the RSA padding check code.
The memory blocks contain secret data and must be
cleared before returning to the system heap.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4062)

Fix SSL_set_tlsext_debug_callback/-tlsextdebug

Some extensions were being displayed twice, before they were parsed, and
again after they were parsed.
The supported_versions extension was not being fully displayed, as it
was processed differently than other extensions.

Move the debug callback to where the extensions are first collected, to
catch all the extensions as they come in, so they are ordered correctly.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3911)

Fix the names of older ciphers.

The names of these ciphers have an "SSL_" prefix, but the RFC names use
"TLS_":
https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-4

This dates back to these ciphers being originally defined in SSLv3. As
SSLv3 is on its way out anyway and this is a new set of APIs,
consistently use the TLS names.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4007)

Fix errors in SSL_state_string_long

TLS_ST_SR_NEXT_PROTO means "SSLv3/TLS read next proto"
Fix typo in the message for TLS_ST_SW_CERT_STATUS

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4054)

app_isdir() cleanup

I think it's better to use `GetFileAttributes` to obtain the attributes
of a file than `FindFirstFile`. If the input name contains `*`, this
function should return failure rather than check whether the first match
happens to be a file or a directory.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/3991)

bn/bn_lcl.h: restore formatting.

Reviewed-by: Rich Salz <rsalz@openssl.org>