Add random serial# support.

Add -rand_serial to CA command and "serial_rand" config option.

Up RAND_BITS to 159, and comment why: now confirms to CABForum
guidelines (Ballot 164) as well as IETF RFC 5280 (PKIX).

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4185)

Avoid a self-assignment.

Clang is generating a warning over an assignment of a variable to itself.
This occurs on an ASCII based machine where the convert to ASCII macro doesn't
do anything.  The fix is to introduce a temporary variable.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4214)

Dead code elimination.

Remove the exit_main function which is never used.
Remove the exit_options array which is also unreferenced.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4213)

This has been added to avoid the situation where some host ctype.h functions
return true for characters > 127.  I.e. they are allowing extended ASCII
characters through which then cause problems.  E.g. marking superscript '2' as
a number then causes the common (ch - '0') conversion to number to fail
miserably.  Likewise letters with diacritical marks can also cause problems.

If a non-ASCII character set is being used (currently only EBCDIC), it is
adjusted for.

The implementation uses a single table with a bit for each of the defined
classes.  These functions accept an int argument and fail for
values out of range or for characters outside of the ASCII set.  They will
work for both signed and unsigned character inputs.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4102)

Fix ctype arguments.

Cast arguments to the various ctype functions to unsigned char to match their
documentation.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4203)

Safely display SNI (just in case)

Thanks to Hubert Kario for pointing this out.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4193)

Document missing EVP_PKEY_method_* items

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4104)

Check # of arguments for remaining commands.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4201)

Add PKEY_CTX setter tests for TLS1-PRF

PKEY_CTX setters tests were previously present for HKDF and scrypt; this
patch also adds tests for the third currently supported KDF, TLS1-PRF.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4196)

More updates following review feedback

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Updates following feedback on OPENSSL_assert() removal

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Remove OPENSSL_assert() from crypto/x509v3

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Remove OPENSSL_assert() from crypto/x509

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Remove OPENSSL_assert() from crypto/threads_none.c

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Remove OPENSSL_assert() from crypto/pem

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Remove OPENSSL_assert() from crypto/kdf

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Remove OPENSSL_assert() from crypto/hmac

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Remove OPENSSL_assert() from crypto/ec

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Remove OPENSSL_assert() from conf_api.c

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Remove OPENSSL_assert() usage from crypto/bn

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Remove OPENSSL_assert() from various crypto/bio files

bss_dgram.c is deferred until later due to ongoing discussions.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Remove OPENSSL_assert() from crypto/asn1/bio_asn1.c

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3740)

Remove double error messages

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4204)

Fix ui_write in apps/apps.c

It used the default UI reader as fallback instead of the UI writer.

Fixes #4147
Fixes #4195

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4198)

Put thread-fork-init inside a run-once guard

Thanks to Christian Heimes for pointing this out.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4191)

Reorder extensions to put SigAlgs last

Force non-empty padding extension.
When enabled, force the padding extension to be at least 1 byte long.
WebSphere application server cannot handle having an empty
extension (e.g. EMS/EtM) as the last extension in a client hello.
This moves the SigAlgs extension last for TLSv1.2 to avoid this
issue.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3921)

Addressed build failure because of missing #ifdef AF_UNIX guard
CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4067)

Add a comment on expectations in the "tar" target

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4179)

Prepare tarball in dist directory

We changed directory to the wrong directory.
This change also separates the preparation phase from the tarball
building phase.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4179)

Turn on error sensitivity in the "tar" target

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4179)

test/asn1_time_test.c: Better check of signed time_t

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4182)

err/err.c: improve readability.

Reviewed-by: Rich Salz <rsalz@openssl.org>

err/err.c: fix "wraparound" bug in ERR_set_error_data.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix windows build after too aggressive e_os.h removal

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4187)

Remove tests dependence on e_os.h

Apart from ssltest_old.c, the test suite relied on e_os.h for the
OSSL_NELEM macro and nothing else.

The ssltest_old.c also requires EXIT and some socket macros.

Create a new header to define the OSSL_NELEM macro and use that instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4186)

Clear outputs in PKCS12_parse error handling.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4145)

When building a tarball, avoid trying to copy submodules

submodules are directories that we don't want in our tarballs, so
avoid them.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4178)

Determine the number of output columns for the list and help commands using
the command names rather than hard coding it (conditionally).

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4162)

Fix OCSP_basic_verify() cert chain construction in case bs->certs is NULL

Now the certs arg is not any more neglected when building the signer cert chain.
Added case to test/recipes/80-test_ocsp.t proving fix for 3-level CA hierarchy.

See also http://rt.openssl.org/Ticket/Display.html?id=4620

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4124)

sha/asm/keccak1600-armv4.pl: optimize for Thumb-2.

Reduce per-round instruction count in Thumb-2 case by 16%. This is
achieved by folding ldr/str pairs to their double-word counterparts.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix some documentation typos.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4173)

fix some typos

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4172)

Fix two MSVC warnings in apps.c

warning C4996: 'fileno': The POSIX name for this item is deprecated.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4172)

[Win] Fix some test method signatures ...

to halves MSVC warnings.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4172)

Copy dlls into fuzz directory

This should fix the recent AppVeyor failures.

[extended tests]

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4171)

bndiv fuzzer: limit the size of the input to avoid timeout

CLA: trivial

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4119)

Fix coding style of EVP_PKEY_CTX_ctrl_uint64

Code review of @dot-asm pointed out style guide violation; this patch
fixes it.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4166)

Print pathnames for 'version -r'

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4168)

STORE: Add documentation on the expectations for returned names

Returned OSSL_STORE_INFO_NAME typed infos are supposed to be a
canonical URI for the corresponding object.  For example, when using
the 'file' scheme loader, the file name is returned, possibly prefixed
with 'file://'

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3856)

STORE: Add info on the expected post_process callback behavior

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3856)

Fix overzealous cleanup command

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4167)

Add SSL_get_pending_cipher()

The existing function SSL_get_current_cipher() queries the
current session for the ciphersuite in use, but there is no way
for application code to determine what ciphersuite has been
negotiated and will be used in the future, prior to ChangeCipherState
(or the TLS 1.3 equivalent) causing the new cipher to take effect and
become visible in the session information.  Expose this information
to appropriate application callbacks to use during the handshake.

The name SSL_get_pending_cipher() was chosen for compatibility with
BoringSSL's routine of that name.

Improve the note on macro implementations in SSL_get_current_cipher.pod
while here.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4070)

Move ALPN handling from finalizer to delayed call

Commit 02f0274e8c0596dcf7e2d104250232a42c650b96 moved ALPN processing
into an extension finalization function, as the only documented ordering
requirement from previous commits was that ALPN processing occur after
SNI processing, and SNI processing is performed before the extension
finalization step.  However, it is useful for applications'
alpn_select callbacks to run after ciphersuite selection as well -- at
least one application protocol specification (HTTP/2) imposes restrictions
on which ciphersuites are usable with that protocol.  Since it is generally
more preferrable to have a successful TLS connection with a default application
protocol than to fail the TLS connection and not be able to have the preferred
application protocol, it is good to give the alpn_select callback information
about the ciphersuite to be used, so that appropriate restrctions can be
enforced in application code.

Accordingly, split the ALPN handling out into a separate tls_handl_alpn()
function akin to tls_handle_status_request(), called from
tls_post_process_client_hello().  This is an alternative to resuscitating
ssl_check_clienthello_tlsext_late(), something of an awkwward name itself.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4070)

Revert "Add some casts for %j"

This reverts commit c4d2e483a39176a476c56d35879423fe6e33c0cd.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4160)

Use new setup_tests in code of rsa_test

Although this piece of code will not be compiled at current stage, but
there seems a plan to re-open the 'no-rsa' option in the future so this
should be fixed.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4161)

Clear error stack on successful OSSL_STORE_open()

Since OSSL_STORE_open() tries with the 'file' scheme loader first, and
then on the loader implied by the URI if the former fails, the former
leaves an error on the error stack.  This is confusing, so let's clear
the error stack on success.  The implementation uses ERR_set_mark,
ERR_pop_to_mark and ERR_clear_last_mark to make sure caller errors are
preserved as much as possible.

Fixes #4089

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4094)

Add ERR_clear_last_mark()

This allows callers to set a mark, and then clear it without removing
the errors.  Useful in case an error is encountered that should be
returned up the call stack.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4094)

Rename crypto/evp/scrypt.c to crypto/evp/pbe_scrypt.c

There already is a scrypt.c in crypto/kdf/, both becoming script.o or
script.obj.  With some linkers, the same object files name more than
once means one of them is dropped, either when building shared
libraries or when building executables from static libraries.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4164)

File::Glob option ':bsd_glob' doesn't work everywhere, replace w/ a wrapper

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4069)

Consolidate the locations where we have our internal perl modules

Instead of having perl modules under test/testlib, util and util/perl,
consolidate them all to be inside util/perl.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4069)

Add some casts for %j

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4160)

Doc fixes

Write missing prime.pod and srp.pod
Implement -c in find-doc-nits (for command options)
Other fixes to some manpages
Use B<-I<digest|cipher>> notation
Split up multiple flags into a single entry in the synopsis.
Add -1 and missing-help to list command.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4144)

Instantiate when RAND_status() checks

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4150)

Fix some typo and comments
[skip ci]

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4149)

sha/asm/keccak1600-avx512.pl: fix buglet in SHA3_squeeze tail.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Wire SHAKE to EVP.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4137)

Add EVP_DigestFinalXOF, interface to extendable-output functions, XOFs.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4137)

Clarify CLI OCSP documentation

This fixes issue #3043, which ultimately was reported because
documentation was not clear on the meaning of the "-ignore_err" option.
Update both command line documentation and add this option to manpage.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4143)

Fix some Typos and indents

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4108)

Move FuzzerSetRand to separate file.

Use an inline rand.inc; this fixes Google's OSS-Fuzz builds.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4141)

[extended tests] Add steps to update an external test suite

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4139)

Update pyca-cryptography to latest commit

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4139)

no-ec2m fixes

Fix warning and don't use binary field certificate for ECDH CMS
key only test.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4134)

Add alternative CMS P-256 cert

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4134)

Fix no-ec

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4134)

Fix minor type warnings and risk of memory leak in testutil/driver.c

Discussion is in https://github.com/openssl/openssl/issues/4127

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4131)

Don't modify resumed session objects

If s->hit is set, s->session corresponds to a session created on
a previous connection, and is a data structure that is potentially
shared across other SSL objects.  As such, there are thread-safety
issues with modifying the structure without taking its lock (and
of course all corresponding read accesses would also need to take
the lock as well), which have been observed to cause double-frees.

Regardless of thread-safety, the resumed session object is intended
to reflect parameters of the connection that created the session,
and modifying it to reflect the parameters from the current connection
is confusing.  So, modifications to the session object during
ClientHello processing should only be performed on new connections,
i.e., those where s->hit is not set.

The code mostly got this right, providing such checks when processing
SNI and EC point formats, but the supported groups (formerly
supported curves) extension was missing it, which is fixed by this commit.

However, TLS 1.3 makes the suppported_groups extension mandatory
(when using (EC)DHE, which is the normal case), checking for the group
list in the key_share extension processing.  But, TLS 1.3 only [0] supports
session tickets for session resumption, so the session object in question
is the output of d2i_SSL_SESSION(), and will not be shared across SSL
objects.  Thus, it is safe to modify s->session for TLS 1.3 connections.

[0] A psk_find_session callback can also be used, but the restriction that
each callback execution must produce a distinct SSL_SESSION structure
can be documented when the psk_find_session callback documentation is
completed.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4123)

Add missing documentation of the default format for commands.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4122)

Fix memory leak in session cache test

When we are using the internal cache we have to make a copy of the
session before removing it from the parent context's cache, since
we want our copy to still be resumable.  However, SSL_CTX_remove_session()
just detaches the session from the SSL_CTX; it does not free the session.
So, we must call SSL_SESSION_free() ourselves before overwriting the
variable that we dup'd from.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4126)

Add -d flag to list -u details (now normally off)

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4125)

Add missing HTML tag in www_body in s_server.c

In the generated HTML document, the `<pre>` tag is not closed. This patch also has a trivial code-style improvement, unrelated to the bug fix.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4088)

Fix trivial nits in documentaion

Code Health (Tuesday?): Parameters' names are not correct.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4117)

Add XXX_security_bits documentation

This is a 'code health' commit to respond to this round of code health
Tuesday...

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4099)

Test server side session caching

In particular this covers the scenario mentioned in #4014

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4072)

Add an SSL_SESSION_dup() function

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4072)

Fix building without scrypt

Building without the scrypt KDF is now possible, the OPENSSL_NO_SCRYPT
define is honored in code. Previous this lead to undefined references.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4116)

Add test for ECDH CMS key only

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4115)

Support CMS decrypt without a certificate for all key types

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4115)

Add documentation for the scrypt PKEY_METHOD

Added manpage for the new scrypt EVP_PKEY_METHOD KDF interface.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4026)

Add PKEY_METHOD macro tests

Added the pkey_meth_kdf_test tests which test the PKEY_METHOD macros (at
the moment, of HKDF and scrypt).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4026)

Add interface to the scrypt KDF by means of PKEY_METHOD

Add an interface that allows accessing the scrypt KDF as a PKEY_METHOD.
This fixes #4021 (at least for the scrypt portion of the issue).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4026)

Various RAND improvements

Try to put DRBG and rand_bytes buffers in secure heap
Read the TSC fewer times (but it's still not enabled).
Short-circuit return in win RAND_poll_ex; other minor tweaks and
format-fixes.
Use the _bytes version of rdrand/rdseed
Fix ia32cap checks.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4100)

Avoid surpising password dialog in X509 file lookup.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4111)

Make RAND_DRBG fork-safe

Use atfork to count child forks, and reseed DRBG when the counts don't
match.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4101)

Change SETUP_TEST_FIXTURE so that the fixture structure is passed by
reference not by value.  This allows an error return from the setup function.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4083)

Add missing include of cryptlib.h

Also use "" not <> for all include cryptlib

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4082)

Add predicatable RAND_METHOD to test ENGINE

The test ENGINE effectively used a predictable PRNG because it supplied
a bogus implementation of SHA256 which the old version of OpenSSL's PRNG
used. The new DRBG does not use SHA256 so it is no longer predictable
if the SHA256 implementation is replaced. Use an explicit predictable
PRNG instead.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4098)

Use passed drbg, not global one

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4097)

Small typo in manpage of x509(1)

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Andy Polyakov <appro@openssl.org>
GH: #4090

Fix typo in files in crypto folder

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Andy Polyakov <appro@openssl.org>
GH: #4093