crypto/engine/eng_devcrypto.c: add digest copy

Copying an EVP_MD_CTX, including the implementation local bits, is a
necessary operation.  In this case, though, it's the same as
initializing the local bits to be "copied to".

Fixes #7495

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7506)

(cherry picked from commit 36af124bfb209b49cb92a5fb9fab627d9cd4a44b)

apps: Stop pretending to care about Netscape keys

The documentation says some commands care, but the code says differently.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7440)

(cherry picked from commit d91d443f0d26262148d1dc9d29f9fdf025b958ca)

Add a constant time flag to one of the bignums to avoid a timing leak.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7549)

(cherry picked from commit 00496b6423605391864fbbd1693f23631a1c5239)

Configuration: when building the dirinfo structure, include shared_sources

This makes sure that any resulting directory target in the build files
also depend on object files meant for shared libraries.

As a side effect, we move the production of the dirinfo structure from
common.tmpl to Configure, to make it easier to check the result.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7452)

(cherry picked from commit b6e660754c2e799cffe4906269fcace0e07c73bc)

Configurations/15-android.conf: detect NDK llvm-ar.

This excluses user from additional PATH adjustments in case NDK has
llvm-ar.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7443)

(cherry picked from commit ddf2e8c8ccad49cff0d6a0a0b26ca892ec4ce33a)

Configurations/15-android.conf: fix implicit __ANDROID_API__ handling.

03ad7c009e16a233c733098db3169c560142ccd3 failed if one didn't pass
explicit -D__ANDROID_API__=N :-(

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7443)

(cherry picked from commit 71f2b3171e8b94b0fbdc7fab2d7ca4ea66edec2b)

Fix a race condition in drbg_add

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7523)

(cherry picked from commit 4011bab1f85d4429bad1e9388bed90a8d0da5639)

Add a client_cert_cb test

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/7413)

(cherry picked from commit 6e46c065b9b97212d63ef1f321b08fb7fa6b320d)

Don't call the client_cert_cb immediately in TLSv1.3

In TLSv1.2 and below a CertificateRequest is sent after the Certificate
from the server. This means that by the time the client_cert_cb is called
on receipt of the CertificateRequest a call to SSL_get_peer_certificate()
will return the server certificate as expected. In TLSv1.3 a
CertificateRequest is sent before a Certificate message so calling
SSL_get_peer_certificate() returns NULL.

To workaround this we delay calling the client_cert_cb until after we
have processed the CertificateVerify message, when we are doing TLSv1.3.

Fixes #7384

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/7413)

(cherry picked from commit e45620140fce22c3251440063bc17440289d730c)

rand_drbg.h: include <openssl/obj_mac.h>

The RAND_DRBG_TYPE preprocessor define depends on a NID, so  we have
to include obj_mac.h to make the header selfcontained.

Fixes #7521

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7524)

(cherry picked from commit cf3d6ef7af7d6c47f5ccd4ce58d822972018b21c)

Fix MSVC warning C4819

CLA: trivial

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7444)

(cherry picked from commit cf4eea12046445fc418507d2d5e14956b4353495)

test/evp_test.c: don't misuse pkey_test_ctrl() in mac_test_run()

pkey_test_ctrl() was designed for parsing values, not for using in
test runs.  Relying on its returned value when it returned 1 even for
control errors made it particularly useless for mac_test_run().

Here, it gets replaced with a MAC specific control function, that
parses values the same way but is designed for use in a _run() rather
than a _parse() function.

This uncovers a SipHash test with an invalid control that wasn't
caught properly.  After all, that stanza is supposed to test that
invalid control values do generate an error.  Now we catch that.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7500)

(cherry picked from commit ce5d64c79c4d809ece8fe28a5b62915467a1c0e1)

Add a test where we reuse the EVP_PKEY_CTX for two HKDF test runs

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7501)

(cherry picked from commit 10d5b415f9e973f44f18eeaf2713868ec813e1d7)

Reset the HKDF state between operations

Fixes #7497

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7501)

(cherry picked from commit ca55d70be031746daddd8bd0611db54ed81f1737)

Windows build: build foo.d after foo.obj

We made the build of foo.obj depend on foo.d, meaning the latter gets
built first.  Unfortunately, the way the compiler works, we are forced
to redirect all output to foo.d, meaning that if the source contains
an error, the build fails without showing those errors.

We therefore remove the dependency and force the build of foo.d to
always happen after build of foo.obj.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7469)

(cherry picked from commit ecc347f5f53a9f2edc2805d50cba07db64267e8a)

ssl/statem: Don't compare size_t with less than zero

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7194)

(cherry picked from commit 60690b5b8396d7d5234cd067206190fb8aca78d2)

VMS & cryptoerr.h: include symhacks.h

Needed to clear a clash between ERR_load_CRYPTO_strings and
ERR_load_crypto_strings

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7194)

(cherry picked from commit cc3a2e4f51679d99507a979f9a920f6068473aa2)

apps/rehash.c: Convert ISO-8859-1 to UTF-8

Believe it or not, the VMS C compiler is remarking on this

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7194)

(cherry picked from commit 4602cc85aff35b7aa5e27dc57ead89f0867637f9)

Rework and simplify resource flow in drbg_add

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7504)

(cherry picked from commit f9e43929c46b38667f67e02765fe0f1c0d3061d6)

randfile.c: fix a Coverity warning

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7510)

(cherry picked from commit 040a03470c7c5bf95fe8e6143db7bef357a22833)

Timing vulnerability in ECDSA signature generation (CVE-2018-0735)

Preallocate an extra limb for some of the big numbers to avoid a reallocation
that can potentially provide a side channel.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7486)

(cherry picked from commit 99540ec79491f59ed8b46b4edf130e17dc907f52)

Timing vulnerability in DSA signature generation (CVE-2018-0734).

Avoid a timing attack that leaks information via a side channel that
triggers when a BN is resized.  Increasing the size of the BNs
prior to doing anything with them suppresses the attack.

Thanks due to Samuel Weiser for finding and locating this.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7486)

(cherry picked from commit a9cfb8c2aa7254a4aa6a1716909e3f8cb78049b6)

DSA mod inverse fix

There is a side channel attack against the division used to calculate one of
the modulo inverses in the DSA algorithm.  This change takes advantage of the
primality of the modulo and Fermat's little theorem to calculate the inverse
without leaking information.

Thanks to Samuel Weiser for finding and reporting this.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7487)

(cherry picked from commit 415c33563528667868c3c653a612e6fc8736fd79)

drbg_lib: avoid NULL pointer dereference in drbg_add

Found by Coverity Scan

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7511)

(cherry picked from commit 59f90557dd6e35cf72ac72016609d759ac78fcb9)

iOS config targets: disable "async" by default

This also gave enough reason to collect the stuff that's common for
all iOS config targets into the template "ios-common".

Fixes #7318

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7403)

(cherry picked from commit 04e3f9a114c2c142356ef1639d68397a72e0c7f8)

Configure: Improve warning if no random seed source was configured

The new Configure summary box (41349b5e6db) now hides the warning
about the missing seed source (2805ee1e095) too much. To make it
more visible again, add warning markers.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7509)

(cherry picked from commit caa8595276c60f009c8621ad466338d2ae39fb86)

Fix data race in RAND_DRBG_generate

Fixes #7394

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7399)

(cherry picked from commit a83dc59afa2e0207180d7218efed19b20d48de95)

RAND_add()/RAND_seed(): fix failure on short input or low entropy

Commit 5b4cb385c18a (#7382) introduced a bug which had the effect
that RAND_add()/RAND_seed() failed for buffer sizes less than
32 bytes. The reason was that now the added random data was used
exlusively as entropy source for reseeding. When the random input
was too short or contained not enough entropy, the DRBG failed
without querying the available entropy sources.

This commit makes drbg_add() act smarter: it checks the entropy
requirements explicitely. If the random input fails this check,
it won't be added as entropy input, but only as additional data.
More precisely, the behaviour depends on whether an os entropy
source was configured (which is the default on most os):

- If an os entropy source is avaible then we declare the buffer
  content as additional data by setting randomness to zero and
  trigger a regular   reseeding.

- If no os entropy source is available, a reseeding will fail
  inevitably. So drbg_add() uses a trick to mix the buffer contents
  into the DRBG state without forcing a reseeding: it generates a
  dummy random byte, using the buffer content as additional data.

Related-to: #7449

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7456)

(cherry picked from commit 8817215d5c52a76f2b184b624bde4df8556dee6d)

Backport some DRBG renamings and typo fixes

In commit 8bf366519661 some renamings andd typo fixes were made
while adding back the DRBG-HMAC and DRBG-HASH implementation.
Since the commit could not be backported, a lot of unnecessary
differences between master and 1.1.1 were introduced.

These differences result in tiresome merge conflicts when
cherry-picking. To minimize these merge-conflicts, this patch
ports all 'non-feature' changes of commit 8bf366519661
(e.g., renamings of private variables, fixes of typographical
errors, comment changes) manually back to 1.1.1.

The commits a83dc59afa2e (#7399) and 8817215d5c52 (#7456)
failed to cherry-pick previously to 1.1.1, with this patch
they both cherry-pick without conflicts.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7505)

Add a test for duplicated DTLS records

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7414)

(cherry picked from commit f1358634af5b84be22cb20fff3dcb613f5f8c978)

Properly handle duplicated messages from the next epoch

Since 1fb9fdc30 we may attempt to buffer a record from the next epoch
that has already been buffered. Prior to that this never occurred.

We simply ignore a failure to buffer a duplicated record.

Fixes #6902

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7414)

(cherry picked from commit 840facc3ccab481e1a0bc2cb6e7740e362df7422)

Fix a possible crash in rand_drbg_get_entropy

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7474)

(cherry picked from commit 21311777ad8b8c2f2fb8eb7a4f9618ceae1e043d)

Configure: Reword the summary output

In commit 820e414d2830 (pr #5247) the summary output of the
Configure command was optimized towards instructing people
how to create issue reports.

It turned out that the wording of this message can confuse new
OpenSSL users and make them think that they are seeing an error
message. This commit makes the summary output start with a success
to prevent a misunderstanding. Also it gives more hints to new
OpenSSL users.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7499)

(cherry picked from commit 41349b5e6dbd72bfbeaf3cf189d64914240628e3)

RAND_load_file(): avoid adding small chunks to RAND_add()

Increase the load buffer size such that it exceeds the chunk
size by a comfortable amount. This is done to avoid calling
RAND_add() with a small final chunk. Instead, such a small
final chunk will be added together with the previous chunk
(unless it's the only one).

Related-to: #7449

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7456)

RAND_load_file(): return error if reseeding failed

The failure of RAND_load_file was only noticed because of the
heap corruption which was reported in #7499 and fixed in commit
5b4cb385c18a. To prevent this in the future, RAND_load_file()
now explicitly checks RAND_status() and reports an error if it
fails.

Related-to: #7449

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7456)

Windows: Produce a static version of the public libraries, always

When building shared libraries on Windows, we had a clash between
'libcrypto.lib' the static routine library and 'libcrypto.lib' the
import library.

We now change it so the static versions of our libraries get '_static'
appended to their names.  These will never get installed, but can
still be used for our internal purposes, such as internal tests.

When building non-shared, the renaming mechanism doesn't come into
play.  In that case, the static libraries 'libcrypto.lib' and
'libssl.lib' are installed, just as always.

Fixes #7492

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7496)

(cherry picked from commit b3023ced6b6a4aece6f4d4ec1f6a93b1c03712b6)

RAND: ensure INT32_MAX is defined

This value is used to set DRBG_MAX_LENGTH

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7467)

(cherry picked from commit f81b043ad856d8b9af5239a4978f8bd4b965dab9)

RAND_add(): fix heap corruption in error path

This bug was introduced by #7382 which enhanced RAND_add() to
accept large buffer sizes. As a consequence, RAND_add() now fails
for buffer sizes less than 32 bytes (i.e. less than 256 bits).
In addition, rand_drbg_get_entropy() forgets to reset the attached
drbg->pool in the case of an error, which leads to the heap corruption.

The problem occurred with RAND_load_file(), which reads the file in
chunks of 1024 bytes each. If the size of the final chunk is less than
32 bytes, then RAND_add() fails, whence RAND_load_file() fails
silently for buffer sizes n = k * 1024 + r with r = 1,...,31.

This commit fixes the heap corruption only. The other issues will
be addressed in a separate pull request.

Thanks to Gisle Vanem for reporting this issue.

Fixes #7449

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7455)

(cherry picked from commit 5b4cb385c18a5bb4e118e300f1c746bf7c2a5628)

build file templates: have targets for all shared library names

We only had targets for the "simple" shared library names (libfoo.so
and not libfoo.so.x.y on Unix, import library libfoo.lib but not
libfoo.dll on Windows).  This has created some confusion why it wasn't
possible to rebuild the less "simple" name directly (just as an
example, someone who mistook the import library on Windows for a
static library, removed the DLL and then found it was difficult to
rebuild directly), so we change the target to include all possible
names.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7451)

(cherry picked from commit d8cac50b023be249cdaba054f43acecf17025ce4)

Don't complain and fail about unknown TLSv1.3 PSK identities in s_server

An unknown PSK identity could be because its actually a session resumption
attempt. Sessions resumptions and external PSKs are indistinguishable so
the callbacks need to fail gracefully if they don't recognise the identity.

Fixes #7433

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7434)

(cherry picked from commit 2d015189b97c60b67e10aed320230357bf6b200f)

Buffer a ClientHello with a cookie received via DTLSv1_listen

Previously when a ClientHello arrives with a valid cookie using
DTLSv1_listen() we only "peeked" at the message and left it on the
underlying fd. This works fine for single threaded applications but for
multi-threaded apps this does not work since the fd is typically reused for
the server thread, while a new fd is created and connected for the client.
By "peeking" we leave the message on the server fd, and consequently we
think we've received another valid ClientHello and so we create yet another
fd for the client, and so on until we run out of fds.

In this new approach we remove the ClientHello and buffer it in the SSL
object.

Fixes #6934

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/7375)

(cherry picked from commit 079ef6bd534d2f708d8013cfcd8ea0d2f600c788)

Use the read and write buffers in DTLSv1_listen()

Rather than using init_buf we use the record layer read and write buffers
in DTLSv1_listen(). These seem more appropriate anyway and will help with
the next commit.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/7375)

(cherry picked from commit 2fc4c77c3f06443f4c476f6f58d83e5e108d1dce)

Test DTLS cookie generation and verification

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7431)

(cherry picked from commit edcd29efd32c51f298ad5ab438e2d4cc5411e9a9)

Fix a DTLS memory leak

Fixes #7428

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7431)

(cherry picked from commit 01666a8c1db3ecfb999e1a8f2c5436d114f95681)

Configurations/15-android.conf: add support for "standalone toolchain".

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7400)

(cherry picked from commit 03ad7c009e16a233c733098db3169c560142ccd3)

arch/async_posix.h: improve portability.

{make|swap|get|set}context are removed in POSIX.1-2008, but glibc
apparently keeps providing it.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7420)

(cherry picked from commit 9d71a24ebf57e7157888af1ca587eafe914bf96f)

Fix tls_cbc_digest_record is slow using SHA-384 and short messages

The formula used for this is now

kVarianceBlocks = ((255 + 1 + md_size + md_block_size - 1) / md_block_size) + 1

Notice that md_block_size=64 for SHA256, which results on the
magic constant kVarianceBlocks = 6.
However, md_block_size=128 for SHA384 leading to kVarianceBlocks = 4.

CLA:trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7342)

(cherry picked from commit cb8164b05e3bad5586c2a109bbdbab1ad65a1a6f)

Apply self-imposed path length also to root CAs

Also, some readers of the code find starting the count at 1 for EE
cert confusing (since RFC5280 counts only non-self-issued intermediate
CAs, but we also counted the leaf).  Therefore, never count the EE
cert, and adjust the path length comparison accordinly.  This may
be more clear to the reader.

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit dc5831da59e9bfad61ba425d886a0b06ac160cd6)

Only CA certificates can be self-issued

At the bottom of https://tools.ietf.org/html/rfc5280#page-12 and
top of https://tools.ietf.org/html/rfc5280#page-13 (last paragraph
of above https://tools.ietf.org/html/rfc5280#section-3.3), we see:

   This specification covers two classes of certificates: CA
   certificates and end entity certificates.  CA certificates may be
   further divided into three classes: cross-certificates, self-issued
   certificates, and self-signed certificates.  Cross-certificates are
   CA certificates in which the issuer and subject are different
   entities.  Cross-certificates describe a trust relationship between
   the two CAs.  Self-issued certificates are CA certificates in which
   the issuer and subject are the same entity.  Self-issued certificates
   are generated to support changes in policy or operations.  Self-
   signed certificates are self-issued certificates where the digital
   signature may be verified by the public key bound into the
   certificate.  Self-signed certificates are used to convey a public
   key for use to begin certification paths.  End entity certificates
   are issued to subjects that are not authorized to issue certificates.

that the term "self-issued" is only applicable to CAs, not end-entity
certificates.  In https://tools.ietf.org/html/rfc5280#section-4.2.1.9
the description of path length constraints says:

   The pathLenConstraint field is meaningful only if the cA boolean is
   asserted and the key usage extension, if present, asserts the
   keyCertSign bit (Section 4.2.1.3).  In this case, it gives the
   maximum number of non-self-issued intermediate certificates that may
   follow this certificate in a valid certification path.  (Note: The
   last certificate in the certification path is not an intermediate
   certificate, and is not included in this limit.  Usually, the last
   certificate is an end entity certificate, but it can be a CA
   certificate.)

This makes it clear that exclusion of self-issued certificates from
the path length count applies only to some *intermediate* CA
certificates.  A leaf certificate whether it has identical issuer
and subject or whether it is a CA or not is never part of the
intermediate certificate count.  The handling of all leaf certificates
must be the same, in the case of our code to post-increment the
path count by 1, so that we ultimately reach a non-self-issued
intermediate it will be the first one (not zeroth) in the chain
of intermediates.

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit ed422a2d0196ada0f5c1b6e296f4a4e5ed69577f)

EVP module documentation pass

Replace ECDH_KDF_X9_62() with internal ecdh_KDF_X9_63()

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/7345)

(cherry picked from commit ffd89124bdfc9e69349492c3f15383bb35520a11)

Fix: 'openssl ca' command crashes when used with 'rand_serial' option

Commit ffb46830e2df introduced the 'rand_serial' option. When it is used,
the 'serialfile' does not get initialized, i.e. it remains a NULL pointer.
This causes a crash when the NULL pointer is passed to the rotate_serial()
call.

This commit fixes the crash and unifies the pointer checking before
calling the rotate_serial() and save_serial() commands.

Fixes #7412

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7417)

(cherry picked from commit aeec793b4bee929cef8ae35ec4b5a783a6e1d7ed)

Build file templates: look at *all* defines

When looking at configured macro definitions, we must look at both
what comes from the config target AND what comes from user
configuration.

Fixes #7396

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/7402)

(cherry picked from commit 92ebf6c4c21ff4b41ba1fd69af74b2039e138114)

Add a missing check on s->s3->tmp.pkey

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7405)

(cherry picked from commit 61bef9bde09dc6099a7c59baa79898e3b003fec3)

print() is a function in Python 3

CLA: trivial

Discovered via #7410 @ https://travis-ci.org/openssl/openssl/jobs/442003489#L440

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7403)

(cherry picked from commit 83e4533a71c5c78278e9763552a5e5f1806473ee)

DRBG: fix reseeding via RAND_add()/RAND_seed() with large input

In pull request #4328 the seeding of the DRBG via RAND_add()/RAND_seed()
was implemented by buffering the data in a random pool where it is
picked up later by the rand_drbg_get_entropy() callback. This buffer
was limited to the size of 4096 bytes.

When a larger input was added via RAND_add() or RAND_seed() to the DRBG,
the reseeding failed, but the error returned by the DRBG was ignored
by the two calling functions, which both don't return an error code.
As a consequence, the data provided by the application was effectively
ignored.

This commit fixes the problem by a more efficient implementation which
does not copy the data in memory and by raising the buffer the size limit
to INT32_MAX (2 gigabytes). This is less than the NIST limit of 2^35 bits
but it was chosen intentionally to avoid platform dependent problems
like integer sizes and/or signed/unsigned conversion.

Additionally, the DRBG is now less permissive on errors: In addition to
pushing a message to the openssl error stack, it enters the error state,
which forces a reinstantiation on next call.

Thanks go to Dr. Falko Strenzke for reporting this issue to the
openssl-security mailing list. After internal discussion the issue
has been categorized as not being security relevant, because the DRBG
reseeds automatically and is fully functional even without additional
randomness provided by the application.

Fixes #7381

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7382)

(cherry picked from commit 3064b55134434a0b2850f07eff57120f35bb269a)

Fix no-engine

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/7365)

(cherry picked from commit 5f9f67b9d494e26941aa8d66d28a6a993c557643)

Indentation fixes.

The PR #7329 left some indentation slightly off.  This fixes it.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/7360)

(cherry picked from commit 5b639d4cb3b9a33536e2ebadf6a03149ea26ba32)

Remove useless check.
Hash can be longer than EC group degree and it will be truncated.

CLA: trivial

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7329)

(cherry picked from commit b770a80f6d0b08659cd03f26388b45512adb84f3)

Fix no-psk

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7306)

(cherry picked from commit 734af93a278a7a06710167219e1f05e525c9dd49)

Fix no-tls1_2

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7308)

(cherry picked from commit 7f1d923aa9dc55dd23a7741e4341ec421c683941)

Safer memory cleanup in (crypto/rsa/rsa_lib.c)

We don't need to use secure clean for public key.

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/7363)

(cherry picked from commit c033101db39c93cf41d80dfee4357f2617ede1a0)

Fix copy&paste error found in Coverity scan

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7377)

(cherry picked from commit 628ee796389b555ddb5fc28355e16e9417ab1724)

rsa/rsa_ossl.c: fix and extend commentary [skip ci].

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/7123)

(cherry picked from commit d1c008f66bad435b18aa45aa59f72bed7c682849)

ssl/s3_enc.c: fix logical errors in ssl3_final_finish_mac.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7085)

(cherry picked from commit 7d0effeacbb50b12bfc24df7614d7cf5c8686f51)

sha/asm/keccak1600-s390x.pl: resolve -march=z900 portability issue.

Negative displacement in memory references was not originally specified,
so that for maximum coverage one should abstain from it, just like with
any other extension. [Unless it's guarded by run-time switch, but there
is no switch in keccak1600-s390x.]

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7239)

(cherry picked from commit fc97c882f443060dffd8eb56a6b8784e52096c86)

Fix a typo in a macro

Fixes #7385

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7385)

(cherry picked from commit 990fe909949a58398b3a0cbbdc52b9bbb9cefaa0)

Fix compiling warnings in example code

The example code in EVP_DigestInit.pod generates warnings if users try
to compile it.

[skip ci]

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7362)

(cherry picked from commit 19ac1bf2de07214ee7ee6d2e118fa3aa8e5850f3)

crypto/rand: fix some style nit's

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7378)

(cherry picked from commit c2e33a05b1eb9dda988aebdeaa529973d7c22142)

rand_unix.c: fix --with-rand-seed=none build

Fixes a compiler warning about an unused syscall_random()
and cleans up the OPENSSL_RAND_SEED preprocessor logic.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/779)

(cherry picked from commit d90e128be6e584d319931c78cb8f8f68d17b6a34)

Fix a nit of copyright date range

Should be 2018 instead of 20018.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7364)

apps: allow empty attribute values with -subj

Historically (i.e., OpenSSL 1.0.x), the openssl applications would
allow for empty subject attributes to be passed via the -subj argument,
e.g., `opensl req -subj '/CN=joe/O=/OU=local' ...`.  Commit
db4c08f0194d58c6192f0d8311bf3f20e251cf4f applied a badly needed rewrite
to the parse_name() helper function that parses these strings, but
in the process dropped a check that would skip attributes with no
associated value.  As a result, such strings are now treated as
hard errors and the operation fails.

Restore the check to skip empty attribute values and restore
the historical behavior.

Document the behavior for empty subject attribute values in the
corresponding applications' manual pages.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7349)

(cherry picked from commit 3d362f190306b62a17aa2fd475b2bc8b3faa8142)

Cleanup typos and grammar in DES_random_key.pod

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7356)

(cherry picked from commit 4fef4981f8cc614559b86a06532b0eeac6ffd0d9)

/dev/crypto engine: give CIOCFSESSION the actual sess-id

We passed that ioctl a pointer to the whole session_op structure,
which wasn't quite right.

Notified by David Legault.

Fixes #7302

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/7304)

(cherry picked from commit 470096e576941566fd094f2fd793dc2948804ea8)

test/secmemtest: test secure memory only if it is implemented

Fixes #7322

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7351)

(cherry picked from commit 8529b156421805415bc7b17758255394de580c61)

Extend the BIO callback tests to check the return value semantics

Check that different return values passed to the BIO callback are
correctly handled.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7344)

(cherry picked from commit 52d78cc5ebc1d4fc021cabbcb09f4efb4c6ae82d)

Fix the BIO callback return code handling

The BIO callback handling incorrectly wrote over the return code passed
to the callback, meaning that an incorrect result was (eventually) returned
to the caller.

Fixes #7343

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7344)

(cherry picked from commit d97ce8d9a0619c1d9d1222dc1b44dbebb58dd966)

Clean out aliases in include/openssl/symhacks.h

Only a few clashing ones remain

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/7331)

(cherry picked from commit b44882a0bd0717e0aab84f5dc3ef81ab673155e9)

Small cleanup (util/mkdef.pl, crypto/bio/bss_log.c, include/openssl/ocsp.h)

BIO_s_log() is declared for everyone, so should return NULL when not
actually implemented.  Also, it had explicit platform limitations in
util/mkdef.pl that didn't correspond to what was actually in code.
While at it, a few other hard coded things that have lost their
relevance were removed.

include/openssl/ocsp.h had a few duplicate declarations.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/7331)

(cherry picked from commit 7e09c5eaa57295f87453286ffe25277c2f2bc73f)

Fix some Coverity warnings

Check some return values on some functions.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7335)

(cherry picked from commit 434893af2bd4c1fa72655f8e5262c8a432713968)

Fix a mem leak in the ocsp app

Free memory allocated in the parent process that is not needed in the
child. We also free it in the parent. Technically this isn't really
required since we end up calling exit() soon afterwards - but to
prevent false positives we free it anyway.

Fixes a Coverity issue.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7335)

(cherry picked from commit c20a76f695922f409c316399f7290530f7728f19)

Add missing cipher aliases to openssl(1)

And references to other manpages are also added in openssl(1).

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7314)

doc/man3/SSL_set_bio.pod: Fix wrong function name in return values section

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7035)

Update enc(1) examples to more recent ciphers and key derivation algorithms

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7248)

(cherry picked from commit eadde90bff01a6755399a4e1f6a3e4a9ed0fd61d)

Reduce stack usage in tls13_hkdf_expand

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7297)

(cherry picked from commit ec0c5f5693e39c5a013f81e6dd9dfd09ec65162d)

Document OPENSSL_VERSION_TEXT macro

This commit documents the OPENSSL_VERSION_TEXT which is currently
missing in the man page.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7301)

(cherry picked from commit 7c69495712e3dc9aa8db38271f0c3faeb2037165)

Use secure_getenv(3) when available.

Change all calls to getenv() inside libcrypto to use a new wrapper function
that use secure_getenv() if available and an issetugid then getenv if not.

CPU processor override flags are unchanged.

Extra checks for OPENSSL_issetugid() have been removed in favour of the
safe getenv.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7047)

(cherry picked from commit 5c39a55d04ea6e6f734b627a050b9e702788d50d)

Create the .rnd file it it does not exist

It's a bit annoying, since some commands try to read a .rnd file,
and print an error message if the file does not exist.

But previously a .rnd file was created on exit, and that does no longer
happen.

Fixed by continuing in app_RAND_load_conf regardless of the error in
RAND_load_file.

If the random number generator is still not initalized on exit, the
function RAND_write_file will fail and no .rnd file would be created.

Remove RANDFILE from openssl.cnf

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/7217)

(cherry picked from commit 0f58220973a02248ca5c69db59e615378467b9c8)

typo-fixes: miscellaneous typo fixes

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7277)

(cherry picked from commit 46d085096c6ead624c61e4b8b301421301511e64)

Fix the max psk len for TLSv1.3

If using an old style TLSv1.2 PSK callback then the maximum possible PSK
len is PSK_MAX_PSK_LEN (256) - not 64.

Fixes #7261

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7267)

(cherry picked from commit f39a02c68abc8936db24499cb3cfcba206a2e7eb)

Add a test for the certificate callback

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/7257)

(cherry picked from commit cd6fe29f5bad1a350a039673e06f83ec7a7ef619)

Delay setting the sig algs until after the cert_cb has been called

Otherwise the sig algs are reset if SSL_set_SSL_CTX() gets called.

Fixes #7244

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/7257)

(cherry picked from commit 524006dd1b80c1a86a20119ad988666a80d8d8f5)

crypto/bn/asm/x86_64-gcc.c: remove unnecessary redefinition of BN_ULONG

This module includes bn.h via other headers, so it picks up the
definition from there and doesn't need to define them locally (any
more?).  Worst case scenario, the redefinition may be different and
cause all sorts of compile errors.

Fixes #7227

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7287)

(cherry picked from commit dda5396aaec315bdbcb080e42fb5cd0191f2ad72)

/dev/crypto engine: add missing RC4 parameter

Fixes #7280

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7281)

(cherry picked from commit f52f2c1ae8f2ec378c5a20fb4104aa38edbabfcb)

Add some missing ciphers in 'enc' document

The original issue is #7273 and this commit fixes part of that issue.

[skip ci]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7275)

util/mkdef.pl, util/add-depends.pl: don't lowercase file names

It turns out to be detrimental on some file systems that may or may not
be case sensitive (such as NTFS, which has a case sensitive mode).

Fixes #7172

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7172)

(cherry picked from commit d3c72e392ab72d418ea5147857701a8730def3fd)

crypto/ui/ui_openssl.c: make sure to recognise ENXIO and EIO too

These both indicate that the file descriptor we're trying to use as a
terminal isn't, in fact, a terminal.

Fixes #7271

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7272)

(cherry picked from commit 276bf8620ce35a613c856f2b70348f65ffe94067)

Reset TLS 1.3 ciphers in SSL_CTX_set_ssl_version()

Historically SSL_CTX_set_ssl_version() has reset the cipher list
to the default.  Splitting TLS 1.3 ciphers to be tracked separately
caused a behavior change, in that TLS 1.3 cipher configuration was
preserved across calls to SSL_CTX_set_ssl_version().  To restore commensurate
behavior with the historical behavior, set the ciphersuites to the default as
well as setting the cipher list to the default.

Closes: #7226

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7270)

(cherry picked from commit 2340ed277b7c5365e83a32eb7d5fa32c4071fb21)

ssl/ssl_ciph.c: make set_ciphersuites static

Fixes #7252

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7253)

(cherry picked from commit f9a22815f386dbe7a13822f0ac3629ae8521cd76)

Trivial test improvements

This commit reuses a variable instead of reevaluating the expression
and updates an outdated comment in the EVP test.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7242)

(cherry picked from commit 523fcfb4c081ec346f117fd493103ddcd521e431)

Fixed typos in hkdf documentation.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7236)

(cherry picked from commit d474100af0827d9ba87f3bb25a34867244552df5)