[test] modernize ecdsatest and extend ECDSA sign KATs

(cherry picked from commit 1a31d8017ee7e8df0eca76fee601b826699c9ac1)

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8314)

Fix trivial typo in EVP_DigestVerifyInit doc

(cherry picked from commit b3883f77df33989b0d4298ca9a21d8595dd9a8c9)

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8319)

Prepare for 1.1.1c-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.1b release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Clarify that SSL_shutdown() must not be called after a fatal error

Follow on from CVE-2019-1559

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update copyright year

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8347)

e_devcrypto: set digest input_blocksize

This restores the behavior of previous versions of the /dev/crypto
engine, in alignment with the default implementation.

Reported-by: Gerard Looije <lglooije@hotmail.com>
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8306)

eng_devcrypto: close open session on init

cipher_init may be called on an already initialized context, without a
necessary cleanup.  This separates cleanup from initialization, closing
an eventual open session before creating a new one.

Move the /dev/crypto session cleanup code to its own function.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8306)

Update NEWS for new release

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8344)

Disable 02-test_errstr.t on msys/mingw as well as MSWin32

There is too high a risk that perl and OpenSSL are linked with
different C RTLs, and thereby get different messages for even the most
mundane error numbers.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8343)

(cherry picked from commit 565a19eef35926b4b9675f6cc3964fb290a5b380)

VMS: disable the shlibload test for now

test/shlibloadtest.c needs added code for VMS shared libraries

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8342)

Rearrange the inclusion of curve448/curve448_lcl.h

The real cause for this change is that test/ec_internal_test.c
includes ec_lcl.h, and including curve448/curve448_lcl.h from there
doesn't work so well with compilers who always do inclusions relative
to the C file being compiled.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8334)

Ensure bn_cmp_words can handle the case where n == 0

Thanks to David Benjamin who reported this, performed the analysis and
suggested the patch. I have incorporated some of his analysis in the
comments below.

This issue can cause an out-of-bounds read. It is believed that this was
not reachable until the recent "fixed top" changes. Analysis has so far
only identified one code path that can encounter this - although it is
possible that others may be found. The one code path only impacts 1.0.2 in
certain builds. The fuzzer found a path in RSA where iqmp is too large. If
the input is all zeros, the RSA CRT logic will multiply a padded zero by
iqmp. Two mitigating factors:

- Private keys which trip this are invalid (iqmp is not reduced mod p).
Only systems which take untrusted private keys care.
- In OpenSSL 1.1.x, there is a check which rejects the oversize iqmp,
so the bug is only reproducible in 1.0.2 so far.

Fortunately, the bug appears to be relatively harmless. The consequences of
bn_cmp_word's misbehavior are:

- OpenSSL may crash if the buffers are page-aligned and the previous page is
non-existent.
- OpenSSL will incorrectly treat two BN_ULONG buffers as not equal when they
are equal.
- Side channel concerns.

The first is indeed a concern and is a DoS bug. The second is fine in this
context. bn_cmp_word and bn_cmp_part_words are used to compute abs(a0 - a1)
in Karatsuba. If a0 = a1, it does not matter whether we use a0 - a1 or
a1 - a0. The third would be worth thinking about, but it is overshadowed
by the entire Karatsuba implementation not being constant time.

Due to the difficulty of tripping this and the low impact no CVE is felt
necessary for this issue.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8326)

(cherry picked from commit 576129cd72ae054d246221f111aabf42b9c6d76d)

Windows: Call TerminateProcess, not ExitProcess

Ty Baen-Price explains:

> Problem and Resolution:
> The following lines of code make use of the Microsoft API ExitProcess:
>
> ```
> Apps\Speed.c line 335: ExitProcess(ret);
> Ms\uplink.c line 22: ExitProcess(1);
> ```
>
> These function calls are made after fatal errors are detected and
> program termination is desired. ExitProcess(), however causes
> _orderly_ shutdown of a process and all its threads, i.e. it unloads
> all dlls and runs all destructors. See MSDN for details of exactly
> what happens
> (https://msdn.microsoft.com/en-us/library/windows/desktop/ms682658(v=vs.85).aspx).
> The MSDN page states that ExitProcess should never be called unless
> it is _known to be safe_ to call it. These calls should simply be
> replaced with calls to TerminateProcess(), which is what should be
> called for _disorderly_ shutdown.
>
> An example of usage:
>
> ```
> TerminateProcess(GetCurrentProcess(), exitcode);
> ```
>
> Effect of Problem:
> Because of a compilation error (wrong c++ runtime), my program
> executed the uplink.c ExitProcess() call. This caused the single
> OpenSSL thread to start executing the destructors of all my dlls,
> and their objects. Unfortunately, about 30 other threads were
> happily using those objects at that time, eventually causing a
> 0xC0000005 ACCESS_VIOLATION. Obviously an ACCESS_VIOLATION is the
> best case scenario, as I'm sure you can imagine at the consequences
> of undiscovered memory corruption, even in a terminating process.

And on the subject of `TerminateProcess()` being asynchronous:

> That is technically true, but I think it's probably synchronous
> "enough" for your purposes, since a call to TerminateProcess
> suspends execution of all threads in the target process. This means
> it's really only asynchronous if you're calling TerminateProcess one
> some _other_ process. If you're calling TerminateProcess on your own
> process, you'll never return from the TerminateProcess call.

Fixes #2489
Was originally RT-4526

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8301)

(cherry picked from commit 925795995018bddb053e863db8b5c52d2a9005d9)

Don't restrict the number of KeyUpdate messages we can process

Prior to this commit we were keeping a count of how many KeyUpdates we
have processed and failing if we had had too many. This simplistic approach
is not sufficient for long running connections. Since many KeyUpdates
would not be a particular good DoS route anyway, the simplest solution is
to simply remove the key update count.

Fixes #8068

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8299)

(cherry picked from commit 3409a5ff8a44ddaf043d83ed22e657ae871be289)

engines/dasync: add explaining comments about AES-128-CBC-HMAC-SHA1

Fixes #7950

It was reported that there might be a null pointer dereference in the
implementation of the dasync_aes_128_cbc_hmac_sha1() cipher, because
EVP_aes_128_cbc_hmac_sha1() can return a null pointer if AES-NI is
not available. It took some analysis to find out that this is not
an issue in practice, and these comments explain the reason to comfort
further NPD hunters.

Detected by GitHub user @wurongxin1987 using the Sourcebrella Pinpoint
static analyzer.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8305)

(cherry picked from commit a4a0a1eb43cfccd128d085932a567e0482fbfe47)

Fix a grammar nit in CRYPTO_get_ex_new_index.pod

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8303)

(cherry picked from commit 84712024da5e5485e8397afc763555355bddf960)

Fix dasync engine

The aes128_cbc_hmac_sha1 cipher in the dasync engine is broken. Probably
by commit e38c2e8535 which removed use of the "enc" variable...but not
completely.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8291)

(cherry picked from commit 695dd3a332fdd54b873fd0d08f9ae720141f24cd)

SSL_CONF_cmd: fix doc for NoRenegotiation

The option is a flag for Options, not a standalone setting.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8292)

(cherry picked from commit 4ac5e43da6d9ee828240e6d347c48c8fae6573a2)

Clear BN_FLG_CONSTTIME on BN_CTX_get()

(cherry picked from commit c8147d37ccaaf28c430d3fb45a14af36597e48b8)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8253)

Test for constant-time flag leakage in BN_CTX

This commit adds a simple unit test to make sure that the constant-time
flag does not "leak" among BN_CTX frames:

- test_ctx_consttime_flag() initializes (and later frees before
  returning) a BN_CTX object, then it calls in sequence
  test_ctx_set_ct_flag() and test_ctx_check_ct_flag() using the same
  BN_CTX object. The process is run twice, once with a "normal"
  BN_CTX_new() object, then with a BN_CTX_secure_new() one.
- test_ctx_set_ct_flag() starts a frame in the given BN_CTX and sets the
  BN_FLG_CONSTTIME flag on some of the BIGNUMs obtained from the frame
  before ending it.
- test_ctx_check_ct_flag() then starts a new frame and gets a number of
  BIGNUMs from it. In absence of leaks, none of the BIGNUMs in the new
  frame should have BN_FLG_CONSTTIME set.

In actual BN_CTX usage inside libcrypto the leak could happen at any
depth level in the BN_CTX stack, with varying results depending on the
patterns of sibling trees of nested function calls sharing the same
BN_CTX object, and the effect of unintended BN_FLG_CONSTTIME on the
called BN_* functions.

This simple unit test abstracts away this complexity and verifies that
the leak does not happen between two sibling functions sharing the same
BN_CTX object at the same level of nesting.

(cherry picked from commit fe16ae5f95fa86ddb049a8d1e2caee0b80b32282)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8253)

[test] unit test for field_inv function pointer in EC_METHOD

(cherry picked from commit 8f58ede09572dcc6a7e6c01280dd348240199568)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8262)

SCA hardening for mod. field inversion in EC_GROUP

This commit adds a dedicated function in `EC_METHOD` to access a modular
field inversion implementation suitable for the specifics of the
implemented curve, featuring SCA countermeasures.

The new pointer is defined as:
`int (*field_inv)(const EC_GROUP*, BIGNUM *r, const BIGNUM *a, BN_CTX*)`
and computes the multiplicative inverse of `a` in the underlying field,
storing the result in `r`.

Three implementations are included, each including specific SCA
countermeasures:
  - `ec_GFp_simple_field_inv()`, featuring SCA hardening through
    blinding.
  - `ec_GFp_mont_field_inv()`, featuring SCA hardening through Fermat's
    Little Theorem (FLT) inversion.
  - `ec_GF2m_simple_field_inv()`, that uses `BN_GF2m_mod_inv()` which
    already features SCA hardening through blinding.

From a security point of view, this also helps addressing a leakage
previously affecting conversions from projective to affine coordinates.

This commit also adds a new error reason code (i.e.,
`EC_R_CANNOT_INVERT`) to improve consistency between the three
implementations as all of them could fail for the same reason but
through different code paths resulting in inconsistent error stack
states.

Co-authored-by: Nicola Tuveri <nic.tuv@gmail.com>
(cherry picked from commit e0033efc30b0f00476bba8f0fa5512be5dc8a3f1)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8262)

Don't set SNI by default if hostname is not dNS name

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8175)

(cherry picked from commit 8e981051ceecd10754f8f6d1291414a7453c8fac)

Fix reference to symbol 'main'.

The AIX binder needs to be instructed that the output will have no entry
point (see AIX' ld manual: -e in the Flags section; autoexp and noentry
in the Binder section).

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8282)

(cherry picked from commit c1b3846242fc1a7791beca42f548c325c35e269b)

Add a test for interleaving app data with handshake data in TLSv1.3

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/8191)

(cherry picked from commit 73e62d40eb53f2bad98dea0083c217dbfad1a335)

Don't interleave handshake and other record types in TLSv1.3

In TLSv1.3 it is illegal to interleave handshake records with non handshake
records.

Fixes #8189

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/8191)

(cherry picked from commit 3d35e3a253a2895f263333bb4355760630a31955)

cygwin: drop explicit O_TEXT

Cygwin binaries should not enforce text mode these days, just
use text mode if the underlying mount point requests it

Signed-off-by: Corinna Vinschen <vinschen@redhat.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8248)

(cherry picked from commit 9b57e4a1ef356420367d843f1ba96037f88316b8)

Add missing dots in dgst man page

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
GH: #8142
(cherry picked from commit e3ac3654892246d7492f1012897e42ad7efd13ce)

Fixed typo

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
GH: #8121
(cherry picked from commit 70680262329004c934497040bfc6940072043f48)

Check for unpaired .cfi_remember_state

Reviewed-by: Richard Levitte <levitte@openssl.org>
GH: #8109
(cherry picked from commit e09633107b7e987b2179850715ba60d8fb069278)

Fix some CFI issues in x86_64 assembly

The add/double shortcut in ecp_nistz256-x86_64.pl left one instruction
point that did not unwind, and the "slow" path in AES_cbc_encrypt was
not annotated correctly. For the latter, add
.cfi_{remember,restore}_state support to perlasm.

Next, fill in a bunch of functions that are missing no-op .cfi_startproc
and .cfi_endproc blocks. libunwind cannot unwind those stack frames
otherwise.

Finally, work around a bug in libunwind by not encoding rflags. (rflags
isn't a callee-saved register, so there's not much need to annotate it
anyway.)

These were found as part of ABI testing work in BoringSSL.

Reviewed-by: Richard Levitte <levitte@openssl.org>
GH: #8109
(cherry picked from commit c0e8e5007ba5234d4d448e82a1567e0c4467e629)

Mark generated functions unused (applies to safestack, lhash, sparse_array)

safestack.h, lhash.h and sparse_array.h all define macros to generate
a full API for the containers as static inline functions.  This
potentially generates unused code, which some compilers may complain
about.

We therefore need to mark those generated functions as unused, so the
compiler knows that we know, and stops complaining about it.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8246)

(cherry picked from commit 48fe4ce104df060dd5d2b4188a56eb554d94d819)

Use order not degree to calculate a buffer size in ecdsatest

Otherwise this can result in an incorrect calculation of the maximum
encoded integer length, meaning an insufficient buffer size is allocated.

Thanks to Billy Brumley for helping to track this down.

Fixes #8209

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8237)

(cherry picked from commit 9fc8f18f59f4a4c853466dca64a23b8af681bf1c)

Fix -verify_return_error in s_client

The "verify_return_error" option in s_client is documented as:

 Return verification errors instead of continuing. This will typically
 abort the handshake with a fatal error.

In practice this option was ignored unless also accompanied with the
"-verify" option. It's unclear what the original intention was. One fix
could have been to change the documentation to match the actual behaviour.
However it seems unecessarily complex and unexpected that you should need
to have both options. Instead the fix implemented here is make the option
match the documentation so that "-verify" is not also required.

Note that s_server has a similar option where "-verify" (or "-Verify") is
still required. This makes more sense because those options additionally
request a certificate from the client. Without a certificate there is no
possibility of a verification failing, and so "-verify_return_error" doing
nothing seems ok.

Fixes #8079

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8080)

(cherry picked from commit 78021171dbcb05ddab1b5daffbfc62504ea709a4)

Don't signal SSL_CB_HANDSHAKE_START for TLSv1.3 post-handshake messages

The original 1.1.1 design was to use SSL_CB_HANDSHAKE_START and
SSL_CB_HANDSHAKE_DONE to signal start/end of a post-handshake message
exchange in TLSv1.3. Unfortunately experience has shown that this confuses
some applications who mistake it for a TLSv1.2 renegotiation. This means
that KeyUpdate messages are not handled properly.

This commit removes the use of SSL_CB_HANDSHAKE_START and
SSL_CB_HANDSHAKE_DONE to signal the start/end of a post-handshake
message exchange. Individual post-handshake messages are still signalled in
the normal way.

This is a potentially breaking change if there are any applications already
written that expect to see these TLSv1.3 events. However, without it,
KeyUpdate is not currently usable for many applications.

Fixes #8069

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8096)

(cherry picked from commit 4af5836b55442f31795eff6c8c81ea7a1b8cf94b)

Ignore cipher suites when setting cipher list

set_cipher_list() sets TLSv1.2 (and below) ciphers, and its success or
failure should not depend on whether set_ciphersuites() has been used to
setup TLSv1.3 ciphers.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7759)

(cherry picked from commit 3c83c5ba4f6502c708b7a5f55c98a10e312668da)

Configure: stop forcing use of DEFINE macros in headers

There are times when one might want to use something like
DEFINE_STACK_OF in a .c file, because it defines a stack for a type
defined in that .c file.  Unfortunately, when configuring with
`--strict-warnings`, clang aggressively warn about unused functions in
such cases, which forces the use of such DEFINE macros to header
files.

We therefore disable this warning from the `--strict-warnings`
definition for clang.

(note for the curious: `-Wunused-function` is enabled via `-Wall`)

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8234)

(cherry picked from commit f11ffa505f8a9345145a26a05bf77b012b6941bd)

Windows/Cygwin dlls need the executable bit set

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8226)

(cherry picked from commit fa63e45262971b9c2a6aeb33db8c52a5a84fc8b5)

Fix null pointer dereference in cms_RecipientInfo_kari_init

CLA: trivial

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8137)

(cherry picked from commit b754a8a1590b8c5c9662c8a0ba49573991488b20)

AArch64 assembly pack: authenticate return addresses.

ARMv8.3 adds pointer authentication extension, which in this case allows
to ensure that, when offloaded to stack, return address is same at return
as at entry to the subroutine. The new instructions are nops on processors
that don't implement the extension, so that the vetification is backward
compatible.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8205)

(cherry picked from commit 9a18aae5f21efc59da8b697ad67d5d37b95ab322)

apps/ocsp.c Use the same HAVE_FORK / NO_FORK as in speed.c

This allows the user to override our defaults if needed, and in a
consistent manner.

Partial fix for #7607

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7624)

(cherry picked from commit ca811248d838058c13236a6c3b688e0ac98c02c8)

test/recipes/02-err_errstr: skip errors that may not be loaded on Windows

Fixes #8091

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8094)

Allow the syntax of the .include directive to optionally have '='

If the old openssl versions not supporting the .include directive
load a config file with it, they will bail out with error.

This change allows using the .include = <filename> syntax which
is interpreted as variable assignment by the old openssl
config file parser.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8141)

(cherry picked from commit 9d5560331d86c6463e965321f774e4eed582ce0b)

Fix null pointer dereference in ssl_module_init

CLA: Trivial

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8183)

(cherry picked from commit 758229f7d22775d7547e3b3b886b7f6a289c6897)

Update d2i_PrivateKey documentation

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8168)

(cherry picked from commit 1980ce45d6bdd2b57df7003d6b56b5df560b9064)

Fix d2i_PublicKey() for EC keys

o2i_ECPublicKey() requires an EC_KEY structure filled with an EC_GROUP.

o2i_ECPublicKey() is called by d2i_PublicKey(). In order to fulfill the
o2i_ECPublicKey()'s requirement, d2i_PublicKey() needs to be called with
an EVP_PKEY with an EC_KEY containing an EC_GROUP.

However, the call to EVP_PKEY_set_type() frees any existing key structure
inside the EVP_PKEY, thus freeing the EC_KEY with the EC_GROUP that
o2i_ECPublicKey() needs.

This means you can't d2i_PublicKey() for an EC key...

The fix is to check to see if the type is already set appropriately, and
if so, not call EVP_PKEY_set_type().

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8168)

(cherry picked from commit 2aa2beb06cc25c1f8accdc3d87b946205becfd86)

Address a bug in the DRBG tests where the reseeding wasn't properly
reinstantiating the DRBG.

Bug reported by Doug Gibbons.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/8184)

(cherry picked from commit b1522fa5ef676b7af0128eab3eee608af3416182)

test/drbgtest.c: call OPENSSL_thread_stop() explicitly

The manual says this in its notes:

    ... and therefore applications using static linking should also call
    OPENSSL_thread_stop() on each thread. ...

Fixes #8171

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8173)

(cherry picked from commit 03cdfe1efaf2a3b5192b8cb3ef331939af7bfeb8)

Make OPENSSL_malloc_init() a no-op

Making this a no-op removes a potential infinite loop than can occur in
some situations.

Fixes #2865

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8167)

(cherry picked from commit ef45aa14c5af024fcb8bef1c9007f3d1c115bd85)

Remove unnecessary trailing whitespace

Trim trailing whitespace. It doesn't match OpenSSL coding standards,
AFAICT, and it can cause problems with git tooling.

Trailing whitespace remains in test data and external source.

Backport-of: https://github.com/openssl/openssl/pull/8092

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8134)

Make some simple getters take const SSL/SSL_CTX

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8145)

(cherry picked from commit 3499327bad401eb510d76266428923d06c9c7bb7)

Fix Invalid Argument return code from IP_Factory in connect_to_server().

Fixes #7732

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8158)

(cherry picked from commit 66a60003719240399f6596e58c239df0465a4f70)

Android build: fix usage of NDK home variable ($ndk_var)

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8153)

(cherry picked from commit adc7e221f12462c6e10bc7c2c7afaf52490cb292)

Add an entry to the CHANGES for the d2i_X509_PUBKEY fix

The commit 5dc40a83c74be579575a512b30d9c1e0364e6a7b forgot
to add a short description to the CHANGES file.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8144)

(cherry picked from commit b2aea0e3d9a15e30ebce8b6da213df4a3f346155)

Fix end-point shared secret for DTLS/SCTP

When computing the end-point shared secret, don't take the
terminating NULL character into account.
Please note that this fix breaks interoperability with older
versions of OpenSSL, which are not fixed.

Fixes #7956

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7957)

(cherry picked from commit 09d62b336d9e2a11b330d45d4f0f3f37cbb0d674)

Fix a crash in reuse of i2d_X509_PUBKEY

If the second PUBKEY is malformed there is use after free.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8122)

(cherry picked from commit 5dc40a83c74be579575a512b30d9c1e0364e6a7b)

Fixed d2i_X509 in-place not re-hashing the ex_flags

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8116)

(cherry picked from commit 53649022509129bce8036c8fb4978dbce9432a86)

Fix a memory leak with di2_X509_CRL reuse

Additionally avoid undefined behavior with
in-place memcpy in X509_CRL_digest.

Fixes #8099

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8112)

(cherry picked from commit a727627922b8a9ec6628ffaa2054b4b3833d674b)

Better phrasing around 1.1.0

Fixes #8129

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8130)

(cherry picked from commit 62b563b9df161a992fde18a0cb0d1a0969158412)

VMS: force 'pinshared'

VMS doesn't currently support unloading of shared object, and we need
to reflect that.  Without this, the shlibload test fails

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8131)

(cherry picked from commit d1dd5d6f4c2f13478aa45557b4546febd51f0cb3)

Fix error message for s_server -psk option

Previously if -psk was given a bad key it would print "Not a hex
number 's_server'".

CLA: Trivial

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/8113)

(cherry picked from commit e57120128fa4e2afa4bda5022a77f73a1e3a0b27)

Reuse already defined macros

instead of duplicity the code.

CLA: trivial

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8127)

(cherry picked from commit c4734493d7da404b1747195a805c8d536dbe6910)

Complain if -twopass is used incorrectly

The option -twopass to the pkcs12 app is ignored if -passin, -passout
or -password is used. We should complain if an attempt is made to use
it in combination with those options.

Fixes #8107

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8114)

(cherry picked from commit 40b64553f577716cb4898895f5fd4530a6266c75)

Fix no-dso builds

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8111)

(cherry picked from commit 522b11e969cbdc82eca369512275f227080a86fa)

Don't leak memory from ERR_add_error_vdata()

If the call the ERR_set_error_data() in ERR_add_error_vdata() fails then
a mem leak can occur. This commit checks that we successfully added the
error data, and if not frees the buffer.

Fixes #8085

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/8105)

(cherry picked from commit fa6b1ee1115c1e5e3a8286d833dcbaa2c1ce2b77)

Android build: use ANDROID_NDK_HOME rather than ANDROID_NDK

It apepars that ANDROID_NDK_HOME is the recommended standard
environment variable for the NDK.

We retain ANDROID_NDK as a fallback.

Fixes #8101

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8103)

(cherry picked from commit 6e826c471b7f0431391a4e9f9484f6ea2833774a)

clarify which functions are the CMS functions which must have CMS_PARTIAL set

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7960)

(cherry picked from commit 61e033308b1c004bd808352fb1d786547dcdf62b)

crypto/bn: fix return value in BN_generate_prime

When the ret parameter is NULL the generated prime
is in rnd variable and not in ret.

CLA: trivial

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8076)

(cherry picked from commit 3d43f9c809e42b960be94f2f4490d6d14e063486)

s_client: fix not to send a command letter of R

Before 1.1.0, this command letter is not sent to a server.

CLA: trivial
(cherry picked from commit bc180cb4887c2e82111cb714723a94de9f6d2c35)

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8081)

(cherry picked from commit 5478e2100260b8d6f9df77de875f37763d8eeec6)

Remove stray -modulus option from the ec manual page.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8082)

(cherry picked from commit d7bcbfd0828616f33008e711eabc6ec00b32e87b)

Add "weak" declarations of symbols used in safestack.h and lhash.h

Only for SunCC for now.

It turns out that some compilers to generate external variants of
unused static inline functions, and if they use other external
symbols, those need to be present as well.  If you then happen to
include one of safestack.h or lhash.h without linking with libcrypto,
the build fails.

Fixes #6912

Signed-off-by: Matthias Kraft <Matthias.Kraft@softwareag.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8087)

(cherry picked from commit 6638b2214761b5f30300534e0fe522448113c6cf)

X509_STORE: fix two misspelled compatibility macros

Fixes #8084

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8086)

(cherry picked from commit 2c75f03b39de2fa7d006bc0f0d7c58235a54d9bb)

Cleanup vxworks support to be able to compile for VxWorks 7

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7569)

(cherry picked from commit 5c8b7b4caa0faedb69277063a7c6b3a8e56c6308)

Revert "Keep the DTLS timer running after the end of the handshake if appropriate"

This commit erroneously kept the DTLS timer running after the end of the
handshake. This is not correct behaviour and shold be reverted.

This reverts commit f7506416b1311e65d5c440defdbcfe176f633c50.

Fixes #7998

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8047)

(cherry picked from commit bcc1f3e2baa9caa83a0a94bd19fb37488ef3ee57)

Make sure we trigger retransmits in DTLS testing

During a DTLS handshake we may need to periodically handle timeouts in the
DTLS timer to ensure retransmits due to lost packets are performed. However,
one peer will always complete a handshake before the other. The DTLS timer
stops once the handshake has finished so any handshake messages lost after
that point will not automatically get retransmitted simply by calling
DTLSv1_handle_timeout(). However attempting an SSL_read implies a
DTLSv1_handle_timeout() and additionally will process records received from
the peer. If those records are themselves retransmits then we know that the
peer has not completed its handshake yet and a retransmit of our final
flight automatically occurs.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8047)

(cherry picked from commit 80c455d5ae405e855391e298a2bf8a24629dd95d)

Update NOTES.ANDROID

Minor typo fix to `adjustment` in the line:
"In such case you have to pass matching target
 name to Configure and shouldn't use -D__ANDROID_API__=N. PATH adjustment
 becomes simpler, $ANDROID_NDK/bin:$PATH suffices."

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8054)

(cherry picked from commit 52bcd4afc84d75f9d22866a3cefaf9ae4e9ff997)

Make ca command silently use default if .attr file does not exist

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7286)

(cherry picked from commit ac454d8d4663e2fcf8a8437fab8aefd883091c37)

PPC: Try out if mftb works before using it

If this fails try out if mfspr268 works.

Use OPENSSL_ppccap=0x20 for enabling mftb,
OPENSSL_ppccap=0x40 for enabling mfspr268,
and OPENSSL_ppccap=0 for enabling neither.

Fixes #8012

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8043)

(cherry picked from commit c8f370485c43729db44b680e41e875ddd7f3108c)

Fix a memory leak in the mem bio

If you use a BIO and set up your own buffer that is not freed, the
memory bio will leak the BIO_BUF_MEM object it allocates.

The trouble is that the BIO_BUF_MEM is allocated and kept around,
but it is not freed if BIO_NOCLOSE is set.

The freeing of BIO_BUF_MEM was fairly confusing, simplify things
so mem_buf_free only frees the memory buffer and free the BIO_BUF_MEM
in mem_free(), where it should be done.

Alse add a test for a leak in the memory bio
Setting a memory buffer caused a leak.

Signed-off-by: Corey Minyard <minyard@acm.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8051)

(cherry picked from commit c6048af23c577bcf85f15122dd03b65f959c9ecb)

Reduce inputs before the RSAZ code.

The RSAZ code requires the input be fully-reduced. To be consistent with the
other codepaths, move the BN_nnmod logic before the RSAZ check.

This fixes an oft-reported fuzzer bug.
https://github.com/google/oss-fuzz/issues/1761

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7187)

(cherry picked from commit 3afd537a3c2319f68280804004e9bf2e798a43f7)

apps/verify.c: Change an old comment to clarify what the callback does

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7922)

(cherry picked from commit 9b10986d7742a5105ac8c5f4eba8b103caf57ae9)

crypto/armcap.c, crypto/ppccap.c: stricter use of getauxval()

Having a weak getauxval() and only depending on GNU C without looking
at the library we build against meant that it got picked up where not
really expected.

So we change this to check for the glibc version, and since we know it
exists from that version, there's no real need to make it weak.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/8028)

(cherry picked from commit 5f40dd158cbfa0a3bd86c32f7a77fec8754bb245)

crypto/uid.c: use own macro as guard rather than AT_SECURE

It turns out that AT_SECURE may be defined through other means than
our inclusion of sys/auxv.h, so to be on the safe side, we define our
own guard and use that to determine if getauxval() should be used or
not.

Fixes #7932

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7933)

(cherry picked from commit aefb980c45134d84f1757de1a9c61d699c8a7e33)

Don't get the mac type in TLSv1.3

We don't use this information so we shouldn't fetch it. As noted in the
comments in #8005.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/8020)

(cherry picked from commit ea09abc80892920ee5db4de82bed7a193b5896f0)

Add missing entries in ssl_mac_pkey_id

Fixes #8005

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/8020)

(cherry picked from commit 7fe0ed75e3e7760226a0a3a5a86cf3887004f6e4)

Check more return values in the SRP code

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8019)

(cherry picked from commit d63bde7827b0be1172f823baf25309b54aa87e0f)

Check a return value in the SRP code

Spotted by OSTIF audit

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8019)

(cherry picked from commit 0a5bda639f8fd59e15051cf757708e3b94bcf399)

Don't artificially limit the size of the ClientHello

We were setting a limit of SSL3_RT_MAX_PLAIN_LENGTH on the size of the
ClientHello. AFAIK there is nothing in the standards that requires this
limit.

The limit goes all the way back to when support for extensions was first
added for TLSv1.0. It got converted into a WPACKET max size in 1.1.1. Most
likely it was originally added to avoid the complexity of having to grow
the init_buf in the middle of adding extensions. With WPACKET this is
irrelevant since it will grow automatically.

This issue came up when an attempt was made to send a very large
certificate_authorities extension in the ClientHello.

We should just remove the limit.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7424)

(cherry picked from commit 7835e97b6ff5cd94a10c5aeac439f4aa145a77b2)

Fix CID 1434549: Unchecked return value in test/evp_test.c

5. check_return: Calling EVP_EncodeUpdate without checking return value
(as is done elsewhere 4 out of 5 times).

Fix CID 1371695, 1371698: Resource leak in test/evp_test.c

- leaked_storage: Variable edata going out of scope leaks the storage it
points to.

- leaked_storage: Variable encode_ctx going out of scope leaks the
storage it points to

Fix CID 1430437, 1430426, 1430429 : Dereference before null check in test/drbg_cavs_test.c

check_after_deref: Null-checking drbg suggests that it
may be null, but it has already been dereferenced on all paths leading
to the check

Fix CID 1440765: Dereference before null check in test/ssltestlib.c

check_after_deref: Null-checking ctx suggests that it may be null, but
it has already been dereferenced on all paths leading to the check.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7993)

(cherry picked from commit 760e2d60e62511a6fb96f547f6730d05eb5f47ec)

More configurable crypto and ssl library initialization

1.  In addition to overriding the default application name,
    one can now also override the configuration file name
    and flags passed to CONF_modules_load_file().

2.  By default we still keep going when configuration file
    processing fails.  But, applications that want to be
    strict about initialization errors can now make explicit
    flag choices via non-null OPENSSL_INIT_SETTINGS that omit
    the CONF_MFLAGS_IGNORE_RETURN_CODES flag (which had so far
    been both undocumented and unused).

3.  In OPENSSL_init_ssl() do not request OPENSSL_INIT_LOAD_CONFIG
    if the options already include OPENSSL_INIT_NO_LOAD_CONFIG.

4.  Don't set up atexit() handlers when called with opts equal to
    OPENSSL_INIT_BASE_ONLY (this flag should only be used alone).

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7969)

Update generator copyright year.

Some Travis builds appear to fail because generated objects get
2019 copyrights now, and the diff complains.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7969)

Add a test for correct handling of the cryptopro bug extension

This was complicated by the fact that we were using this extension for our
duplicate extension handling tests. In order to add tests for cryptopro
bug the duplicate extension handling tests needed to change first.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7984)

(cherry picked from commit 9effc496ad8a9b0ec737c69cc0fddf610a045ea4)

Don't complain if we receive the cryptopro extension in the ClientHello

The cryptopro extension is supposed to be unsolicited and appears in the
ServerHello only. Additionally it is unofficial and unregistered - therefore
we should really treat it like any other unknown extension if we see it in
the ClientHello.

Fixes #7747

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7984)

(cherry picked from commit 23fed8ba0ec895e1b2a089cae380697f15170afc)

doc/man1/x509.pod: fix typo

This looks like a copy&paste error from req.pod to x509.pod.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7995)

(cherry picked from commit 67ee899cb51d3e3d7b5f00b878f8f82a097b93f0)

Restore compatibility with GOST2001 implementations.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7985)

(cherry picked from commit 673e0bbbe4b9cbd19a247c0b18c171bb0421915a)

Fix no-cmac

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7979)

(cherry picked from commit 87bbbfb1e4fc2035e8f9ec1d6313a41c410a3218)

Support _onexit() in preference to atexit() on Windows

This enables cleanup to happen on DLL unload instead of at process exit.

[extended tests]

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7983)

Introduce a no-pinshared option

This option prevents OpenSSL from pinning itself in memory.

Fixes #7598

[extended tests]

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7983)

Test atexit handlers

Test that atexit handlers get called properly at process exit, unless we
have explicitly asked for them not to be.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7983)