Add option to disable Extended Master Secret

Add SSL_OP64_NO_EXTENDED_MASTER_SECRET, that can be set on either
an SSL or an SSL_CTX. When processing a ClientHello, if this flag
is set, do not indicate that the EMS TLS extension was received in
either the ssl3 object or the SSL_SESSION.  Retain most of the
sanity checks between the previous and current session during
session resumption, but weaken the check when the current SSL
object is configured to not use EMS.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3910)

Use order not degree to calculate a buffer size in ecdsatest

Otherwise this can result in an incorrect calculation of the maximum
encoded integer length, meaning an insufficient buffer size is allocated.

Thanks to Billy Brumley for helping to track this down.

Fixes #8209

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8237)

Fix no-stdio

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8238)

Configure: make --strict-warnings a regular user provided compiler option

This makes `--strict-warnings` into a compiler pseudo-option, i.e. it
gets treated the same way as any other compiler option given on the
configuration command line, but is retroactively replaced by actual
compiler warning options, depending on what compiler is used.

This makes it easier to see in what order options are given to the
compiler from the configuration command line, i.e. this:

    ./config -Wall --strict-warnings

would give the compiler flags in the same order as they're given,
i.e.:

    -Wall -Werror -Wno-whatever ...

instead of what we got previously:

    -Werror -Wno-whatever ... -Wall

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8239)

Fix -verify_return_error in s_client

The "verify_return_error" option in s_client is documented as:

 Return verification errors instead of continuing. This will typically
 abort the handshake with a fatal error.

In practice this option was ignored unless also accompanied with the
"-verify" option. It's unclear what the original intention was. One fix
could have been to change the documentation to match the actual behaviour.
However it seems unecessarily complex and unexpected that you should need
to have both options. Instead the fix implemented here is make the option
match the documentation so that "-verify" is not also required.

Note that s_server has a similar option where "-verify" (or "-Verify") is
still required. This makes more sense because those options additionally
request a certificate from the client. Without a certificate there is no
possibility of a verification failing, and so "-verify_return_error" doing
nothing seems ok.

Fixes #8079

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8080)

Don't signal SSL_CB_HANDSHAKE_START for TLSv1.3 post-handshake messages

The original 1.1.1 design was to use SSL_CB_HANDSHAKE_START and
SSL_CB_HANDSHAKE_DONE to signal start/end of a post-handshake message
exchange in TLSv1.3. Unfortunately experience has shown that this confuses
some applications who mistake it for a TLSv1.2 renegotiation. This means
that KeyUpdate messages are not handled properly.

This commit removes the use of SSL_CB_HANDSHAKE_START and
SSL_CB_HANDSHAKE_DONE to signal the start/end of a post-handshake
message exchange. Individual post-handshake messages are still signalled in
the normal way.

This is a potentially breaking change if there are any applications already
written that expect to see these TLSv1.3 events. However, without it,
KeyUpdate is not currently usable for many applications.

Fixes #8069

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8096)

Ignore cipher suites when setting cipher list

set_cipher_list() sets TLSv1.2 (and below) ciphers, and its success or
failure should not depend on whether set_ciphersuites() has been used to
setup TLSv1.3 ciphers.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7759)

Configure: stop forcing use of DEFINE macros in headers

There are times when one might want to use something like
DEFINE_STACK_OF in a .c file, because it defines a stack for a type
defined in that .c file.  Unfortunately, when configuring with
`--strict-warnings`, clang aggressively warn about unused functions in
such cases, which forces the use of such DEFINE macros to header
files.

We therefore disable this warning from the `--strict-warnings`
definition for clang.

(note for the curious: `-Wunused-function` is enabled via `-Wall`)

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8234)

Sparse array iterators include index position.

Iterators over the sparse array structures have gained an initial argument
which indicates the index into the array of the element.  This can be used,
e.g., to delete or modify the associated value.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8229)

Windows/Cygwin dlls need the executable bit set

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8226)

test/build.info: add missing ../apps/include

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8227)

Added new EVP/KDF API.
Changed PKEY/KDF API to call the new API.
Added wrappers for PKCS5_PBKDF2_HMAC() and EVP_PBE_scrypt() to call the new EVP KDF APIs.
Documentation updated.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6674)

Sparse array limit testing: reduce the range limit for the number of bits
in a sparse array pointer block.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8221)

Fix null pointer dereference in cms_RecipientInfo_kari_init

CLA: trivial

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8137)

Move libapps headers into their own directory

This got triggered by test/testutil.h including ../apps/opt.h.

Some compilers do all inclusions from the directory of the C file
being compiled, so when a C file includes a header file with a
relative file spec, and that header file also includes another header
file with a relative file spec, the compiler no longer follows.

As a specific example, test/testutil/basic_output.c included
../testutil.h.  Fine so far, but then, test/testutil.h includes
../apps/opt.h, and the compiler ends up trying to include (seen from
the source top) test/apps/opt.h rather than apps/opt.h, and fails.

The solution could have been to simply add apps/ as an inclusion
directory.  However, that directory also has header files that have
nothing to do with libapps, so we take this a bit further, create
apps/include and move libapps specific headers there, and then add
apps/include as inclusion directory in the build.info files where
needed.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/8210)

Fix master build.
The recent change from ENGINES to MODULES broke the configure it seems.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8219)

Macro typo fix not propagated

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8217)

Fix typo in comment

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/8218)

To use BN_BITS2, we'd better include openssl/bn.h

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8212)

eng_devcrypto.c: close open session on init

cipher_init may be called on an already initialized context, without a
necessary cleanup.  This separates cleanup from initialization, closing
an eventual open session before creating a new one.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7859)

CHANGES: add note about building devcrypto dynamic

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7859)

e_devcrypto: make the /dev/crypto engine dynamic

Engine has been moved from crypto/engine/eng_devcrypto.c to
engines/e_devcrypto.c.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7859)

AArch64 assembly pack: authenticate return addresses.

ARMv8.3 adds pointer authentication extension, which in this case allows
to ensure that, when offloaded to stack, return address is same at return
as at entry to the subroutine. The new instructions are nops on processors
that don't implement the extension, so that the vetification is backward
compatible.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8205)

Add sparse array data type.

This commit adds a space and time efficient sparse array data structure.
The structure's raw API is wrapped by inline functions which provide type
safety.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8197)

Rework build: small correction in unix-Makefile.tmpl

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8162)

Rework build: add special cases for AIX

When reworking the way library file names and extensions were formed,
AIX was lost in the process.  This restores the previous
functionality.

Fixes #8156

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8162)

apps/ocsp.c Use the same HAVE_FORK / NO_FORK as in speed.c

This allows the user to override our defaults if needed, and in a
consistent manner.

Partial fix for #7607

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7624)

test/recipes/02-err_errstr: skip errors that may not be loaded on Windows

Fixes #8091

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8094)

(cherry picked from commit 0e1b0e510dfe078b3fb2586d987d7b49ff8ef0b2)

Build: correct BASE shlib_version_as_filename

This function is designed to use $config{shlib_version} directly
instead of taking an input argument, yet the BASE variant didn't do
this.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8146)

ENGINE modules aren't special, so call them MODULES

The only thing that makes an ENGINE module special is its entry
points.  Other than that, it's a normal dynamically loadable module,
nothing special about it.  This change has us stop pretending anything
else.

We retain using ENGINE as a term for installation, because it's
related to a specific installation directory, and we therefore also
mark ENGINE modules specifically as such with an attribute in the
build.info files.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8147)

Updated test command line parsing to support commmon commands

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6975)

Allow the syntax of the .include directive to optionally have '='

If the old openssl versions not supporting the .include directive
load a config file with it, they will bail out with error.

This change allows using the .include = <filename> syntax which
is interpreted as variable assignment by the old openssl
config file parser.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8141)

Fix comment typo

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8196)

Fix null pointer dereference in ssl_module_init

CLA: Trivial

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8183)

Update d2i_PrivateKey documentation

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8168)

Fix d2i_PublicKey() for EC keys

o2i_ECPublicKey() requires an EC_KEY structure filled with an EC_GROUP.

o2i_ECPublicKey() is called by d2i_PublicKey(). In order to fulfill the
o2i_ECPublicKey()'s requirement, d2i_PublicKey() needs to be called with
an EVP_PKEY with an EC_KEY containing an EC_GROUP.

However, the call to EVP_PKEY_set_type() frees any existing key structure
inside the EVP_PKEY, thus freeing the EC_KEY with the EC_GROUP that
o2i_ECPublicKey() needs.

This means you can't d2i_PublicKey() for an EC key...

The fix is to check to see if the type is already set appropriately, and
if so, not call EVP_PKEY_set_type().

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8168)

Address a bug in the DRBG tests where the reseeding wasn't properly
reinstantiating the DRBG.

Bug reported by Doug Gibbons.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/8184)

test/drbgtest.c: call OPENSSL_thread_stop() explicitly

The manual says this in its notes:

    ... and therefore applications using static linking should also call
    OPENSSL_thread_stop() on each thread. ...

Fixes #8171

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8173)

Make OPENSSL_malloc_init() a no-op

Making this a no-op removes a potential infinite loop than can occur in
some situations.

Fixes #2865

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8167)

Add CHANGES entry for blake2mac

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8172)

blake2: avoid writing to output buffer when using default digest length

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7726)

blake2: add evpmac test vectors

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7726)

blake2: backport changes to blake2s

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7726)

blake2: add EVP_MAC man page

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7726)

blake2: register MAC objects

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7726)

blake2b: add EVP_MAC API

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7726)

blake2b: add support for parameter setting and keyed hash

The param block structure is used as a container for parameter values
Added blake2b keyed init

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7726)

blake2: add implementation support for variable digest length

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7726)

Remove unnecessary trailing whitespace

Trim trailing whitespace. It doesn't match OpenSSL coding standards,
AFAICT, and it can cause problems with git tooling.

Trailing whitespace remains in test data and external source.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8092)

crypto/poly1305/asm/poly1305-s390x.pl: add vx code path.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7991)

Make some simple getters take const SSL/SSL_CTX

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8145)

Fix Invalid Argument return code from IP_Factory in connect_to_server().

Fixes #7732

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8158)

Android build: fix usage of NDK home variable ($ndk_var)

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8153)

Build: correct assembler generation in crypto/rc4/build.info

In the removal of BEGINRAW / ENDRAW, attention to the difference
between capital .S and lowercase .s wasn't duly paid.  This corrects
the error.

Fixes #8155

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8157)

Add an entry to the CHANGES for the d2i_X509_PUBKEY fix

The commit 5dc40a83c74be579575a512b30d9c1e0364e6a7b forgot
to add a short description to the CHANGES file.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8144)

VMS: Clean away stray debugging prints from descrip.mms.tmpl

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8140)

Fix end-point shared secret for DTLS/SCTP

When computing the end-point shared secret, don't take the
terminating NULL character into account.
Please note that this fix breaks interoperability with older
versions of OpenSSL, which are not fixed.

Fixes #7956

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7957)

poly1305/asm/poly1305-ppc.pl: add vector base 2^26 implementation.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8120)

perlasm/ppc-xlate.pl: add VSX word load/store instructions.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8120)

Fix a crash in reuse of i2d_X509_PUBKEY

If the second PUBKEY is malformed there is use after free.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8122)

Fixed d2i_X509 in-place not re-hashing the ex_flags

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8116)

Fix a memory leak with di2_X509_CRL reuse

Additionally avoid undefined behavior with
in-place memcpy in X509_CRL_digest.

Fixes #8099

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8112)

Better phrasing around 1.1.0

Fixes #8129

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8130)

Configure: clean away unused variables and double assignments

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8125)

Build: clean away RENAME and SHARED_NAME

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8125)

Build: remove EXTRA

We never used it for anything

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8125)

Build: Remove BEGINRAW / ENDRAW / OVERRIDE

It was an ugly hack to avoid certain problems that are no more.

Also added GENERATE lines for perlasm scripts that didn't have that
explicitly.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8125)

Build cleanup: Remove the VMS hack from test/build.info

There was a hack specifically for VMS, which involved setting a make
variable to indicate that test/libtestutil contains a 'main'.

Instead, we use the new attributes 'has_main' to indicate this, and
let the VMS build file template fend with it appropriately.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8125)

VMS: force 'pinshared'

VMS doesn't currently support unloading of shared object, and we need
to reflect that.  Without this, the shlibload test fails

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8131)

Fix error message for s_server -psk option

Previously if -psk was given a bad key it would print "Not a hex
number 's_server'".

CLA: Trivial

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/8113)

Reuse already defined macros

instead of duplicity the code.

CLA: trivial

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8127)

Document and add macros for additional DSA options

EVP_PKEY_CTRL_DSA_PARAMGEN_Q_BITS and EVP_PKEY_CTRL_DSA_PARAMGEN_MD are only
exposed from EVP_PKEY_CTX_ctrl, which means callers must write more error-prone
code (see also issue #1319). Add the missing wrapper macros and document them.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8093)

Complain if -twopass is used incorrectly

The option -twopass to the pkcs12 app is ignored if -passin, -passout
or -password is used. We should complain if an attempt is made to use
it in combination with those options.

Fixes #8107

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8114)

Fix no-dso builds

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8111)

Don't leak memory from ERR_add_error_vdata()

If the call the ERR_set_error_data() in ERR_add_error_vdata() fails then
a mem leak can occur. This commit checks that we successfully added the
error data, and if not frees the buffer.

Fixes #8085

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/8105)

Android build: use ANDROID_NDK_HOME rather than ANDROID_NDK

It apepars that ANDROID_NDK_HOME is the recommended standard
environment variable for the NDK.

We retain ANDROID_NDK as a fallback.

Fixes #8101

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8103)

crypto/cms: Add support for CAdES Basic Electronic Signatures (CAdES-BES)

A CAdES Basic Electronic Signature (CAdES-BES) contains, among other
specifications, a collection of  Signing Certificate reference attributes,
stored in the signedData ether as ESS signing-certificate or as
ESS signing-certificate-v2. These are described in detail in Section 5.7.2
of RFC 5126 - CMS Advanced Electronic Signatures (CAdES).

This patch adds support for adding  ESS signing-certificate[-v2] attributes
to CMS signedData. Although it implements only a small part of the RFC, it
is sufficient many cases to enable the `openssl cms` app to create signatures
which comply with legal requirements of some European States (e.g Italy).

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7893)

add an additional async notification communication method based on callback

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Signed-off-by: Ping Yu <ping.yu@intel.com>
Signed-off-by: Steven Linsell <stevenx.linsell@intel.com>
(Merged from https://github.com/openssl/openssl/pull/7573)

clarify which functions are the CMS functions which must have CMS_PARTIAL set

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7960)

crypto/bn: fix return value in BN_generate_prime

When the ret parameter is NULL the generated prime
is in rnd variable and not in ret.

CLA: trivial

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8076)

s_client: fix not to send a command letter of R

Before 1.1.0, this command letter is not sent to a server.

CLA: trivial
(cherry picked from commit bc180cb4887c2e82111cb714723a94de9f6d2c35)

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8081)

Remove stray -modulus option from the ec manual page.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8082)

Add "weak" declarations of symbols used in safestack.h and lhash.h

Only for SunCC for now.

It turns out that some compilers to generate external variants of
unused static inline functions, and if they use other external
symbols, those need to be present as well.  If you then happen to
include one of safestack.h or lhash.h without linking with libcrypto,
the build fails.

Fixes #6912

Signed-off-by: Matthias Kraft <Matthias.Kraft@softwareag.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8087)

X509_STORE: fix two misspelled compatibility macros

Fixes #8084

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8086)

Cleanup vxworks support to be able to compile for VxWorks 7

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7569)

Fix s_client so that it builds on Windows

Fixes #8050

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8065)

Revert "Keep the DTLS timer running after the end of the handshake if appropriate"

This commit erroneously kept the DTLS timer running after the end of the
handshake. This is not correct behaviour and shold be reverted.

This reverts commit f7506416b1311e65d5c440defdbcfe176f633c50.

Fixes #7998

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8047)

Make sure we trigger retransmits in DTLS testing

During a DTLS handshake we may need to periodically handle timeouts in the
DTLS timer to ensure retransmits due to lost packets are performed. However,
one peer will always complete a handshake before the other. The DTLS timer
stops once the handshake has finished so any handshake messages lost after
that point will not automatically get retransmitted simply by calling
DTLSv1_handle_timeout(). However attempting an SSL_read implies a
DTLSv1_handle_timeout() and additionally will process records received from
the peer. If those records are themselves retransmits then we know that the
peer has not completed its handshake yet and a retransmit of our final
flight automatically occurs.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8047)

Build: change remaining $unified_info{install} checks to use attributes

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8063)

Rework build: Windows dependency building fix

One variable misssing

Fixes #8060

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8061)

Build: pass attributes down to make rule generators

For good measure, we pass down attributes when calling obj2shlib,
obj2lib, obj2dso, obj2bin, or in2script.  We currently don't use them
in our build file templates, but might as well for future use.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7581)

Build: use attributes to indicate installed script classes

We have two classes of scripts to be installed, those that are
installed as "normal" programs, and those that are installed as "misc"
scripts.  These classes are installed in different locations, so the
build file templates must pay attention.

Because we didn't have the tools to indicate what scripts go where, we
had these scripts hard coded in the build template files, with the
maintenance issues that may cause.  Now that we have attributes, those
can be used to classify the installed scripts, and have the build file
templates simply check the attributes to know what's what.

Furthermore, the 'tsget.pl' script exists both as 'tsget.pl' and
'tsget', which is done by installing a symbolic link (or copy).  This
link name is now given through an attribute, which results in even
less hard coding in the Unix Makefile template.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7581)

Build: Change all _NO_INST to use attributes instead.

This means that all PROGRAMS_NO_INST, LIBS_NO_INST, ENGINES_NO_INST
and SCRIPTS_NO_INST are changed to be PROGRAM, LIBS, ENGINES and
SCRIPTS with the associated attribute 'noinst'.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7581)

Configure: add attributes to end product build.info variables

Among others, this avoids having special variables like
PROGRAMS_NO_INST.  Instead, we can have something like this:

    PROGRAMS{noinst}=foo bar

Configure itself is entirely agnostic to these attributes, they are
simply passed to the build file templates, to be used as they see fit.

Attributes can also have values, for example:

    SCRIPTS{linkname=foo}=foo.pl

This could help indicate to build file templates that care that the
perl script 'foo.pl' should also exist with the name 'foo', preferably
as a symbolic link.

Fixes #7568

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7581)

Configure: teach the tokenizer to handle other separators than spaces

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7581)

Update NOTES.ANDROID

Minor typo fix to `adjustment` in the line:
"In such case you have to pass matching target
 name to Configure and shouldn't use -D__ANDROID_API__=N. PATH adjustment
 becomes simpler, $ANDROID_NDK/bin:$PATH suffices."

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8054)

Rework building: Get rid of old %unified_info structures

Now that we have the names of libraries on different systems
established through platform modules, we can remove the old structure
to establish the same thing, i.e. $unified_info{sharednames} and
$unified_info{rename}.  That means removing support for the RENAME and
SHARED_NAME keywords in build.info as well.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7473)

Rework building: adapt some scripts

The platform module collection is made in such a way that any Perl
script that wants to take part of the available information can use
them just as well as the build system.

This change adapts test/recipes/90-test_shlibload.t, util/mkdef.pl,
and util/shlib_wrap.sh.in

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7473)

Rework building: Unix changes to handle extensions and product names

Add platform::Unix, which is a generic Unix module to support product
name and extensions functionlity.  However, this isn't quite enough,
as mingw and Cygwin builds are done using the same templates, but
since shared libraries work as on Windows and are named accordingly,
platform::mingw and platform::Cygwin were also added to provide the
necessary tweaks.

This reworks Configurations/unix-Makefile.tmpl to work out product
names in platform::Unix et al terms.  In this one, we currently do
care about the *_extension config attributes, and the modules adapt
accordingly where it matters.

This change also affected crypto/include/internal/dso_conf.h.in, since
the DSO extension is meant to be the same as the short shared library
extension, which isn't '.so' everywhere.

'shared_extension' attributes that had the value
'.so.\$(SHLIB_VERSION_NUMBER)' are removed, platform::Unix provides
an extension where the shared library version number is hard-coded
instead.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7473)

Rework building: VMS changes to handle extensions and product names

Add platform::VMS, which is a generic VMS module.  Additional modules
to support specific building aspects (such as specific compilers) may
be added later, but since we currently work on file names and those
are generic enough, this is also enough.

This reworks Configurations/descrip.mms.tmpl to work out product names
in platform::VMS terms.  Something to be noted is that the new
functionality ignores the *_extension config attributes, as they were
never used.  VMS is very consistent in its use of extensions, so there
is no reason to believe much will change in this respect.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7473)