2nd chunk: CRMF code (crypto/crmf/, ) and its integration
in INSTALL, Configure, crypto/build.info, include/openssl/crmferr.h,
crypto/err/, include/openssl/err.h, and (to be updated:) util/libcrypto.num

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7646)

Certificate Management Protocol (CMP, RFC 4210) extension to OpenSSL
Also includes CRMF (RFC 4211) and HTTP transfer (RFC 6712)

CMP and CRMF API is added to libcrypto, and the "cmp" app to the openssl CLI.
Adds extensive man pages and tests.  Integration into build scripts.

Incremental pull request based on OpenSSL commit 1362190b1b78 of 2018-09-26

1st chunk: CRMF API (include/openssl/crmf.h) and its documentation (reviewed)

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7328)

added generated files

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6652)

FIPS 186-4 RSA Generation & Validation

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6652)

s390x assembly pack: import chacha from cryptogams repo

featuring 6x"horizontal" code path which is up to 25%
faster than present 4x"vertical" for larger blocks.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Add documentation

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Add provider tests

Two tests are added, one that tests the internal API, the other tests
the public API.  Those two tests both test the same provider, which
acts both as a built-in provider and as a loadable provider module.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Add provider module infrastructure

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Replumbing: New public API to load or add providers

Adding a provider means creating an internal provier object and adding
it to the store.  This allows the addition of built in providers, be it
in the OpenSSL libraries or in any application.

"Loading" a provider is defined broadly.  A built in provider is already
"loaded" in essence and only needs activating, while a provider in a
dynamically loadable module requires actually loading the module itself.
In this API, "loading" a provider does both.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Replumbing: Add the Provider Object, type OSSL_PROVIDER

The OSSL_PROVIDER is the core object involved in loading a provider
module, initialize a provider and do the initial communication of
provider wide and core wide dispatch tables.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Replumbing: Add MODULESDIR macro and OPENSSL_MODULES environment variable

These will be used to point out general OpenSSL modules directory.
ENGINE modules are kept apart for backward compatibility.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

fix truncation of integers on 32bit AIX

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8417)

Add regenerated header files

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8397)

util/mkerr.pl: Add an inclusion of symhacks.h in all error files

This does no harm, and ensures that the inclusion isn't mistakenly
removed in the generated *err.h where it's actually needed.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8397)

openssl app for macs that uses the new EVP_MAC interface (the code inside dgst uses EVP_PKEY)

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7661)

sha/asm/keccak1600-ppc64.pl: up 10% performance improvement.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8444)

Fix two spelling errors

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8447)

Fix typo in ifndef OPENSSL_NO_ENGINES.

All other instances are OPENSSL_NO_ENGINE without the trailing "S".
Fixes build when configured with no-engine.

CLA: trivial

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8449)

Print all loaded engines with openssl list -engines

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8434)

Change arg to cms_CompressedData_init_bio to be const

The argument to this function is declared const in the header file. However
the implementation did not have this. This issue is only visible when using
enable-zlib.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8433)

Update documentation of RSA_padding_check_PKCS1_OAEP_mgf1

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8365)

Do the error handling in pkey_rsa_decrypt in constant time

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8365)

Add a simple test for RSA_SSLV23_PADDING

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8365)

Update documentation regarding required output buffer memory size
of RSA_private_decrypt/RSA_public_encrypt.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8365)

Fix memory overrun in rsa padding check functions

Fixes #8364 and #8357

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8365)

Limit DEVRANDOM_WAIT to linux

Fixes #8416

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8428)

Add a test for underflow in ecp_nistp521.c

The previous commit fixed an underflow that may occur in ecp_nistp521.c.
This commit adds a test for that condition. It is heavily based on an
original test harness by Billy Brumley.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8405)

Avoid an underflow in ecp_nistp521.c

The function felem_diff_128_64 in ecp_nistp521.c substracts the number |in|
from |out| mod p. In order to avoid underflow it first adds 32p mod p
(which is equivalent to 0 mod p) to |out|. The comments and variable naming
suggest that the original author intended to add 64p mod p. In fact it
has been shown that with certain unusual co-ordinates it is possible to
cause an underflow in this function when only adding 32p mod p while
performing a point double operation. By changing this to 64p mod p the
underflow is avoided.

It turns out to be quite difficult to construct points that satisfy the
underflow criteria although this has been done and the underflow
demonstrated. However none of these points are actually on the curve.
Finding points that satisfy the underflow criteria and are also *on* the
curve is considered significantly more difficult. For this reason we do
not believe that this issue is currently practically exploitable and
therefore no CVE has been assigned.

This only impacts builds using the enable-ec_nistp_64_gcc_128 Configure
option.

With thanks to Bo-Yin Yang, Billy Brumley and Dr Liu for their significant
help in investigating this issue.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8405)

FIPS AES_GCM IV gen changes

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8393)

constify *_dup() and *i2d_*() and related functions as far as possible, introducing DECLARE_ASN1_DUP_FUNCTION

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8029)

Update ChaCha20-Poly1305 documentation

Correctly describe the maximum IV length.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8406)

Test an overlong ChaCha20-Poly1305 nonce

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8406)

Prevent over long nonces in ChaCha20-Poly1305

ChaCha20-Poly1305 is an AEAD cipher, and requires a unique nonce input for
every encryption operation. RFC 7539 specifies that the nonce value (IV)
should be 96 bits (12 bytes). OpenSSL allows a variable nonce length and
front pads the nonce with 0 bytes if it is less than 12 bytes. However it
also incorrectly allows a nonce to be set of up to 16 bytes. In this case
only the last 12 bytes are significant and any additional leading bytes are
ignored.

It is a requirement of using this cipher that nonce values are unique.
Messages encrypted using a reused nonce value are susceptible to serious
confidentiality and integrity attacks. If an application changes the
default nonce length to be longer than 12 bytes and then makes a change to
the leading bytes of the nonce expecting the new value to be a new unique
nonce then such an application could inadvertently encrypt messages with a
reused nonce.

Additionally the ignored bytes in a long nonce are not covered by the
integrity guarantee of this cipher. Any application that relies on the
integrity of these ignored leading bytes of a long nonce may be further
affected.

Any OpenSSL internal use of this cipher, including in SSL/TLS, is safe
because no such use sets such a long nonce value. However user
applications that use this cipher directly and set a non-default nonce
length to be longer than 12 bytes may be vulnerable.

CVE-2019-1543

Fixes #8345

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8406)

apps/openssl.c: avoid memory leaks

The trace API doesn't know that the BIOs we give it, let alone those
we attach to callbacks as 'void *data', need to be cleaned up.  This
must be done in the application.

To ensure this cleanup is done as late as possible, use atexit().

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Add a log about the tracing functionality

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Document the tracing functionality

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Make it possible to trace the trace functionality itself

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Adapt BN_CTX_DEBUG to the new generic trace API

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Adapt OPENSSL_DEBUG_DECRYPT to the new generic trace API

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Adapt OPENSSL_POLICY_DEBUG to the new generic trace API

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Adapt OPENSSL_DEBUG_KEYGEN to the new generic trace API

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Adapt OPENSSL_DEBUG_PKCS5V2 to the new generic trace API

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Adapt ENGINE_REF_COUNT_DEBUG to the new generic trace API

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Adapt ENGINE_TABLE_DEBUG to the new generic trace API

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Adapt ENGINE_CONF_DEBUG to the new generic trace API

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Adapt OPENSSL_INIT_DEBUG to the new generic trace API

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Adapt CIPHER_DEBUG to the new generic trace API

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Adapt SSL_DEBUG to the new generic trace API

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Make it possible to disable the TRACE API

This disabled the tracing functionality by making functions do
nothing, and making convenience macros produce dead code.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

apps/openssl.c: Adapt to enable tracing output

Use the environment variables OPENSSL_TRACE to determine what's going
to be enabled.  The value of this variables is a comma separated list
of trace and debugging names, which correspond to the trace category
macros defined in include/openssl/trace.h.

For example, setting OPENSSL_DEBUG=TRACE,SSL will enable debugging output
for the types OSSL_TRACE_CATEGORY_TRACE and OSSL_TRACE_CATEGORY_SSL.

This also slightly changes the handling of the prefix method in
apps/apps.c.  This is for the better, as the prefix method pointer was
unneccessarily stored in two places.

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Add generic trace API

The idea is that the application shall be able to register output
channels or callbacks to print tracing output as it sees fit.

OpenSSL internals, on the other hand, want to print thoses texts using
normal printing routines, such as BIO_printf() or BIO_dump() through
well defined BIOs.

When the application registers callbacks, the tracing functionality
sets up an internal BIO that simply forwards received text to the
appropriate application provided callback.

Co-authored-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8198)

Make the sparse array code use ossl_uintmax_t as its index rather than size_t.

This should never reduce the range covered and might increase it on some
platforms.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8415)

padlock: generate assembler source for static libraries too

The GENERATE lines for generating the padlock assembler source were
wrongly placed in such a way that they only applied to the shared
library build.

[extended tests]

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/8412)

Don't write the tick_identity to the session

Sessions must be immutable once they can be shared with multiple threads.
We were breaking that rule by writing the ticket index into it during the
handshake. This can lead to incorrect behaviour, including failed
connections in multi-threaded environments.

Reported by David Benjamin.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8383)

Add advice on setting CXX at the same time as CC

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8370)

Travis: use enable-buildtest-c++

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8370)

Configure: make C++ build tests optional and configurable

Disabled by default

Fixes #8360

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8370)

VMS: fix util/[un]local.com.in to handle attributes

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8382)

testutil: ensure good treatment of argv on non-Unix platforms

From a Unix point of view, some other platform families have certain
quirks.  Windows command prompt doesn't expand globs into actual file
names, so we must do this.  VMS has some oddity with argv pointer size
that can cause crashes if you're not careful (by copying it to a less
surprising pointer size array).

The fixups already exist and are used in the apps/ code.  However, the
testutil code started using the opt routines from apps/ without
including the non-Unix fixups.  This change fixes that.

For VMS' sake, libtestutil gets an app_malloc() shim, to avoid sucking
in all of apps/apps.c.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8381)

VMS: move copy_argc to its own module and make it an aux source

copy_argv was never initialization code.

Make it self-cleaning too.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8381)

Deprecate the "hw" configuration options, make "padlockeng" disablable

The "hw" and "hw-.*" style options are historical artifacts, sprung
from the time when ENGINE was first designed, with hardware crypto
accelerators and HSMs in mind.

Today, these options have largely lost their value, replaced by
options such as "no-{foo}eng" and "no-engine".

This completes the transition by making "hw" and "hw-.*" deprecated,
but automatically translated into more modern variants of the same.

In the process, we get rid of the last regular expression in
Configure's @disablables, a feature that was ill supported anyway.
Also, padlock now gets treated just as every other engine.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8380)

openssl_strerror_r: Fix handling of GNU strerror_r

GNU strerror_r may return either a pointer to a string that the function
stores in buf, or a pointer to some (immutable) static string in which case
buf is unused.

In such a case we need to set buf manually.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8371)

Check for negative return for signature size.Addresses Coverity 1442933

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8392)

Fix seeding from random device w/o getrandom syscall

Use select to wait for /dev/random in readable state,
but do not actually read anything from /dev/random,
use /dev/urandom first.

Use linux define __NR_getrandom instead of the
glibc define SYS_getrandom, in case the kernel headers
are more current than the glibc headers.

Fixes #8215

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8251)

Fix no-ec, no-sm2 and no-sm3

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8372)

Add section order check in util/find-doc-nits

This patch checks if the EXAMPLES section in a pod file is placed
before the RETURN VALUES section.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8338)

Place return values after examples in doc

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8338)

apps/ca.c: only output DER with SPKAC input and when -out is chosen

So say the docs

Fixes #8055

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8368)

The use of the likes of UINT32_MAX requires internal/numbers.h

Found a few more cases.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8366)

Configure: support a few more "make variables" defaulting from env

CFLAGS, CXXFLAGS, CPPFLAGS, LDFLAGS, and LDLIBS

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8325)

.travis.yml: change -std=c89 to -ansi

For C, -ansi is equivalent to -std=c90
For C++, -ansi is equivalent to -std=c++98

We also place -ansi in CPPFLAGS instead of the usual command line config,
to avoid getting it when linking (clang complains)

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8325)

Configuration: divide devteam flags into language specific sets

Some of the devteam flags are not for C++

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8325)

Do buildtests on our public header files with C++ as well

This ensures that we don't mistakenly use C++ keywords anywhere public.

Related to #8313

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8325)

OpenSSL::Util::Pod: allow slashes in names

The names in the NAME section may describe headers, which contain a slash
for OpenSSL headers.  We deal with that by converting slashes to dashes
for the file names.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8286)

Replumbing: Add include/openssl/core.h, initially with core types

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8286)

Implement SSH KDF

SSH's KDF is defined in RFC 4253 in Section 7.2

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7290)

Make the padlock engine build correctly

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8220)

Ensure configured module specific and application specific defines are used

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8220)

Add PADLOCK_ASM to dso_defines rather than lib_defines

Since the padlock code is an engine, the assembler is for a module,
not a library link to when building a program...  there's a
distinction.

Fixes #2311

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8220)

The use of the likes of UINT32_MAX requires internal/numbers.h

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8354)

Fix the default digest algorthm of SM2

Currently SM2 shares the ameth with EC, so the current default digest
algorithm returned is SHA256. This fixes the default digest algorithm of
SM2 to SM3, which is the only valid digest algorithm for SM2 signature.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8186)

Support raw input data in apps/pkeyutl

Some signature algorithms require special treatment for digesting, such
as SM2. This patch adds the ability of handling raw input data in
apps/pkeyutl other than accepting only pre-hashed input data.

Beside, SM2 requries an ID string when signing or verifying a piece of data,
this patch also adds the ability for apps/pkeyutil to specify that ID
string.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8186)

Revert "Configure: stop forcing use of DEFINE macros in headers"

Github PR #8246 provides a better solution to the problem.

This reverts commit f11ffa505f8a9345145a26a05bf77b012b6941bd.

[extended tests]

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8247)

Add BN_native2bn and BN_bn2nativepad, for native BIGNUM import/export

These are a couple of utility functions, to make import and export of
BIGNUMs to byte strings in platform native for (little-endian or
big-endian) easier.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8346)

[test] modernize ecdsatest and extend ECDSA sign KATs

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8314)

Fix trivial typo in EVP_DigestVerifyInit doc

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8319)

Clarify that SSL_shutdown() must not be called after a fatal error

Follow on from CVE-2019-1559

Reviewed-by: Richard Levitte <levitte@openssl.org>

Disable 02-test_errstr.t on msys/mingw as well as MSWin32

There is too high a risk that perl and OpenSSL are linked with
different C RTLs, and thereby get different messages for even the most
mundane error numbers.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8343)

Rearrange the inclusion of curve448/curve448_lcl.h

The real cause for this change is that test/ec_internal_test.c
includes ec_lcl.h, and including curve448/curve448_lcl.h from there
doesn't work so well with compilers who always do inclusions relative
to the C file being compiled.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8334)

(cherry picked from commit f408e2a352b59f2e7aa2160bfb6285725fe88ea7)

Ensure bn_cmp_words can handle the case where n == 0

Thanks to David Benjamin who reported this, performed the analysis and
suggested the patch. I have incorporated some of his analysis in the
comments below.

This issue can cause an out-of-bounds read. It is believed that this was
not reachable until the recent "fixed top" changes. Analysis has so far
only identified one code path that can encounter this - although it is
possible that others may be found. The one code path only impacts 1.0.2 in
certain builds. The fuzzer found a path in RSA where iqmp is too large. If
the input is all zeros, the RSA CRT logic will multiply a padded zero by
iqmp. Two mitigating factors:

- Private keys which trip this are invalid (iqmp is not reduced mod p).
Only systems which take untrusted private keys care.
- In OpenSSL 1.1.x, there is a check which rejects the oversize iqmp,
so the bug is only reproducible in 1.0.2 so far.

Fortunately, the bug appears to be relatively harmless. The consequences of
bn_cmp_word's misbehavior are:

- OpenSSL may crash if the buffers are page-aligned and the previous page is
non-existent.
- OpenSSL will incorrectly treat two BN_ULONG buffers as not equal when they
are equal.
- Side channel concerns.

The first is indeed a concern and is a DoS bug. The second is fine in this
context. bn_cmp_word and bn_cmp_part_words are used to compute abs(a0 - a1)
in Karatsuba. If a0 = a1, it does not matter whether we use a0 - a1 or
a1 - a0. The third would be worth thinking about, but it is overshadowed
by the entire Karatsuba implementation not being constant time.

Due to the difficulty of tripping this and the low impact no CVE is felt
necessary for this issue.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8326)

fix x509 -force_pubkey option to take effect with cert input or self-signing; improve its doc

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8165)

CID 1442838:  API usage errors

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/8318)

CID 1442835:  Integer Overflow

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/8318)

CID 1442836:  Resource leaks

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/8318)

Windows: Call TerminateProcess, not ExitProcess

Ty Baen-Price explains:

> Problem and Resolution:
> The following lines of code make use of the Microsoft API ExitProcess:
>
> ```
> Apps\Speed.c line 335: ExitProcess(ret);
> Ms\uplink.c line 22: ExitProcess(1);
> ```
>
> These function calls are made after fatal errors are detected and
> program termination is desired. ExitProcess(), however causes
> _orderly_ shutdown of a process and all its threads, i.e. it unloads
> all dlls and runs all destructors. See MSDN for details of exactly
> what happens
> (https://msdn.microsoft.com/en-us/library/windows/desktop/ms682658(v=vs.85).aspx).
> The MSDN page states that ExitProcess should never be called unless
> it is _known to be safe_ to call it. These calls should simply be
> replaced with calls to TerminateProcess(), which is what should be
> called for _disorderly_ shutdown.
>
> An example of usage:
>
> ```
> TerminateProcess(GetCurrentProcess(), exitcode);
> ```
>
> Effect of Problem:
> Because of a compilation error (wrong c++ runtime), my program
> executed the uplink.c ExitProcess() call. This caused the single
> OpenSSL thread to start executing the destructors of all my dlls,
> and their objects. Unfortunately, about 30 other threads were
> happily using those objects at that time, eventually causing a
> 0xC0000005 ACCESS_VIOLATION. Obviously an ACCESS_VIOLATION is the
> best case scenario, as I'm sure you can imagine at the consequences
> of undiscovered memory corruption, even in a terminating process.

And on the subject of `TerminateProcess()` being asynchronous:

> That is technically true, but I think it's probably synchronous
> "enough" for your purposes, since a call to TerminateProcess
> suspends execution of all threads in the target process. This means
> it's really only asynchronous if you're calling TerminateProcess one
> some _other_ process. If you're calling TerminateProcess on your own
> process, you'll never return from the TerminateProcess call.

Fixes #2489
Was originally RT-4526

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8301)

Don't restrict the number of KeyUpdate messages we can process

Prior to this commit we were keeping a count of how many KeyUpdates we
have processed and failing if we had had too many. This simplistic approach
is not sufficient for long running connections. Since many KeyUpdates
would not be a particular good DoS route anyway, the simplest solution is
to simply remove the key update count.

Fixes #8068

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8299)

engines/dasync: add explaining comments about AES-128-CBC-HMAC-SHA1

Fixes #7950

It was reported that there might be a null pointer dereference in the
implementation of the dasync_aes_128_cbc_hmac_sha1() cipher, because
EVP_aes_128_cbc_hmac_sha1() can return a null pointer if AES-NI is
not available. It took some analysis to find out that this is not
an issue in practice, and these comments explain the reason to comfort
further NPD hunters.

Detected by GitHub user @wurongxin1987 using the Sourcebrella Pinpoint
static analyzer.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8305)

engines/e_devcrypto: set digest input_blocksize

This restores the behavior of previous versions of the /dev/crypto
engine, in alignment with the default implementation.

Reported-by: Gerard Looije <lglooije@hotmail.com>
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8213)

engines/e_devcrypto: fixes logic in close_devcrypto

Call close(cfd) before setting cfd = -1.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8213)

engines/e_devcrypto.c: fix cipher_ctrl function

This fixes commit c703a80, which had a mistake in cipher_ctrl function.

Move the /dev/crypto session cleanup code to its own function.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8213)