Updated doc for BN_clear, BN_CTX_end when param is NULL

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8532)

replace 'OpenSSL license' by 'Apache License 2.0'

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8527)

removed BN_clear NULL checks

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8507)

Replumbing: Add a mechanism to pre-populate the provider store

OpenSSL will come with a set of well known providers, some of which
need to be accessible from the start.  These are typically built in
providers, or providers that will work as fallbacks.

We do this when creating a new provider store, which means that this
will happen in every library context, regardless of if it's the global
default one, or an explicitely created one.

We keep the data about the known providers we want to make accessible
this way in crypto/provider_predefined.h, which may become generated.
For now, though, we make it simple and edited manually.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8480)

Replumbing: add fallback provider capability

To ensure that old applications aren't left without any provider, and
at the same time not forcing any default provider on applications that
know how to deal with them, we device the concept of fallback
providers, which are automatically activated if no other provider is
already activated.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8480)

Update pkeyutl documentation about the digest option

DSA can accept other digests other than SHA1. EC ignores the digest option
altogether.

Fixes #8425

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8432)

Single step kdf implementation

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8230)

Move ASN1_BROKEN macros

They're only used in one place, and only for a legacy datatype.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8302)

EVP_PKEY_get0_engine documentation

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8329)

Providing missing accessor to EVP_PKEY.engine

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8329)

apps/speed.c: properly address NO_EC2M on systems without SIGALRM

The ecdh_c array is allocated of the same size as ecdh_choices,
whose size depends on whether the support for binary curves is enabled
or not.  (The same goes for ecdsa_c).
On systems without SIGALRM, ecdh_c is indexed by predefined constants
intended for representing the index of the ciphers in the ecdh_choices
array.
However, in case of NO_EC2M some of the #defined constants won't match
and would actually access the ecdh_c out-of-bounds.

Use enum instead of a macro to define the curve indexes so they're
within the bounds of the ecdh_c array.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8422)

apps/speed.c: skip binary curves when compiling with OPENSSL_NO_EC2M

openssl speed doesn't take into account that the library could be
compiled without the support for the binary curves and happily uses
them, which results in EC_GROUP_new_by_curve_name() errors.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8422)

Add documentation for the -sigopt option.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/8520)

Configure: untabify indentation

    The indentation in the Configure file is currently very strange when
    viewed in an editor with a tab width of four spaces, because it has
    mixed tab-and-whitespace indentation, which was apparently done with
    a tab width of eight spaces.

    This commit converts all tabs to spaces using expand(1) with default
settings. To verify that there are only whitespace changes, use

       git show --ignore-space-change  <this commit>

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8492)

Fix resource leak coverity 1443711.
Free the allocated pointer on error.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8503)

Fix compiling error for mips32r6 and mips64r6

There are some compiling errors for mips32r6 and mips64r6:

crypto/bn/bn-mips.S:56: Error: opcode not supported on this processor: mips2 (mips2) `mulu $1,$12,$7'
crypto/mips_arch.h: Assembler messages:
crypto/mips_arch.h:15: Error: junk at end of line, first unrecognized character is `&'

Signed-off-by: Hua Zhang <hua.zhang1974@hotmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8464)

Add missing '.text' in crypto/bn/asm/ppc.pl

Fixes #8495

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8496)

Fix no-posix-io

'openssl pkeyutl' uses stat() to determine the file size when signing using
Ed25519/Ed448, and this was guarded with OPENSSL_NO_POSIX_IO.

It is however arguable if stat() is a POSIX IO function, considering
that it doesn't use file descriptors, and even more so since we use
stat() elsewhere without that guard.

This will allow test/recipes/20-test_pkeyutl.t to be able to do its
work for Ed25519/Ed448 signature tests.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8498)

Added NULL check to BN_clear() & BN_CTX_end()

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8518)

Fix strict-warnings build on FreeBSD

The 'key' member of the (system-defined!) struct session op is of
type c_caddr_t, which can be (signed) char, so inter-casting with the
unsigned char* input to cipher_init() causes -Wpointer-sign errors, and we
can't change the signature of cipher_init() due to the function pointer
type required by EVP_CIPHER_meth_set_init().

As the least-bad option, introduce a void* cast to quell the following
warning:
engines/e_devcrypto.c:356:36: error: passing 'c_caddr_t' (aka 'const char *') to
      parameter of type 'const unsigned char *' converts between pointers to integer
      types with different sign [-Werror,-Wpointer-sign]
        return cipher_init(to_ctx, cipher_ctx->sess.key, EVP_CIPHER_CTX_iv(ctx),
                                   ^~~~~~~~~~~~~~~~~~~~
engines/e_devcrypto.c:191:66: note: passing argument to parameter 'key' here
static int cipher_init(EVP_CIPHER_CTX *ctx, const unsigned char *key,

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8509)

doc/man3/OSSL_PARAM_TYPE.pod: modify Example 2 to allow unspecified params

A parameter requestor is never obligated to ask for all available
parameters on an object.  Unfortunately, Example 2 showed a code
pattern that introduced such an obligation, and therefore needed a
small adjustment.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8523)

Add -new and -subj options to x509 app for direct cert generation

Complete and improve error output of parse_name() in apps/apps.c

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8193)

Clear the point S before freeing in ec_scalar_mul_ladder

The secret point R can be recovered from S using the equation R = S - P.
The X and Z coordinates should be sufficient for that.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8504)

Clear the secret point in ecdh_simple_compute_key

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8501)

add 'Signature Value:' line and correct indentation when printing X.509 signature value

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6226)

update reference output of test_x509 in test/certs/cyrillic.*

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6226)

remove 'keyid:' when printing simple X509 authority keyID (without issuer and serial)

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6226)

remove needless empty lines when printing certificates

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6226)

Add generic EVP method fetcher

This is an interface between Core dispatch table fetching and
EVP_{method}_fetch().  All that's needed from the diverse method
fetchers are the functions to create a method structure from a
dispatch table, a function that ups the method reference counter and a
function to free the method (in case of failure).

This routine is internal to the EVP API andis therefore only made
accessible within crypto/evp, by including evp_locl.h

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8341)

Replumbing: better reference counter control in ossl_method_construct()

Fully assume that the method constructors use reference counting.
Otherwise, we may leak memory, or loose track and do a double free.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8341)

Replumbing: pass callback data to the algo destructor too

All relevant OSSL_METHOD_CONSTRUCT_METHOD callbacks got the callback
data passed to them, except 'destruct'.  There's no reason why it
shouldn't get that pointer passed, so we make a small adjustment.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8341)

PPC assembly pack: fix copy-paste error in CTR mode

There are two copy-paste errors in handling CTR mode. When dealing
with a 2 or 3 block tail, the code branches to the CBC decryption exit
path, rather than to the CTR exit path.

This can lead to data corruption: in the Linux kernel we have a copy
of this file, and the bug leads to corruption of the IV, which leads
to data corruption when we call the encryption function again later to
encrypt subsequent blocks.

Originally reported to the Linux kernel by Ondrej Mosnáček <omosnacek@gmail.com>

CLA: trivial

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8510)

coverity fixes for bntest.c

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8508)

EVP_PBE_scrypt() handles salt=NULL as salt=""

Modify EVP_PBE_scrypt() to maintain OpenSSL 1.1.1 behavior: salt=NULL
is now handled as salt="" (and saltlen=0).

Commit 5a285addbf39f91d567f95f04b2b41764127950d changed the behavior
of EVP_PBE_scrypt(salt=NULL). Previously, salt=NULL was accepted, but
the function now fails with KDF_R_MISSING_SALT.

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8483)

test/params_api_test.c: fix size_t assumptions

size_t isn't always as large as a int64_t, so the compiler complains
about possible data loss.  In this case, we are in control, so a
simple cast will do.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8497)

Update to xenial

[extended tests]

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8411)

Configurations/windows-makefile.tmpl: small fixes

Fixes #8467 #8478

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8468)

Guard some SM2 functions with OPENSSL_NO_SM2

Fixes the no-ec build

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8481)

trace: update the documentation

This commit adds some missing symbols and other minor enhancements.
In particular, it establishes the term 'channel' as a synonym for
a BIO object attached to a trace category, and introduces the
concept of a 'simple' channel versus a 'callback' channel.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8463)

trace: ensure correct grouping

It is important that output to the trace channels occurs only inside
a trace group. This precondtion is satisfied whenever the standard
TRACE macros are used. It can be violated only by a bad programming
mistake, like copying the 'trc_out' pointer and using it outside
the trace group.

This commit enforces correct pairing of the OSSL_TRACE_CTRL_BEGIN and
OSSL_TRACE_CTRL_END callbacks, and checks that OSSL_TRACE_CTRL_WRITE
callbacks only occur within such groups.

While implementing it, it turned out that the group assertion failed

  apps/openssl.c:152: OpenSSL internal error: \
                      Assertion failed: trace_data->ingroup

because the set_trace_data() function invokes some callbacks which
generate trace output, but the correct channel type was set only
after the set_trace_data() call.

To fix the failed assertions, the correct channel type is now set
inside the set_trace_data() call, instead of doing it afterwards.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8463)

trace: don't leak the line prefix

The openssl app registers trace callbacks which automatically
set a line prefix in the OSSL_TRACE_CTRL_BEGIN callback.
This prefix needs to be cleared in the OSSL_TRACE_CTRL_END
callback, otherwise a memory leak is reported when openssl
is built with crypto-mdebug enabled.

This leak causes the tests to fail when tracing and memory
debugging are enabled.

The leak can be observed by any command that produces trace
output, e.g. by

  OPENSSL_TRACE=ANY util/shlib_wrap.sh  apps/openssl version
  ...
  [00:19:14]  4061 file=apps/bf_prefix.c, line=152, ...
  26 bytes leaked in 1 chunks

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8463)

trace: rename the trace channel types

Since the term 'channel' is already used as synonym for a BIO object attached
to a trace category, having a 't_channel' channel type and a 't_callback' channel
type somehow overburdens this term. For that reason the type enum constants are
renamed to 'SIMPE_CHANNEL' and 'CALLBACK_CHANNEL'.
(The conversion to capital letters was done to comply to the coding style.)

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8463)

trace: remove some magic numbers

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8463)

VMS: only use the high precision on VMS v8.4 and up

Fixes #8487
Amends #7230

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8488)

internal/refcount.h: allow non-atomic build

Configure with -DOPENSSL_DEV_NO_ATOMICS and you get refcount without
atomics.  This is intended for internal development only, to check the
refcounting is properly coded.  It should never become a configuration
option, hence the name of the macro.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8479)

crypto/provider_core.c: correct definition and use of lock

Fixes #8476

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8477)

Enable pkeyutl to use Ed448 and Ed25519

With the recent addition of the -rawin option it should be possible for
pkeyutl to sign and verify with Ed448 and Ed2559. The main remaining
stumbling block is that those algorirthms only support "oneshot" operation.
This commit enables pkeyutl to handle that.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8431)

Configure: disable new trace api by default

Fixes #8472

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8474)

Don't fail when tracing is disabled

When tracing is disabled, don't generate errors, especially during
init.  Instead, just pretend the everything is fine.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8475)

Fix memory leak in ectest

Fixes #8462

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8466)

test/params_test.c: use TEST_double_eq to check doubles

TEST_ulong_eq was used previously because TEST_double_eq didn't exist
at the time.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8469)

test/params_test.c: make more global variables static

Again, compilers that don't like them being undeclared...

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8469)

test/params_test.c: make construct_api_params() static

With enough warning flags, compilers complain when a non-static
function hasn't been properly declared...

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8469)

prevent app_get_pass() from revealing cleartext password on syntax error

When the argument for '-pass' was badly formed, that argument got
displayed in full.  This turns out to not be such a good idea if the
user simply forgot to start the argument with 'pass:', or spellt the
prefix incorrectly.  We therefore change the display to say that a
colon is missing or only showing the incorrect prefix.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6218)

test/params_test.c: Add run-time constructed OSSL_PARAM variant

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8396)

test/params_test.c: Add API test case, and mixed methods

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8396)

test/params_test.c: Add "real world" parameter testing

test/params_test.c is a program that tries to mimic how a provider and
an application would or could handle OSSL_PARAM arrays.

For the moment, this program tests a very raw way of handling
OSSL_PARAM arrays.  It is, however, written in a way that will
hopefully make it possible to extend with other methods as APIs arise,
and to set up test cases where a "provider" handles the array one way
while the "application" handles it another way.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8396)

Fix no-dso

Don't try to load external provider modules in tests if we don't have DSO
capability

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8459)

Fix various no-* options

no-cmac, no-poly1305 and no-siphash all caused the new test_mac to fail.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8458)

Fix memory leaks in pkread.c demo file

Also make various changes to bring the file into line with current coding
style.

Fixes #8456

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/8457)

Support SM2 certificate verification

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8321)

Replumbing: Add constructor of libcrypto internal method structures

This queries the provider for its available functionality (unless a
matching method structured is already cached, in which case that's
used instead), and creates method structure with the help of a passed
constructor.  The result is cached if the provider allows it (or if
caching is forced).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8340)

Replumbing: Add support for the provider query_operation function

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8340)

Replumbing: Add an OSSL_PROVIDER iterator with callback

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8340)

Change OSSL_PARAM_UTF8_STRING_PTR to OSSL_PARAM_UTF8_PTR

In unrelated code

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8451)

generated files

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8451)

OSSL_PARAM helper functions.

Provide a number of functions to allow parameters to be set and
retrieved in a type safe manner.  Functions are provided for many
integral types plus double, BIGNUM, UTF8 strings and OCTET strings.

All of the integer functions will widen the parameter data as
required.  This permits a degree of malleability in the parameter
definition.  For example a type can be changed from a thirty two bit
integer to a sixty four bit one without changing application code.
Only four and eight byte integral sizes are supported here.

A pair of real functions are available for doubles.

A pair of functions is available for BIGNUMs.  These accept any sized
unsigned integer input and convert to/from a BIGNUM.

For each OCTET and UTF8 strings, four functions are defined.  This
provide get and set functionality for string and for pointers to
strings.  The latter avoiding copies but have other inherent risks.

Finally, some utility macros and functions are defined to allow
OSSL_PARAM definition arrays to be specified in a simple manner.
There are two macro and one function for most types.  The exception
being BIGNUM, for which there is one macro and one function.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8451)

Rename 'buffer' to 'data' in OSSL_PARAM

The OSSL_PARAM attribute names |buffer| and |buffer_size| may lead to
confusion, as they may make some think that the memory pointed at is
an intermediate memory are.  This is not generally the case, so we
rename |buffer| and |buffer_size| to |data| and |data_size|

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8451)

2nd chunk: CRMF code (crypto/crmf/, ) and its integration
in INSTALL, Configure, crypto/build.info, include/openssl/crmferr.h,
crypto/err/, include/openssl/err.h, and (to be updated:) util/libcrypto.num

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7646)

Certificate Management Protocol (CMP, RFC 4210) extension to OpenSSL
Also includes CRMF (RFC 4211) and HTTP transfer (RFC 6712)

CMP and CRMF API is added to libcrypto, and the "cmp" app to the openssl CLI.
Adds extensive man pages and tests.  Integration into build scripts.

Incremental pull request based on OpenSSL commit 1362190b1b78 of 2018-09-26

1st chunk: CRMF API (include/openssl/crmf.h) and its documentation (reviewed)

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7328)

added generated files

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6652)

FIPS 186-4 RSA Generation & Validation

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6652)

s390x assembly pack: import chacha from cryptogams repo

featuring 6x"horizontal" code path which is up to 25%
faster than present 4x"vertical" for larger blocks.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Add documentation

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Add provider tests

Two tests are added, one that tests the internal API, the other tests
the public API.  Those two tests both test the same provider, which
acts both as a built-in provider and as a loadable provider module.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Add provider module infrastructure

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Replumbing: New public API to load or add providers

Adding a provider means creating an internal provier object and adding
it to the store.  This allows the addition of built in providers, be it
in the OpenSSL libraries or in any application.

"Loading" a provider is defined broadly.  A built in provider is already
"loaded" in essence and only needs activating, while a provider in a
dynamically loadable module requires actually loading the module itself.
In this API, "loading" a provider does both.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Replumbing: Add the Provider Object, type OSSL_PROVIDER

The OSSL_PROVIDER is the core object involved in loading a provider
module, initialize a provider and do the initial communication of
provider wide and core wide dispatch tables.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Replumbing: Add MODULESDIR macro and OPENSSL_MODULES environment variable

These will be used to point out general OpenSSL modules directory.
ENGINE modules are kept apart for backward compatibility.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

fix truncation of integers on 32bit AIX

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8417)

Add regenerated header files

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8397)

util/mkerr.pl: Add an inclusion of symhacks.h in all error files

This does no harm, and ensures that the inclusion isn't mistakenly
removed in the generated *err.h where it's actually needed.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8397)

openssl app for macs that uses the new EVP_MAC interface (the code inside dgst uses EVP_PKEY)

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7661)

sha/asm/keccak1600-ppc64.pl: up 10% performance improvement.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8444)

Fix two spelling errors

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8447)

Fix typo in ifndef OPENSSL_NO_ENGINES.

All other instances are OPENSSL_NO_ENGINE without the trailing "S".
Fixes build when configured with no-engine.

CLA: trivial

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8449)

Print all loaded engines with openssl list -engines

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8434)

Change arg to cms_CompressedData_init_bio to be const

The argument to this function is declared const in the header file. However
the implementation did not have this. This issue is only visible when using
enable-zlib.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8433)

Update documentation of RSA_padding_check_PKCS1_OAEP_mgf1

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8365)

Do the error handling in pkey_rsa_decrypt in constant time

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8365)

Add a simple test for RSA_SSLV23_PADDING

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8365)

Update documentation regarding required output buffer memory size
of RSA_private_decrypt/RSA_public_encrypt.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8365)

Fix memory overrun in rsa padding check functions

Fixes #8364 and #8357

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8365)

Limit DEVRANDOM_WAIT to linux

Fixes #8416

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/8428)

Add a test for underflow in ecp_nistp521.c

The previous commit fixed an underflow that may occur in ecp_nistp521.c.
This commit adds a test for that condition. It is heavily based on an
original test harness by Billy Brumley.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8405)

Avoid an underflow in ecp_nistp521.c

The function felem_diff_128_64 in ecp_nistp521.c substracts the number |in|
from |out| mod p. In order to avoid underflow it first adds 32p mod p
(which is equivalent to 0 mod p) to |out|. The comments and variable naming
suggest that the original author intended to add 64p mod p. In fact it
has been shown that with certain unusual co-ordinates it is possible to
cause an underflow in this function when only adding 32p mod p while
performing a point double operation. By changing this to 64p mod p the
underflow is avoided.

It turns out to be quite difficult to construct points that satisfy the
underflow criteria although this has been done and the underflow
demonstrated. However none of these points are actually on the curve.
Finding points that satisfy the underflow criteria and are also *on* the
curve is considered significantly more difficult. For this reason we do
not believe that this issue is currently practically exploitable and
therefore no CVE has been assigned.

This only impacts builds using the enable-ec_nistp_64_gcc_128 Configure
option.

With thanks to Bo-Yin Yang, Billy Brumley and Dr Liu for their significant
help in investigating this issue.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8405)

FIPS AES_GCM IV gen changes

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8393)

constify *_dup() and *i2d_*() and related functions as far as possible, introducing DECLARE_ASN1_DUP_FUNCTION

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8029)

Update ChaCha20-Poly1305 documentation

Correctly describe the maximum IV length.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8406)

Test an overlong ChaCha20-Poly1305 nonce

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8406)