EVP: Fix calls to evp_pkey_export_to_provider()

The calls weren't quite right, as this function has changed its behaviour.
We also change the internal documentation of this function, and document
evp_pkey_downgrade().

Fixes #11549

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11550)

TLSEXT_SIGALG_gostr34102012 0x0840 and 0x0841 support

For GOST2012-GOST8912-GOST8912 IANA introduce signature Signature Algorithm parametrs
https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-16

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11482)

Document X509_verify_ex() and X509_REQ_verify_ex()

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11507)

Introduce an internal version of X509_check_issued()

The internal version is library context aware.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11507)

Create a libctx aware X509_verify_ex()

This is the same as X509_verify() except that it takes a libctx and propq
parameter and signature verification is done using those.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11507)

Put sys-specific files in build.info

Don't wrap whole files in if[n]def, test in build.info if they
should be compiled.  rand_win isn't done as there are multiple
ways to say "this is windows."

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11263)

Use build.info, not ifdef for crypto modules

Don't wrap conditionally-compiled files in global ifndef tests.
Instead, test if the feature is disabled and, if so, do not
compile it.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11263)

Add DH keygen to providers

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11332)

Add DSA keygen to provider

Moved some shared FFC code into the FFC files.
Added extra paramgen parameters for seed, gindex.
Fixed bug in ossl_prov util to print bignums.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11303)

Make sure we always send an alert in libssl if we hit a fatal error

We had a spot where a fatal error was occurring but we hadn't sent an
alert. This results in a later assertion failure.

Reviewed-by: Paul Yang <kaishen.yy@antfin.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/11537)

When calling EC_POINT_point2buf we must use a libctx

In a similar way to commit 76e23fc5 we must ensure that we use a libctx
whenever we call EC_POINT_point2buf because it can end up using crypto
algorithms.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11535)

Document the new libctx aware private key functions

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11494)

Make sure we use the libctx in libssl when loading PrivateKeys

Since loading a private key might require algorithm fetches we should
make sure the correct libctx is used.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11494)

Teach PEM_read_bio_PrivateKey about libctx

Now that d2i_PrivateKey_ex() and other similar functions exist we should
use it when loading a PEM PrivateKey.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11494)

Teach the OSSL_STORE code about libctx

We restrict this to just the PrivateKey loading code at the moment.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11494)

Use the libctx in Ed448 private key decoding

The Ed448 private key deconding needs to use a library ctx. So we
implement a priv_decode_with_libctx function for it.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11494)

Teach d2i_PrivateKey et al about libctx

The Ed448 private key decoding makes algorithm fetches. Therefore we teach
d2i_PrivateKey et al about libctx and make sure it is passed through the
layers.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11494)

Fix no-pic static builds

The cipher_tdes_common causes build failure as being duplicated
in libcrypto static builds.

[extended tests]

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11544)

EVP: fix memleak in evp_pkey_downgrade()

The EVP_KEYMGMT pointer in the pkey is removed when downgrading, but
wasn't necessarily freed when need, thus leaving an incorrect
reference count.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11328)

EVP: Add a temporary SM2 hack to key generation

The reason to do this is many-fold.  We need EC key generation for
other work.  However, SM2 are currently closely related to EC keys
with legacy methods, but not with provider methods.

To avoid having to wait on provider support for SM2, we temporarly
do an extra check for what the legacy methods identify as SM2 keys
(either the EVP_PKEY_SM2 pkey id was used, or the SM2 curve), and
redirect to legacy code in one case, and in the other case, we
forcedly downgrade provider side EC keys with SM2 curves to legacy
SM2 keys, using available tools.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11328)

EC: Refactor EVP_PKEY_CTX curve setting macros for param generation

The macros are converted to functions, and are modified to support
provider implementations.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11328)

PROV: Implement EC param / key generation

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11328)

KEYMGMT: Add functions to get param/key generation parameters

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11328)

Fix Dynamic engine loading so that the call to ENGINE_load_builtin_engines() is performed.

Fixes #11510

PR #11240 Added support for passing the libctx to the config loader.
As part of this work the call to OPENSSL_load_builtin_modules() + ENGINE_load_builtin_engines() was deferred until module_run() is called.
The call to ENGINE_load_builtin_engines() has been added to ENGINE_by_id().

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11543)

mkerr: remove legacy guards from generated error headers

In pull request #9333, legacy guards were added to the generated
error headers, but the mkerr.pl script was not adjusted accordingly.
So the legacy guards were removed by subsequent `make update` calls.

Fixing the mkerr.pl script properly was disproportionately complicated
by the fact that adding legacy guards only made sense for files which
already existed in version 1.1.1. To keep things simple, it was decided
to drop the legacy guards from the generated headers entirely.

Fixes #10569

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11541)

Add ex_data to EVP_PKEY.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11515)

BIO_do_accept: correct error return value

`BIO_do_accept` was returning incorrect values when unable to bind to a port.

Fixes #7717
CLA: trivial

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11505)

Fix some errors in documentation

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11476)

Initialize files that declare internal symbols

util/other-internal.syms is like util/other.syms, but for internal symbols.
Likewise, util/missingcrypto-internal.txt and util/missingssl-internal.txt
are like util/missingcrypto.txt and util/missingssl.txt

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11476)

Rework util/find-doc-nits to distinguish internal documentation

We didn't really distinguish internal and public documentation, or
matched that with the state of the documented symbols.  we therefore
needed to rework the logic to account for the state of each symbol.

To simplify things, and make them consistent, we load all of
util/*.num, util/*.syms and util/missing*.txt unconditionally.

Also, we rework the reading of the manuals to happen only once (or
well, not quite, Pod::Checker reads from file too, but at the very
least, our script isn't reading the same file multiple times).

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11476)

Add manpage entry for X509_check_purpose()

Fixes #10263

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10301)

Fix EVP_DigestSign interface when used with DES CMAC

DES implementations were missing the dup/copy ctx routines
required by CMAC implementation. A regression test is added.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11498)

EVP: legacy_ctrl_to_param() to handle provider side keys

There was one spot where this function would look at ctx->pmeth
directly to determine if it's for RSASSA-PSS, which fails when
presented with an EVP_PKEY_CTX holding a provider side key.
Switching to use EVP_PKEY_is_a() should make things better.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11501)

CMS KARI: Temporarly downgrade newly generated EVP_PKEYs to legacy

The EVP_PKEY_ASN1_METHOD code used by CMS_RecipientInfo_kari_decrypt()
and cms_RecipientInfo_kari_encrypt() is quite complex and needs more
careful thought to work with provider side keys.  Unfortunately, we
need to get key generation in place, among others for ECC keys, so we
add a temporary hack, similar to what's already done in TLS code, that
downgrades a provider side EVP_PKEY to become EVP_PKEY_ASN1_METHOD /
EVP_PKEY_METHOD based.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11501)

Don't compile commands if disabled

Rather than wrapping whole files in "ifndef OPENSSL_NO_xxx" we handle
the changes in build.info

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/11250)

Fix krb5 external test

Since commit c3845ceba84aab9ddeb43f043549238fd10de63b ("Build file
templates: don't set OPENSSL_{ENGINES,MODULES}") the krb5 external test
has been failing.  This is because it relied on OPENSSL_MODULES already
being set -- even though it did assign to OPENSSL_MODULES itself (and
thus got skipped by the cleanup pass in that commit), it was doing so
only to canonicalize the existing value to an absolute path, not as a de
novo assignment.

Catch up to the rest of the tree and just set it directly as the
"providers" path from the build top (but still canonicalized to an
absolute path).

[extended tests]

Fixes: 11492

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11499)

Add common internal crypto/ modules in liblegacy.a

Just as for the FIPS module, there's code in the legacy module that need
this.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11504)

Enable Ed25519 signing/verifying to use the libctx

Ed25519 needs to fetch a digest and so needs to use the correct libctx.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11496)

Add the auto generated der files to .gitignore

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11495)

Document the new library context aware CT functions

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11483)

Ensure libssl uses the new library context aware CT code

Ensure that when we create a CTLOG_STORE we use the new library context
aware function.

Also ensure that when we create a CT_POLICY_EVAL_CTX we associate it with
the library context.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11483)

Make the CT code library context aware

Add the new functions CTLOG_STORE_new_with_libctx(),
CTLOG_new_with_libctx() and CTLOG_new_from_base64_with_libctx() to pass
in the library context/property query string to use a library context
is to be used.

We also add the function CT_POLICY_EVAL_CTX_new_with_libctx() to enable
the creation of a CT_POLICY_EVAL_CTX to be associated with a libctx and
property query string.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11483)

Enable export_to functions to have access to the libctx

The EC export_to function calls EC_POINT_point2buf that can later
generate a random number in some circumstances. Therefore we pass in a
BN_CTX associated with the library context. This means we have to change
the export_to function signature to accept the library context.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11493)

OpenSSL::OID: Don't use List::Util

It turns out that the pairwise functions of List::Util came into perl
far later than 5.10.0.  We can't use that under those conditions, so
must revert to a quick internal implementation of the functions we're
after.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11503)

Update RAND_METHOD definition in man page

The `add` and `seed` callbacks were changed to return `int` instead of
`void` in b6dcdbfc94c482f6c15ba725754fc9e827e41851 (first included in
tag OpenSSL_1_1_0-pre1).

The `add` callback was changed to take a `double` instead of an `int`
in 853f757ecea74a271a7c5cdee3f3b5fe0d3ae863.

CLA: trivial
Fixes: #10199

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11486)

Travis build matrix adjustments

- do not exclude all clang builds on Linux
- exclude the constantly timeouting -fsanitize=address build on OS/X
- drop some mostly duplicate builds
- change the base linux distro to Bionic
- drop sudo as that is no longer needed - always on
- drop -D__NO_STRING_INLINES where not needed
- memleak test is not working with old clang

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11468)

Add a null provider which implements no algorithms.

By loading the null provider into the default context, it is possible
to verify that it is not accidentally being used.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11488)

Move legacy ciphers into the legacy provider

DES, idea, seed, rc2, rc4, rc5, cast and blowfish have been moved out of the default provider.
Code shared between desx and tdes has been moved into a seperate file (cipher_tdes_common.c).
3 test recipes failed due to using app/openssl calls that used legacy ciphers.
These calls have been updated to supply both the default and legacy providers.
Fixed openssl app '-provider' memory leak

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11419)

Document the new X509_STORE_CTX_new_with_libctx() function

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11457)

Explicitly cache the X509v3_extensions in one more place in libssl

Make sure we cache the extensions for a cert using the right libctx.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11457)

Use the libctx and propq from the X509_STORE_CTX

Now that X509_STORE_CTX contain a libctx we should use it in a couple of
places where we cache the X509v3 extensions.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11457)

Use X509_STORE_CTX_new_with_libctx() in libssl

Libssl is OPENSSL_CTX aware so we should use it when creating an
X509_STORE_CTX.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11457)

Add X509_STORE_CTX_new_with_libctx()

Make it possible to create an X509_STORE_CTX with an associated libctx
and propq.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11457)

TLS: Temporarly downgrade newly generated EVP_PKEYs to legacy

The transfer of TLS encodedpoint to backends isn't yet fully supported
in provider implementations.  This is a temporary measure so as not to
get stuck in other development.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11358)

Fix export of provided EC keys

The exporter freed a buffer too soon, and there were attempts to use
its data later, which was overwritten by something else at that
point.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11358)

EVP & TLS: Add necessary EC_KEY data extraction functions, and use them

libssl code uses EVP_PKEY_get0_EC_KEY() to extract certain basic data
from the EC_KEY.  We replace that with internal EVP_PKEY functions.

This may or may not be refactored later on.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11358)

EVP: add EVP_PKEY_is_a() and EVP_PKEY_can_sign()

EVP_PKEY_is_a() is the provider side key checking function corresponding
to checking EVP_PKEY_id() or an EVP_PKEY against macros like EVP_PKEY_EC.
It also works with legacy internal keys.

We also add a warning indoc/man3/EVP_PKEY_set1_RSA.pod regarding the
reliability of certain functions that only understand legacy keys.

Finally, we take the opportunity to clean up doc/man3/EVP_PKEY_set1_RSA.pod
to better conform with man-page layout norms, see man-pages(7) on Linux.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11358)

params: add a warning about the PTR types.

The warning is deter the unsure -- if in doubt the PTR type is almost certainly
NOT what you should be using.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11480)

AES CTR-DRGB: do not leak timing information

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11147)

Integer overflow in ASN1_STRING_set.

Addressing a potential integer overflow condition.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11473)

params: avoid a core dump with a null pointer and a get string call

Previous a get string (UTF8 or octet) params call would memcpy(2) from a NULL
pointer if the OSSL_PARAM didn't have its data field set.  This change makes
the operation fail rather than core dump and it returns to param size (if set).

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11474)

[crypto/ec] blind coordinates in ec_wNAF_mul for robustness

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Nicola Tuveri <nicola.tuveri@ibm.com>
(Merged from https://github.com/openssl/openssl/pull/11439)

Fix the error handling in EC_POINTs_mul

This was pointed out by a false-positive
-fsanitizer warning ;-)

However from the cryptographical POV the
code is wrong:
A point R^0 on the wrong curve
is infinity on the wrong curve.

[extended tests]

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11475)

Fix misleading error msg for PBM check w/o secret in OSSL_CMP_validate_msg()

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/11448)

Fix error reporting glitch in X509_STORE_CTX_print_verify_cb() in t_x509.c

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/11448)

Fix bugs in 3GPP exception checking and improve diagnostics in crypt/cmp/cmp_vfy.c

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/11448)

PROV: Add DERlib support for ECDSA and EC keys

This replaces crypto/ec/ecdsa_aid.c with new code and generated OIDs

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11450)

PROV: Add DERlib support for DSA

This replaces crypto/dsa/dsa_aid.c with new code and generated OIDs

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11450)

PROV: Add DERlib support for RSA

This replaces crypto/rsa/rsa_aid.c with new code and generated OIDs

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11450)

PROV: Add the beginning of a DER writing library

This library is meant to be small and quick.  It's based on WPACKET,
which was extended to support DER writing.  The way it's used is a
bit unusual, as it's used to write the structures backward into a
given buffer.  A typical quick call looks like this:

    /*
     * Fill in this structure:
     *
     * something ::= SEQUENCE {
     *     id OBJECT IDENTIFIER,
     *     x [0] INTEGER OPTIONAL,
     *     y [1] BOOLEAN OPTIONAL,
     *     n INTEGER
     * }
     */
    unsigned char buf[nnnn], *p = NULL;
    size_t encoded_len = 0;
    WPACKET pkt;
    int ok;

    ok =   WPACKET_init_der(&pkt, buf, sizeof(buf)
        && DER_w_start_sequence(&pkt, -1)
        && DER_w_bn(&pkt, -1, bn)
        && DER_w_boolean(&pkt, 1, bool)
        && DER_w_precompiled(&pkt, -1, OID, sizeof(OID))
        && DER_w_end_sequence(&pkt, -1)
        && WPACKET_finish(&pkt)
        && WPACKET_get_total_written(&pkt, &encoded_len)
        && (p = WPACKET_get_curr(&pkt)) != NULL;

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11450)

Add perl support to parse and DER encode ASN.1 OID specs

We have an old OID database that's not as readable as would be
desired, and we have spots with hand coded DER for well known OIDs.

The perl modules added here give enough support that we can parse
OBJECT IDENTIFIER definitions and encode them as DER.

OpenSSL::OID is a general OID parsing and encoding of ASN.1
definitions, and supports enough of the X.680 syntax to understand
what we find in RFCs and similar documents and produce the DER
encoding for them.

oids_to_c is a specialized module to convert the DER encoding from
OpenSSL::OID to C code.  This is primarily useful in file templates
that are processed with util/dofile.pl.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11450)

Add test cases for the non CA certificate with pathlen:0

Accept verification without -x509_strict and reject it with it.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11463)

Set X509_V_ERR_INVALID_EXTENSION error for invalid basic constraints

If we encounter certificate with basic constraints CA:false,
pathlen present and X509_V_FLAG_X509_STRICT is set we set
X509_V_ERR_INVALID_EXTENSION error.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11463)

Allow certificates with Basic Constraints CA:false, pathlen:0

Do not mark such certificates with EXFLAG_INVALID although they
violate the RFC 5280, they are syntactically correct and
openssl itself can produce such certificates without any errors
with command such as:

openssl x509 -req -signkey private.pem -in csr.pem -out cert.pem \
  -extfile <(echo "basicConstraints=CA:FALSE,pathlen:0")

With the commit ba4356ae4002a04e28642da60c551877eea804f7 the
EXFLAG_INVALID causes openssl to not consider such certificate
even as leaf self-signed certificate which is breaking existing
installations.

Fixes: #11456

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11463)

Add a WPACKET test for the new DER capability

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11462)

Add "endfirst" writing to WPACKET

Support the concept of writing to the end of the packet first.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11462)

Fix a gcc warning about possible null pointer

In function 'ccm_tls_cipher',
    inlined from 'ccm_cipher_internal' at providers/common/ciphers/cipher_ccm.c:359:16,
    inlined from 'ccm_stream_final' at providers/common/ciphers/cipher_ccm.c:265:9:
providers/common/ciphers/cipher_ccm.c:317:5: error: argument 2 null where non-null expected [-Werror=nonnull]
  317 |     memcpy(ctx->iv + EVP_CCM_TLS_FIXED_IV_LEN, in, EVP_CCM_TLS_EXPLICIT_IV_LEN);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from include/internal/cryptlib.h:14,
                 from providers/common/include/prov/ciphercommon.h:14,
                 from providers/common/ciphers/cipher_ccm.c:12:
providers/common/ciphers/cipher_ccm.c: In function 'ccm_stream_final':
/home/ed/gnu/arm-linux-gnueabihf-linux64/arm-linux-gnueabihf/sys-include/string.h:44:14: note: in a call to function 'memcpy' declared here
   44 | extern void *memcpy (void *__restrict __dest,
      |              ^~~~~~

[extended tests]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10344)

Expand the XTS documentation

Explain that XTS does not support streaming, and that the IV value is the
tweak.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11461)

Remove an unnecessary call to BN_CTX_free.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11452)

Fix bugs in EC code introduced with FIPS changes.

a9612d6c034f47c4788c67d85651d0cd58c3faf7 introduced possible memory leaks in EC_GROUP_cmp and EC_POINTs_mul, and a possible BN_CTX_end without BN_CTX_start in ec_field_inverse_mod_ord.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11452)

Add data driven SELF TEST code for signatures and key agreement

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11036)

HTTP client: make server/proxy and port params more consistent; minor other improvements

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/11404)

Chunk 10 of CMP contribution to OpenSSL: CMP http client and related tests

    Also improve the generic HTTP client w.r.t. proxy and no_proxy options.

    Certificate Management Protocol (CMP, RFC 4210) extension to OpenSSL
    Also includes CRMF (RFC 4211) and HTTP transfer (RFC 6712).
    Adds the CMP and CRMF API to libcrypto and the "cmp" app to the CLI.
    Adds extensive documentation and tests.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/11404)

TLS Cipher Suite 0xC102 Support

For GOST2012-GOST8912-GOST8912 was used 0xFF85 identifier,
but new identifier 0xc102 was assigned.
Because of old software we will support both numbers.

https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-2
https://datatracker.ietf.org/doc/draft-smyshlyaev-tls12-gost-suites/

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11403)

Fix a printf format error in cmp_client.c

The value is of type uint64 but the format
%ld is not suitable for that, need to use %jd.

[extended tests]

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11454)

Fixups in CHANGES.md

Fixed one ambiguous line and removed a few unnecessary backticks.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11447)

Extend the sslprovider_test to be able to additionally test FIPS

Previously we could test an empty default ctx, with the default provider
loaded into another ctx. Now we do the same with the FIPS provider.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/11401)

Make sure we always use the correct libctx in sslprovidertest.c

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/11401)

Make sure we use the libctx when creating an EVP_PKEY_CTX in libssl

We should use EVP_PKEY_CTX_new_from_pkey() to ensure we use the correct
libctx.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/11401)

Use EVP_DigestSignInit_ex and EVP_DigestVerifyInit_ex in libssl

We need to make sure we use the correct libctx for all operations in
libssl.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/11401)

Don't double free a DH object

Having created a DH object and assigned it to an EVP_PKEY - we should
not free both the EVP_PKEY and the original DH. This will lead to a
double free occurring.

This issue was discovered and reported by GitHub Security Lab team member
Agustin Gianni.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11441)

[crypto/ec] Ladder tweaks

- Convert to affine coords on ladder entry. This lets us use more efficient
  ladder step formulae.

- Convert to affine coords on ladder exit. This prevents the current code
  awkwardness where conversion happens twice during serialization: first to
  fetch the buffer size, then again to fetch the coords.

- Instead of projectively blinding the input point, blind both accumulators
  independently.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/11435)

[test] Make sm2_internal_test less fragile to changes in the ec module

Since these are KATs, the trailing randomness consumed by the ec module
does not really matter. So make the fake random buffer circular.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/11435)

apps: support sendfile in s_server when ktls enabled

When the -WWW or -HTTP option is specified, s_server can choose
to use SSL_sendfile to transmit the file requested by client
with KTLS is enabled, taking full advantage of the performance
advantages of Kernel TLS, and adding the '-sendfile' command
line parameter to control this behavior.

Signed-off-by: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
Reviewed-by: Paul Yang <kaishen.yy@antfin.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11318)

Add EVP_PKEY_gettable_params support for accessing EVP_PKEY key data fields

Currently only RSA, EC and ECX are supported (DH and DSA need to be added to the keygen
PR's seperately because the fields supported have changed significantly).

The API's require the keys to be provider based.

Made the keymanagement export and get_params functions share the same code by supplying
support functions that work for both a OSSL_PARAM_BLD as well as a OSSL_PARAM[].
This approach means that complex code is not required to build an
empty OSSL_PARAM[] with the correct sized fields before then doing a second
pass to populate the array.

The RSA factor arrays have been changed to use unique key names to simplify the interface
needed by the user.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11365)

Windows makefile generator: Don't delete long lists of files in one go

The Windows command line has its limits, and we're hitting it hard.
We therefore generate one 'del' command for each explicit file for the
'clean' target.

Fixes #11163

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/11171)

Give a better error if we can't find a sig alg

Some scenarios where we could not find a suitable sig alg just
gave "internal error" as the reason - which isn't very helpful. A
more suitable reason code already exists - so we use that.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11425)

Fix off-by-1 bug on provider_activate with custom error strings

Starting `cnt` from 1 would work if we weren't using cnt itself to
access elements of the array returned calling the provider callback.

As it is before this commit, we have 2 problems:
- first, in the unlikely case that the incoming array was "empty" (only
  contains the terminator item) we would skip past it and potentially
  end up with oob reads;
- otherwise, at the end of the while loop, `cnt` will be equal to the
  number of items in the input array, not 1 more. We then add 1 more to
  the zalloc call to account for the library name item, and we fill all
  of it (relying on zalloc to have zeroed the terminator item).
  The first read access that will read the list up to the terminator
  will result in a OOB read as we did not allocate enough space to also
  contain the terminator.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11427)

Add test for providers exposing OSSL_FUNC_PROVIDER_GET_REASON_STRINGS

This test currently fails, next commit has the description of the bug
and the fix.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11427)

Document the new X509v3_cache_extensions() function

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11409)