doc: revamp the README file

 * Add an OpenSSL logo and CI badges
 * Add a table of contents
 * Add a lot of links

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10545)

doc: add OpenSSL logo

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10545)

doc: introduce some minimalistic markdown without essential changes

The goal is to transform the standard documents

    README, INSTALL, SUPPORT, CONTRIBUTING, ...

from a pure text format into markdown format, but in such a way
that the documentation remains nicely formatted an easy readable
when viewed with an normal text editor.

To achieve this goal, we use a special form of 'minimalistic' markdown
which interferes as little as possible with the reading flow.

 * avoid [ATX headings][] and use [setext headings][] instead
   (works for `<h1>` and `<h2>` headings only).
 * avoid [inline links][] and use [reference links][] instead.
 * avoid [fenced code blocks][], use [indented-code-blocks][] instead.

The transformation will take place in several steps. This commit
introduces mostly changes the formatting and does not chang the
content significantly.

[ATX headings]:         https://github.github.com/gfm/#atx-headings
[setext headings]:      https://github.github.com/gfm/#setext-headings
[inline links]:         https://github.github.com/gfm/#inline-link
[reference links]:      https://github.github.com/gfm/#reference-link
[fenced code blocks]:   https://github.github.com/gfm/#fenced-code-blocks
[indented code blocks]: https://github.github.com/gfm/#indented-code-blocks

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10545)

doc: convert standard project docs to markdown

In the first step, we just add the .md extension and move some
files around, without changing any content. These changes will
occur in the following commits.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10545)

Use a wrapper for pod2html

Remove unused util/process_docs.pl

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10856)

apps x509: restrict CAkeyform option to OPT_FMT_PDE

CAkeyform may be set to PEM, DER or ENGINE, but the current options
are not using the proper optionformat 'E' (OPT_FMT_PDE) for this.

Set the valtype for CAkeyform to 'E' and use OPT_FMT_PDE when extracting
the option value.

This amends 0ab6fc79a9a ("Fix regression on x509 keyform argument") which
did the same thing for keyform and changed the manpage synopsis entries
for both keyform and CAkeyform but did not change the option section.
Hence, change the option section for both of them.

CLA: trivial

Co-developed-by: Torben Hohn <torben.hohn@linutronix.de>
Signed-off-by: Torben Hohn <torben.hohn@linutronix.de>
Signed-off-by: Bastian Germann <bage@linutronix.de>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11085)

bugfix in cmac calculation example

The example never executes code inside of the while loop, as read()
returns bigger number than 0. Thus the end result is wrong.

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11143)

Avoid arm64 builds timeout due to silent make taking too long time

Also reuse one of the arm64 builds as a no-deprecated build
Also include a single ppc64le-build

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11181)

secmem: ignore small minsize arguments to CRYPTO_secure_malloc_init().

If the user specifies a minimum allocation size that is smaller than
the free list structure (or zero), calculate the minimum possible size rather
than failing.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11149)

sec_mem: add note about the minimum size parameter.

Add a note indicating that the minimum size parameter to
CRYPTO_secure_malloc_init() should be small.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11149)

Mem-sec small code adjustment

Conditional code readability improvement.

Remove unused macro

Commit #11042 has introduced a new, unused, CRYPTO_EX_INDEX macro.
Remove before version release.

Reviewed-by: Paul Yang <kaishen.yy@antfin.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11103)

x86_64: Replace .asciz "GNU" with .byte

Replace .asciz "GNU" with .byte since .asciz isn't supported on Solaris.
Fixes https://github.com/openssl/openssl/issues/11132

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11137)

Check that the DRBG's internal state has been zeroized after uninstantiation

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11111)

DRBG: delay initialization of DRBG method until instantiation

Previously, the initialization was done immediately in RAND_DRBG_set(),
which is also called in RAND_DRBG_uninstantiate().

This made it difficult for the FIPS DRBG self test to verify that the
internal state had been zeroized, because it had the side effect that
the drbg->data structure was reinitialized immediately.

To solve the problem, RAND_DRBG_set() has been split in two parts

    static int rand_drbg_set(RAND_DRBG *drbg, int type, unsigned int flags);
    static int rand_drbg_init_method(RAND_DRBG *drbg);

and only the first part is called from RAND_DRBG_uninstantiate().

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11111)

config: Drop linux-alpha-gcc+bwx

Its entry in Configuration/10-main.conf was dropped in commit
7ead0c89185c ("Configure: fold related configurations more aggressively
and clean-up.") probably because all but one of its bn_ops were removed
(RC4_CHAR remained). Benchmarks on an Alpha EV7 indicate that RC4_INT is
better than RC4_CHAR so rather than restoring the configuation, remove
it from config.

CLA: trivial
Bug: https://bugs.gentoo.org/697840

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11130)

Fix default provider merge glitch

Property "default" no longer exists, replace "default=yes" with
"provider=default"

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11150)

Refactor evp_pkey_make_provided() to do legacy to provider export

Previously, evp-keymgmt_util_export_to_provider() took care of all
kinds of exports of EVP_PKEYs to provider side keys, be it from its
legacy key or from another provider side key.  This works most of the
times, but there may be cases where the caller wants to be a bit more
in control of what sort of export happens when.

Also, when it's time to remove all legacy stuff, that job will be much
easier if we have a better separation between legacy support and
support of provided stuff, as far as we can take it.

This changes moves the support of legacy key to provider side key
export from evp-keymgmt_util_export_to_provider() to
evp_pkey_make_provided(), and makes sure the latter is called from all
EVP_PKEY functions that handle legacy stuff.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11074)

DSA: More conforming names in crypto/dsa/dsa_aid.c

Made macro names that refer to a known base OID, an commented accordingly.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10557)

doc/man3/EVP_PKEY_CTX_ctrl.pod: cleanup

Clean up a manual we've touched, according to conventions found in
Linux' man-pages(7); function arguments in descriptions should be in
italics, and types, macros and similar should be in bold, with the
exception for NULL.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10557)

PROV: Avoid MDC2 in the RSA signature implementation in the FIPS module

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10557)

test/recipes/30-test_evp_data/evppkey.txt

Tests that go through provider cannot recognise PKEY_CTRL_INVALID from
PKEY_CTRL_ERROR any more, because provided implementations' param
setting functions return 0 or 1.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10557)

test/evp_extra_test.c: adapt for RSA signature tests

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10557)

rsa_get0_all_params(): Allow zero CRT params

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10557)

PROV: add RSA signature implementation

This includes legacy PSS controls to params conversion, and an attempt
to generalise the parameter names when they are suitable for more than
one operation.

Also added crypto/rsa/rsa_aid.c, containing proper AlgorithmIdentifiers
for known RSA+hash function combinations.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10557)

Don't exclude quite so much in a no-sock build

We were excluding more code than we needed to in the OCSP/HTTP code in
the event of no-sock. We should also not assume that a BIO passed to our
API is socket based.

This fixes the no-sock build

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/11134)

Introduce the provider property

Replace the properties default, fips and legacy with a single property
called "provider". So, for example, instead of writing "default=yes" to
get algorithms from the default provider you would instead write
"provider=default". We also have a new "fips" property to indicate that
an algorithm is compatible with FIPS mode. This applies to all the
algorithms in the FIPS provider, as well as any non-cryptographic
algorithms (currently only serializers).

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11097)

Add DRBG self tests

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/11010)

pkey: additional EC related options

Add options to change the parameter encoding and point conversions for EC
public and private keys.  These options are present in the deprecated 'ec'
utility.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11113)

pkey: update command line tool examples in light of deprecations.

Specifically, refer from the deprecated tools to the pkey equivalents.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11113)

apps: distinguish between a parameter error and an unknown parameter.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11049)

pmeth_lib: detect unsupported OSSL_PARAM.

When converting legacy controls to OSSL_PARAMs, return the unsupported -2
value correctly.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11049)

Params: add argument to the _from_text calls to indicate if the param exists.

The extra argument is a integer pointer and is optional.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11049)

Add *.d.tmp files to .gitignore

These are temporary files generated by the build process that should not
be checked in.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11122)

Deprecate the low level Diffie-Hellman functions.

Use of the low level DH functions has been informally discouraged for a
long time.  We now formally deprecate them.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11024)

DH: add CHANGES entry listing the deprecated DH functions.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11024)

DH: fix header file indentation

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11024)

Deprecate the low level RSA functions.

Use of the low level RSA functions has been informally discouraged for a
long time. We now formally deprecate them.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11063)

rsa: document deprecated RSA command line apps

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11063)

rsa: document deprecated low level functions

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11063)

NEWS: DH, DSA, ECDH, ECDSA and RSA public key function deprecation note

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11063)

rsa.h: fix preprocessor indentation

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11063)

Remove unneeded switch statement to fix warning

https://github.com/openssl/openssl/issues/10958

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11125)

Handle max_fragment_length overflow for DTLS

Allow for encryption overhead in early DTLS size check
and send overflow if validated record is too long

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11096)

[PROV][EC] Update documentation

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10631)

[BN] harden `BN_copy()` against leaks from memory accesses

`BN_copy()` (and indirectly `BN_dup()`) do not propagate the
`BN_FLG_CONSTTIME` flag: the propagation has been turned on and off a
few times in the past years, because in some conditions it has shown
unintended consequences in some code paths.

Without turning the propagation on once more, we can still improve
`BN_copy()` by avoiding to leak `src->top` in case `src` is flagged with
`BN_FLG_CONSTTIME`.
In this case we can instead use `src->dmax` as the number of words
allocated for `dst` and for the `memcpy` operation.

Barring compiler or runtime optimizations, if the caller provides `src`
flagged as const time and preallocated to a public size, no leak should
happen due to the copy operation.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10631)

[EC] harden EC_KEY against leaks from memory accesses

We should never leak the bit length of the secret scalar in the key,
so we always set the `BN_FLG_CONSTTIME` flag on the internal `BIGNUM`
holding the secret scalar.

This is important also because `BN_dup()` (and `BN_copy()`) do not
propagate the `BN_FLG_CONSTTIME` flag from the source `BIGNUM`, and
this brings an extra risk of inadvertently losing the flag, even when
the called specifically set it.

The propagation has been turned on and off a few times in the past
years because in some conditions has shown unintended consequences in
some code paths, so at the moment we can't fix this in the BN layer.

In `EC_KEY_set_private_key()` we can work around the propagation by
manually setting the flag after `BN_dup()` as we know for sure that
inside the EC module the `BN_FLG_CONSTTIME` is always treated
correctly and should not generate unintended consequences.

Setting the `BN_FLG_CONSTTIME` flag alone is never enough, we also have
to preallocate the `BIGNUM` internal buffer to a fixed public size big
enough that operations performed during the processing never trigger
a realloc which would leak the size of the scalar through memory
accesses.

Fixed Length
------------

The order of the large prime subgroup of the curve is our choice for
a fixed public size, as that is generally the upper bound for
generating a private key in EC cryptosystems and should fit all valid
secret scalars.

For preallocating the `BIGNUM` storage we look at the number of "words"
required for the internal representation of the order, and we
preallocate 2 extra "words" in case any of the subsequent processing
might temporarily overflow the order length.

Future work
-----------

A separate commit addresses further hardening of `BN_copy()` (and
indirectly `BN_dup()`).

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10631)

[PROV][KEYMGMT][EC] Import/export of priv_key as padded const time BN

For EC keys it is particularly important to avoid leaking the bit length
of the secret scalar.

Key import/export should never leak the bit length of the secret
scalar in the key.

For this reason, on export we use padded BIGNUMs with fixed length,
using the new `ossl_param_bld_push_BN_pad()`.

When importing we also should make sure that, even if short lived,
the newly created BIGNUM is marked with the BN_FLG_CONSTTIME flag as
soon as possible, so that any processing of this BIGNUM might opt for
constant time implementations in the backend.

Setting the BN_FLG_CONSTTIME flag alone is never enough, we also have
to preallocate the BIGNUM internal buffer to a fixed size big enough
that operations performed during the processing never trigger a
realloc which would leak the size of the scalar through memory
accesses.

Fixed length
------------

The order of the large prime subgroup of the curve is our choice for
a fixed public size, as that is generally the upper bound for
generating a private key in EC cryptosystems and should fit all valid
secret scalars.

For padding on export we just use the bit length of the order
converted to bytes (rounding up).

For preallocating the BIGNUM storage we look at the number of "words"
required for the internal representation of the order, and we
preallocate 2 extra "words" in case any of the subsequent processing
might temporarily overflow the order length.

Future work
-----------

To ensure the flag and fixed size preallocation persists upon
`EC_KEY_set_private_key()`, we need to further harden
`EC_KEY_set_private_key()` and `BN_copy()`.
This is done in separate commits.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10631)

[PROV][KMGMT][KEXCH][EC] Implement EC keymgtm and ECDH

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10631)

[CMS] Test decryption of a ciphertext encrypted from 1.1.1

Current CMS en/decryption tests only validate that our current decyption
and encryption algorithms are compatible, but they say nothing about
correctness of the output for the given set of parameters.

As a partial fix in absence of proper KAT tests, we decrypt ciphertexts
generated with OpenSSL 1.1.1.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10631)

[EC] Constify internal EC_KEY pointer usage

A pair of internal functions related to EC_KEY handling could benefit
from declaring `EC_KEY *` variables as `const`, providing clarity for
callers and readers of the code, in addition to enlisting the compiler
in preventing some mistakes.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10631)

[PROV][KEYMGMT][DH][DSA] use BN_clear_free for secrets

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10631)

x86_64: Don't assume 8-byte pointer size

Since pointer in x32 is 4 bytes, add x86_64-support.pl to define
pointer_size and pointer_register based on flavour to support
stuctures like:

struct {  void *ptr; int blocks;  }

This fixes 90-test_sslapi.t on x32.  Verified with

$ ./Configure shared linux-x86_64
$ make
$ make test

and

$ ./Configure shared linux-x32
$ make
$ make test

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10988)

DOC:Fix typos in x509v3_config.pod

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11104)

DOC:Fix typos in man5/config.pod

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11104)

DOC:Fix typos in openssl-enc.pod.in + openssl.pod

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11104)

DOC:Fix typos in ossl_cmp_print_log.pod

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11104)

DOC:Fix typos in ossl_param_bld_init

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11104)

Deprecate EC command line apps.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11106)

TEST: Optionally silence OpenSSL::Test::setup()

test/generate_ssl_tests.pl uses OpenSSL::Test to get to some of its
practical location functions.  A recent note in the setup() code made
its result not quite match the original (we do check that), so there's
a need to silence setup(), which we do with a simple optional argument.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/tconversion.pl to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/80-test_ssl_old.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/80-test_ssl_new.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/80-test_ocsp.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/80-test_cms.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/80-test_ca.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/20-test_pkeyutl.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/20-test_enc_more.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/25-test_x509.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/25-test_req.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/25-test_crl.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/20-test_enc.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/20-test_dgst.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/15-test_rsapss.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Modify test/recipes/15-test_mp_rsa.t to leave artifacts behind

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

TEST: Create test specific output directories

We had all tests run with test/test-runs/ as working directory, and
tests cleaned up after themselves...  which is well and good, until
you want to have a look at what went wrong when a complex test fails,
and you have to recreate everything it does manually.

To remedy this, we have OpenSSL::Test create the result directory
dynamically (and cleaning it up first if it's already there) and let
the test recipe have that as working directory.

Test recipes are now encouraged to name their diverse output files
uniquely, and not to clean them up, to allow a developer to have a
look at the files that were produced.

With continuous integration that allows this, the result directories
could also be archived and be left as a build artifact.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

DOC: Add documentation related to X509_LOOKUPs

Most of all, the base X509_LOOKUP functionality is now documented.
Furthermore, the names X509_LOOKUP_METHOD and X509_STORE are added for
reference.

Some functions were moved from X509_LOOKUP_meth_new.pod

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/10986)

Fix podchecker warnings

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11045)

Make util/find-doc-nits runnable from the build tree

Because we generate an increasing number of POD files, some of them
end up in the build tree.  This makes it difficult for find-doc-nits
to work as desired when the build tree is separate from the source
tree.

The best supported way to make it work in such an environment is to
run it from the build tree and let it use the build information from
configdata.pm to find all the POD files.  To make this smooth enough,
we add a function 'files' that returns an array of file names
corresponding to criteria from the caller.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11045)

ffc: use sizeof(*pointer) instead of sizeof(struct) in memset(3) call.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11101)

Coverity 1458438: fix uninitialised memory access.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11101)

Coverity 1458439: fix resource leak

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11101)

chunk 7 of CMP contribution to OpenSSL

add CMP message validation and related tests; while doing so:
* add ERR_add_error_mem_bio() to crypto/err/err_prn.c
* move ossl_cmp_add_error_txt() as ERR_add_error_txt() to crypto/err/err_prn.c
* add X509_STORE_CTX_print_verify_cb() to crypto/x509/t_x509.c,
  adding internally x509_print_ex_brief(), print_certs(), and print_store_certs()
* move {ossl_cmp_,}X509_STORE_get1_certs() to crypto/x509/x509_lu.c

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/10620)

fix various formatting nits in CMP contribution chunks 1-6 found by the new util/check-format.pl

in addition:
correct wording in doc, comments, and parameter names: self-signed -> self-issued where appropriate

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/10620)

improve CMP logging according to comments on CMP chunk 7 preview

in particular:
consolidate documentation of CMP logging and error reporting functions
fix compilation problem with clang on some platforms
rename OSSL_CMP_log etc. to ossl_cmp_log etc. since these macros are CMP-internal
move chopping of trailing separator to ossl_cmp_add_error_txt(), also fix handling of leading separator
internalize X509_print_ex_brief() as x509_print_ex_brief()

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/10620)

add internal doc files actually belonging to CMP contribution chunk 6

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/10620)

Also check for errors in x86_64-xlate.pl.

In https://github.com/openssl/openssl/pull/10883, I'd meant to exclude
the perlasm drivers since they aren't opening pipes and do not
particularly need it, but I only noticed x86_64-xlate.pl, so
arm-xlate.pl and ppc-xlate.pl got the change.

That seems to have been fine, so be consistent and also apply the change
to x86_64-xlate.pl. Checking for errors is generally a good idea.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: David Benjamin <davidben@google.com>
(Merged from https://github.com/openssl/openssl/pull/10930)

Add $debug variable and use it

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10852)

Fix a bug for aarch64 BigEndian
FIXES #10692 #10638
a bug for aarch64 bigendian with instructions 'st1' and 'ld1' on AES-GCM mode.

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10751)

crypto/bn: fix debug build

Fixes:

    gcc  -I. -Iinclude -Iproviders/common/include -Iproviders/implementations/include -Icrypto/include  -DSTATIC_LEGACY -DBN_DEBUG -DREF_DEBUG     -DCONF_DEBUG -DBN_CTX_DEBUG -DOPENSSL_NO_ASM -ggdb -g2 -Wformat -Wshadow -Wmissing-prototypes -Wmissing-declarations -Werror      -DOPENSSL_BUILDING_OPENSSL -DOPENSSLDIR="\"/home/wsh/opt/openssl/ssl\"" -DENGINESDIR="\"/home/wsh/opt/openssl/lib/engines-3\""     -DMODULESDIR="\"/home/wsh/opt/openssl/lib/ossl-modules\""   -MMD -MF crypto/bn/libcrypto-lib-bn_conv.d.tmp -MT crypto/bn/libcrypto-lib-bn_conv.o -c -o crypto/bn/libcrypto-lib-bn_conv.o crypto/bn/bn_conv.c
    In file included from crypto/bn/bn_conv.c:12:
    crypto/bn/bn_conv.c: In function 'BN_hex2bn':
    crypto/bn/bn_local.h:178:31: error: implicit declaration of function 'ossl_assert'; did you mean 'ossl_ssize_t'? [-Werror=implicit-function-declaration]
                             (void)ossl_assert((_top == 0 && !_bnum2->neg) || \
                                   ^~~~~~~~~~~
    crypto/bn/bn_conv.c:188:5: note: in expansion of macro 'bn_check_top'
         bn_check_top(ret);
         ^~~~~~~~~~~~

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10993)

Add FFC param/key validation

Embed libctx in dsa and dh objects and cleanup internal methods to not pass libctx (This makes it consistent with the rsa changes)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10910)

x86_64: Add endbranch at function entries for Intel CET

To support Intel CET, all indirect branch targets must start with
endbranch.  Here is a patch to add endbranch to function entries
in x86_64 assembly codes which are indirect branch targets as
discovered by running openssl testsuite on Intel CET machine and
visual inspection.

Verified with

$ CC="gcc -Wl,-z,cet-report=error" ./Configure shared linux-x86_64 -fcf-protection
$ make
$ make test

and

$ CC="gcc -mx32 -Wl,-z,cet-report=error" ./Configure shared linux-x32 -fcf-protection
$ make
$ make test # <<< passed with https://github.com/openssl/openssl/pull/10988

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10982)

include/openssl/whrlpool.h: correct unbalanced deprecation guards

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11087)

DOC: document in more detail what a BIO_read_ex() via BIO_f_buffer() does

The BIO_f_buffer() documentation tells in enough detail how it affects
BIO_gets(), but not how it affects BIO_read_ex().  This change
remedies that.

Fixes #10859

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10890)

Document OSSL_SERIALIZER_PUBKEY_TO_DER_PQ and friends

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11071)

PROV: Build the main FIPS module code with FIPS_MODE defined

Without that, its main source wasn't compiled correctly.

Fixes #11089

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/11090)

apps x509: passing PKCS#11 URL as -signkey

OpenSSL 1.1.0 has extended option checking, and rejects passing a PKCS#11
engine URL to "-signkey" option. The actual code is ready to take it.

Change the option parsing to allow an engine URL to be passed and modify
the manpage accordingly.

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11086)

Make secure-memory be a config option

Adding support for "no-secure-memory" was simple, a one-liner.
Fixing all the "ifdef OPENSSL_SECURE_MEMORY" to be "ifndef NO_xxx"
was a bit more work.  My original goof, for not following the OpenSSL
pattern "ifndef NO_" used everywhere else.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/11023)

Adapt i2d_PUBKEY for provider only keys

It uses EVP_PKEY serializers to get the desired results.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11078)

Adapt i2d_PrivateKey for provider only keys

It uses EVP_PKEY serializers to get the desired results.

One might think that ddoing this might make things a bit dicy for
existing serializers, as they should obviously use i2d functions.
However, since our serializers use much more primitive functions
such as i2d_ASN1_INTEGER(), or keytype specific ones such as
i2d_RSAPrivateKey(), there is no clash.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11056)

Forgotten GOST2012 support in non-vital places

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/11072)