Smarter build of system error text database

We stored copies of the system error texts in a fixed line size array,
which is a huge waste.  Instead, use a static memory pool and pack all
the string in there.  The wasted space at the end, if any, gives us
some leeway for longer strings than we have measured so far.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7681)

openssl s_server: don't use sendto() with connected UDP socket

Fixes #7675

On macOS, if you call `connect()` on a UDP socket you cannot then
call `sendto()` with a destination, otherwise it fails with Err#56
('socket is already connected').

By calling `BIO_ctrl_set_connected()` on the wbio we can tell it
that the socket has been connected and make it call `send()` rather
than `sendto()`.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7676)

rsa/rsa_ossl.c: cache MONT_CTX for public modulus earlier.

Blinding is performed more efficiently and securely if MONT_CTX for public
modulus is available by the time blinding parameter are instantiated. So
make sure it's the case.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7586)

Travis CI: Use flake8 to find Python syntax errors or undefined names

CLA: trivial

In Travis CI, add a Python linting step that runs flake8 tests in Travis CI
to find syntax errors and undefined names. (http://flake8.pycqa.org)

__E901,E999,F821,F822,F823__ are the "_showstopper_" flake8 issues that can halt
the runtime with a SyntaxError, NameError, etc. Most other flake8 issues are
merely "style violations" -- useful for readability but they do not effect
runtime safety.

* F821: undefined name `name`
* F822: undefined name `name` in `__all__`
* F823: local variable name referenced before assignment
* E901: SyntaxError or IndentationError
* E999: SyntaxError -- failed to compile a file into an Abstract Syntax Tree

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7410)

Update copyright year

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7668)

Update CHANGES and NEWS for new release

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/7663)

Unix build: for mingw and cygwin, create the right location for DLLs

Mingw and Cygwin builds install the DLLs in the application directory,
not the library directory, so ensure that one is created for them when
installing the DLLs.

Fixes #7653

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7654)

Add documentation for -pkeyopt_passin

Add documentation to new parameter and two examples showcasing scrypt
KDF.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5697)

Add option to read pkeyopts interactively

This patch adds the ability to interactively enter passphrases for
the pkeyutl application. For example, you could use

$ openssl pkeyutl -kdf TLS1-PRF -kdflen 8 -pkeyopt md:md5
  -pkeyopt_passin secret -pkeyopt_passin seed

To have the "secret" and "seed" values read interactively from keyboard
(with hidden input). Alternatively, the pass phrase argument syntax is
also supported, e.g.:

$ openssl pkeyutl -kdf TLS1-PRF -kdflen 8 -pkeyopt md:md5
  -pkeyopt_passin secret:stdin -pkeyopt_passin seed:env:SEEDVAR

To have "secret" read from stdin and "seed" from the environment
variable SEEDVAR.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5697)

sha/asm/sha512p8-ppc.pl: optimize epilogue.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7643)

sha/asm/sha512p8-ppc.pl: fix typo in prologue.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7643)

Configuration: only include shared_sources in dirinfo in shared config

Without this precaution, we end up having directory targets depend on
shlib object files for which there are no rules.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7644)

test/siphash_internal_test.c: ensure the SIPHASH structure is zeroed

Fixes #7641

[extended tests]

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7642)

Add a missing SSLfatal call

A missing SSLfatal call can result in an assertion failed error if the
condition gets triggered.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7594)

Deprecate SSL_set_tmp_ecdh

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7522)

Making SRP_user_pwd functions public

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7522)

Added SRP_VBASE_add0_user()

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7522)

SRP module documentation

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7522)

Add SSL_CTX_set_tmp_ecdh.pod

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7522)

SSL extra chain certificates doc

Signed-off-by: Antoine Salon <asalon@vmware.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7522)

Fix no-ec and no-tls1_2

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7620)

Fix typo in util/perl/OpenSSL/Test.pm

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7633)

test/recipes/90-test_shlibload.t needs $target{shared_extension}

We therefore must add defaults.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7626)

Fix rpath-related Linux "test_shlibload" failure.

When libssl and libcrypto are compiled on Linux with "-rpath", but
not "--enable-new-dtags", the RPATH takes precedence over
LD_LIBRARY_PATH, and we end up running with the wrong libraries.
This is resolved by using full (or at least relative, rather than
just the filename to be found on LD_LIBRARY_PATH) paths to the
shared objects.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7626)

KMAC implementation using EVP_MAC

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7597)

Remove markdown links from HTML comments in issue templates

HTML comments aren't rendered, so markdown link syntax is irrelevant
inside them, and more confusing than useful.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/7632)

Add issue templates and a user support page

This will hopefully help directing our users to better user support
resources as well as give some relevant advice in issue templates.

https://help.github.com/articles/setting-up-your-project-for-healthy-contributions/

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7623)

Added missing signature algorithm reflection functions

    SSL_get_signature_nid()      -- local signature algorithm
    SSL_get_signature_type_nid() -- local signature algorithm key type
    SSL_get_peer_tmp_key()       -- Peer key-exchange public key
    SSL_get_tmp_key              -- local key exchange public key

Aliased pre-existing SSL_get_server_tmp_key(), which was formerly
just for clients, to SSL_get_peer_tmp_key().  Changed internal
calls to use the new name.

Reviewed-by: Matt Caswell <matt@openssl.org>

Merge the CA list documentation for clarity

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7503)

Add a test for SSL_CTX_set0_CA_list()/SSL_CTX_set_client_CA_list()

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7503)

Separate ca_names handling for client and server

SSL(_CTX)?_set_client_CA_list() was a server side only function in 1.1.0.
If it was called on the client side then it was ignored. In 1.1.1 it now
makes sense to have a CA list defined for both client and server (the
client now sends it the the TLSv1.3 certificate_authorities extension).
Unfortunately some applications were using the same SSL_CTX for both
clients and servers and this resulted in some client ClientHellos being
excessively large due to the number of certificate authorities being sent.

This commit seperates out the CA list updated by
SSL(_CTX)?_set_client_CA_list() and the more generic
SSL(_CTX)?_set0_CA_list(). This means that SSL(_CTX)?_set_client_CA_list()
still has no effect on the client side. If both CA lists are set then
SSL(_CTX)?_set_client_CA_list() takes priority.

Fixes #7411

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7503)

Test use of a brainpool ECDSA certificate

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7442)

Add some test brainpool certificates

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7442)

Don't negotiate TLSv1.3 if our EC cert isn't TLSv1.3 capable

TLSv1.3 is more restrictive about the curve used. There must be a matching
sig alg defined for that curve. Therefore if we are using some other curve
in our certificate then we should not negotiate TLSv1.3.

Fixes #7435

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7442)

Fix SipHash init order.

Setting the SipHash hash size and setting its key is done with two
independent functions...  and yet, the internals depend on both.

Unfortunately, the function to change the size wasn't adapted for the
possibility that the key was set first, with a different hash size.

This changes the hash setting function to fix the internal values
(which is easy, fortunately) according to the hash size.

evpmac.txt value for digestsize:8 is also corrected.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7613)

Some deabbreviations

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7614)

Unbreak SECLEVEL 3 regression causing it to not accept any ciphers.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Richard Levitte <levitte@openssl.org>
GH: #7391

Recreate the OS390-Unix config target

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5035)

Check return value of EVP_PKEY_new

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7427)

[crypto/bn] swap BN_FLG_FIXED_TOP too

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/7599)

Add EVP_PKEY_supports_digest_nid()

Rather than relying only on mandatory default digests, add a way for
the EVP_PKEY to individually report whether each digest algorithm is
supported.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7408)

Honour mandatory digest on private key in has_usable_cert()

If the private key says it can only support one specific digest, then
don't ask it to perform a different one.

Fixes: #7348

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7408)

Stop marking default digest for EC keys as mandatory

ASN1_PKEY_CTRL_DEFAULT_MD_NID is documented to return 2 for a mandatory
digest algorithm, when the key can't support any others. That isn't true
here, so return 1 instead.

Partially fixes #7348

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7408)

Fix issues with do_rand_init/rand_cleanup_int

Fixes #7022

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7588)

VMS build: colon after target must be separated with a space

... otherwise, it's taken to be part of a device name.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7602)

Have install targets depend on more precise build targets

We only had the main 'install' target depend on 'all'.  This changes
the dependencies so targets like install_dev, install_runtime_libs,
install_engines and install_programs depend on build targets that are
correspond to them more specifically.  This increases the parallel
possibilities.

Fixes #7466

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7583)

Allow parallel install

When trying 'make -j{n} install', you may occasionally run into
trouble because to sub-targets (install_dev and install_runtime) try
to install the same shared libraries.  That makes parallel install
difficult.

This is solved by dividing install_runtime into two parts, one for
libraries and one for programs, and have install_dev depend on
install_runtime_libs instead of installing the shared runtime
libraries itself.

Fixes #7466

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7583)

VMS build: don't add a comma before 'extradefines'

The variable extradefines will have the starting comma, if needed.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7591)

rand_unix.c: open random devices on first use only

Commit c7504aeb640a (pr #6432) fixed a regression for applications in
chroot environments, which compensated the fact that the new OpenSSL CSPRNG
(based on the NIST DRBG) now reseeds periodically, which the previous
one didn't. Now the reseeding could fail in the chroot environment if the
DEVRANDOM devices were not present anymore and no other entropy source
(e.g. getrandom()) was available.

The solution was to keep the file handles for the DEVRANDOM devices open
by default. In fact, the fix did more than this, it opened the DEVRANDOM
devices early and unconditionally in rand_pool_init(), which had the
unwanted side effect that the devices were opened (and kept open) even
in cases when they were not used at all, for example when the getrandom()
system call was available. Due  to a bug (issue #7419) this even happened
when the feature was disabled by the application.

This commit removes the unconditional opening of all DEVRANDOM devices.
They will now only be opened (and kept open) on first use. In particular,
if getrandom() is available, the handles will not be opened unnecessarily.

This change does not introduce a regression for applications compiled for
libcrypto 1.1.0, because the SSLEAY RNG also seeds on first use. So in the
above constellation the CSPRNG will only be properly seeded if it is happens
before the forking and chrooting.

Fixes #7419

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7437)

Test: enable internal tests for shared Windows builds

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7462)

Test: link drbgtest statically against libcrypto

and remove duplicate rand_drbg_seedlen() implementation again.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7462)

Give a better error if an attempt is made to set a zero length groups list

Previously we indicated this as a malloc failure which isn't very
helpful.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/7479)

Ignore disabled ciphers when deciding if we are using ECC

use_ecc() was always returning 1 because there are default (TLSv1.3)
ciphersuites that use ECC - even if those ciphersuites are disabled by
other options.

Fixes #7471

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/7479)

Add missing RAND initialisation call.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7587)

Rename the rand_drbg_st data member "pool" to "seed_pool"

... to make the intended use more clear and differentiate
it from the data member "adin_pool".

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7575)

util/add-depends.pl: go through shared_sources too

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7545)

Remove outdated e_chil.txt file

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7476)

Fix a race condition in drbgtest.c

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7531)

Fix error handling in RAND_DRBG_uninstantiate

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7517)

Fix error handling in drbgtest.c

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7517)

Fix error handling in rand_drbg_new

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7517)

Fix error handling in RAND_DRBG_set

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7517)

Fix return formatting.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7564)

Cleanse the key log buffer.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7564)

EVP_MAC ctrl numbering duplicate removal.

Both EVP_MAC_CTRL_SET_MD and EVP_MAC_CTRL_SET_CIPHER were numbered 4.
This would preclude any future MAC from using both.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/7566)

GMAC: Add subdir info in crypto/build.info for this to build

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/@7572)

Fix a collision in function err numbers

'make update' complains about this

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7571)

Simplify the processing of skipped source directories

We kept a number of arrays of directory names to keep track of exactly
which directories to look for build.info.  Some of these had the extra
function to hold the directories to actually build.

With the added SUBDIRS keyword, these arrays are no longer needed.
The logic for skipping certain directories needs to be kept, though.
That is now very much simplified, and is made opportunistic.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7558)

Collapse different classes of macro databases

We have $config{openssl_algorithm_defines}, $config{openssl_other_defines}
and $config{openssl_thread_defines}.  These are treated exactly the same
in include/openssl/opensslconf.h.in, so having them separated into three
different databases isn't necessary, the reason for the separation being
long gone.  Therefore, we collapse them into one and the same,
$config{openssl_feature_defines}.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7558)

Add SUBDIRS settings in relevant build.info files

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7558)

Build: make it possibly to specify subdirs in build.info

This adds a keyword SUBDIRS for build.info, to be used like this:

    SUBDIRS=foo bar

This tells Configure that it should look for 'build.info' in the
relative subdirectories 'foo' and 'bar' as well.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7558)

Build: Make it possible to have defines assigned to end products as well

This simple fix allows the following construct:

    PROGRAMS=foo
    SOURCE[foo]=foo.c bar.c
    DEFINE[foo]=FOO=1 BAR=0

These will trickle down to the build of object files, so building
foo.o and bar.o will be done with these options: -DFOO=1 -DBAR=0
(exact syntax depending on platform, of course)

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7553)

Build: adapt VMS build file template to use the extra macros

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7553)

Build: adapt Windows makefile template to use the extra macros

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7553)

Build: adapt Unix Makefile template to use the extra macros

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7553)

Build: make it possible to assign macro definitions for specific outputs

Sometimes, some specific program or object file might need an extra
macro definition of its own.  This allows that to be easily done.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7553)

Add poly1305 MAC support

This is based on the latest EVP MAC interface introduced in PR #7393.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7459)

Clarify the POD source for the list command.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7563)

GMAC implementation
Remove GMAC demo program because it has been superceded by the EVP MAC one

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7548)

Restore sensible "sess_accept" counter tracking

Commit 9ef9088c1585e13b9727796f15f77da64dbbe623 switched the SSL/SSL_CTX
statistics counters to using Thread-Sanitizer-friendly primitives.
However, it erroneously converted an addition of -1
(for s->session_ctx->stats.sess_accept) to an addition of +1, since that
is the only counter API provided by the internal tsan_assist.h header
until the previous commit.  This means that for each accepted (initial)
connection, the session_ctx's counter would get doubly incremented, and the
(switched) ctx's counter would also get incremented.

Restore the counter decrement so that each accepted connection increments
exactly one counter exactly once (in net effect).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7464)

Add tsan_decr() API, counterpart of tsan_counter()

The existing tsan_counter() API increments a reference counter.
Provide a new API, tsan_decr(), to decrement such a reference counter.
This can be used, for example, when a reference is added to the session_ctx's
sess_accept stats but should more properly be tracked in the regular ctx's
statistics.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7464)

crypto/engine/eng_devcrypto.c: ensure we don't leak resources

If engine building fails for some reason, we must make sure to close
the /dev/crypto handle.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7506)

crypto/engine/eng_devcrypto.c: open /dev/crypto only once

We opened /dev/crypto once for each session, which is quite unnecessary.
With this change, we open /dev/crypto once at engine init, and close
it on unload.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7506)

crypto/engine/eng_devcrypto.c: new compilers are strict on prototypes

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7506)

crypto/engine/eng_devcrypto.c: add digest copy

Copying an EVP_MD_CTX, including the implementation local bits, is a
necessary operation.  In this case, though, it's the same as
initializing the local bits to be "copied to".

Fixes #7495

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7506)

doc/man7/EVP_MAC_*.pod: incorrect english corrected

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/7552)

apps: Stop pretending to care about Netscape keys

The documentation says some commands care, but the code says differently.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7440)

Fix a doc-nit in EVP_PKEY_CTX_ctrl.pod

[skip-ci]

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7546)

Add a constant time flag to one of the bignums to avoid a timing leak.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7549)

openssl list -mac-algorithms support.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7541)

Configure: ensure empty arrays aren't created inadvertently

Just refering to a hash table element as an array reference will
automatically create that element.  Avoid that by defaulting to
a separate empty array reference.

Fixes #7543

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7544)

Avoid two memory allocations in each RAND_DRBG_bytes

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7507)

Configuration: when building the dirinfo structure, include shared_sources

This makes sure that any resulting directory target in the build files
also depend on object files meant for shared libraries.

As a side effect, we move the production of the dirinfo structure from
common.tmpl to Configure, to make it easier to check the result.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7452)

Configurations/15-android.conf: detect NDK llvm-ar.

This excluses user from additional PATH adjustments in case NDK has
llvm-ar.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7443)

Configurations/15-android.conf: fix implicit __ANDROID_API__ handling.

03ad7c009e16a233c733098db3169c560142ccd3 failed if one didn't pass
explicit -D__ANDROID_API__=N :-(

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7443)

Fix a race condition in drbg_add

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7523)

Add a client_cert_cb test

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/7413)

Don't call the client_cert_cb immediately in TLSv1.3

In TLSv1.2 and below a CertificateRequest is sent after the Certificate
from the server. This means that by the time the client_cert_cb is called
on receipt of the CertificateRequest a call to SSL_get_peer_certificate()
will return the server certificate as expected. In TLSv1.3 a
CertificateRequest is sent before a Certificate message so calling
SSL_get_peer_certificate() returns NULL.

To workaround this we delay calling the client_cert_cb until after we
have processed the CertificateVerify message, when we are doing TLSv1.3.

Fixes #7384

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/7413)

Add blurbs about EVP_MAC in NEWS and CHANGES

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7526)

Have a couple of SipHash test uses the EVP_PKEY method

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7494)