Windows builds: remove over-quotation of LIBZ macro

The LIBZ macro definition was already quoted in BASE_windows, then got
quotified once more in windows-makefile.tmpl.  That's a bit too much
quotations, ending up with the compiler being asked to define the
macro |"LIBZ=\"ZLIB1\""| (no, not the macro LIBZ with the value
"ZLIB1").  This is solved by removing the extra quoting in
BASE_windows.

Along with this, change the quotation of macro definitions and include
file specification, so we end up with things like -I"QuotedPath" and
-D"Macro=\"some weird value\"" rather than "-IQuotedPath" and
"-DMacro=\"some weird value\"".

Fixes #5827

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5833)

Make sure we use a nonce when a nonce is required

If a nonce is required and the get_nonce callback is NULL, request 50%
more entropy following NIST SP800-90Ar1 section 9.1.

Reviewed-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
GH: #5503

Update documentation on how to close a connection

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
GH: #5823

Fix minor typo in comment in Configure

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5825)

Remove -Wmisleading-indentation from gcc devteam warning options
because this one is enabled by default anyways

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5817)

Fix a gcc-8 warning -Wcast-function-type

Casting to the generic function type "void (*)(void)"
prevents the warning.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/5816)

Refuse to run test_cipherlist unless shared library matches build

test/cipherlist_test.c is an internal consistency check, and therefore
requires that the shared library it runs against matches what it was
built for.  test/recipes/test_cipherlist.t is made to refuse running
unless library version and build version match.

This adds a helper program test/versions.c, that simply displays the
library and the build version.

Partially fixes #5751

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5753)

(cherry picked from commit cde87deafa7486f26bdf954867a6d72ca4ea06e7)

Fix a possible crash in BN_from_montgomery_word

Thanks to Darovskikh Andrei for for reporting this issue.

Fixes: #5785

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5793)

Revert commit 4a56d9a2

We have been unable to trace the contributor of that code to gain their
agreement for the licence change so the code has to be removed.

This commit reverts that contribution. The contribution had no functional
impact so the original way of doing things is still valid. However the
surrounding code has changed significantly so that the exact code as it
was orignally cannot be used. This commit uses the original code as a basis,
but rewrites it to use the PACKET API.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5808)

GOST MAC algorithms don't support EVP_PKEY_new_raw_private_key()

We should use the old EVP_PKEY_new_mac_key() instead.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5808)

Update trace code to know about GOST ciphersuites

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5808)

Remove some code

This commit removes the contribution of a user that we cannot
trace to gain their consent for the licence change.

I also cleaned up the return/error-return flow a bit.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5811)

Add documentation for the RAND_DRBG API

The RAND_DRBG API was added in PR #5462 and modified by PR #5547.
This commit adds the corresponding documention.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5461)

EVP,KDF: Add more error code along some return 0 in ...
methods :
 - EVP_PBE_scrypt
 - EVP_PKEY_meth_add0
 - EVP_PKEY_meth_new
 - EVP_PKEY_CTX_dup

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5783)

BIO_s_accept.pod: add documentation for the new macros

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5007)

Document BIO_{get,set}_conn_ip_family macros.

Add BIO_get_conn_ip_family and BIO_set_conn_ip_family macros to
util/private.num and document them in BIO_s_connect.pod.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5007)

Update the BIO_s_connect documentation after the BIO refactoring.

The BIO was refactored in 417be660e1cd21a2ee085569ff98b0c4249b5416,
but the manual wasn't fully updated to reflect some of the changes.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5007)

Document RAND_DRBG fork-safety locking model

Add some more exposition on why unlocked access to the global rand_fork_count
is safe, and provide a comment for the struct rand_drbg_st fork_count field.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/4110)

Configurations/10-main.conf: fix VC-noCE-common template.

picker() is type agnostic, but its output consumer is not. Or rather
it doesn't work if picker() picks nothing when consumer expects
array. So ensure array is returned when array is expected.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5770)

Configure: harmonize syntax.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5770)

Fix setbuf use for VMS C

The VMS C RTL has setbuf() working for short pointers only, probably
the FILE pointer will always be in P0 (the lower 4GB).  Fortunately,
this only generates a warning about possible data loss (doesn't apply
in this case) that we can simply turn off.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5789)

.travis.yml: with fast fuzz testing, there is no point avoiding it

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5776)

Faster fuzz test: teach the fuzz test programs to handle directories

Instead of invoking the fuzz test programs once for every corpora
file, we invoke them once for each directory of corpora files.  This
dramatically reduces the number of program invikations, as well as the
time 99-test_fuzz.t takes to complete.

fuzz/test-corpus.c was enhanced to handle directories as well as
regular files.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5776)

Revert "util/dofile.pl: only quote stuff that actually needs quoting"

This wasn't a good solution, too many things depend on the quotes being
there consistently.

This reverts commit 49cd47eaababc8c57871b929080fc1357e2ad7b8.

Fixes #5772

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5773)

crypto/e_aes.c: use S390X_AES_FC macro

... to compute s390x aes function code from keylength.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5250)

crypto/evp/e_aes.c: add size_t casts to increase readability

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5250)

s390x assembly pack: add KMF code path for aes-cfb/cfb8

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5250)

s390x assembly pack: add KMO code path for aes-ofb

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5250)

s390x assembly pack: add KM code path for aes-ecb

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5250)

Rewrite the X509->alert mapping code

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5780)

Remove some code

This commit removes the contribution of a user that we cannot trace to
gain their consent for the licence change.

After this commit the various IS_*() macros in the auto-generated file
conf_def.h may incorrectly return true if the supplied character has its
most significant bit set. The IS_*() macros should be able to correctly
handle 8-bit characters. Note that UTF-8 support is not a requirement.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5767)

Tolerate a Certificate using a non-supported group on server side

If a server has been configured to use an ECDSA certificate, we should
allow it regardless of whether the server's own supported groups list
includes the certificate's group.

Fixes #2033

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5601)

Revert "Temporarily disable some tests that hang"

This reverts commit 37a385956461ab526ecea2739a8a40364a8db259.

These tests should now be fixed by commit e6e9170d6.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5765)

Allow NULL for some _free routines.

Based on the description in https://github.com/openssl/openssl/pull/5757,
this re-implements the "allow NULL to be passed" behavior of a number of
xxx_free routines.  I also fixed up some egregious formatting errors
that were nearby.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5761)

o_time.c: use gmtime_s with MSVC
ts/ts_rsp_sign.c: change to OPENSSL_gmtime.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5720)

doc/man1/x509.pod: corrected "S/MIME signing" requirements

When the "certificate purpose" is checked and KeyUsage extension is present,
either 'digitalSignature' or 'nonRepudiation' is accepted.

Manual page corrected to reflect the above.

Signed-off-by: Konstantin Shemyak <konstantin@shemyak.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5727)

statem/statem_clnt.c: omit redundant check in tls_construct_client_hello.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5729)

rand/randfile.c: permit non-regular files in RAND_load_file.

Apparently applications rely on RAND_load_file's ability to work with
non-regular files, customarily with /dev/urandom, so that the ban was
not exactly appropriate.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5737)

Configure: make LIST command work with dynamic 15-android.conf.

This is quick-n-dirty ad-hoc solution, the problem asks for more
elegant one...

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5755)

Temporarily disable some tests that hang

The previous commit causes some tests to hang so we temporarily disable them.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5757)

Remove some code for a contributor that we cannot find

This removes some code because we cannot trace the original contributor
to get their agreement for the licence change (original commit e03ddfae).

After this change there will be numerous failures in the test cases until
someone rewrites the missing code.

All *_free functions should accept a NULL parameter. After this change
the following *_free functions will fail if a NULL parameter is passed:

BIO_ACCEPT_free()
BIO_CONNECT_free()
BN_BLINDING_free()
BN_CTX_free()
BN_MONT_CTX_free()
BN_RECP_CTX_free()
BUF_MEM_free()
COMP_CTX_free()
ERR_STATE_free()
TXT_DB_free()
X509_STORE_free()
ssl3_free()
ssl_cert_free()
SSL_SESSION_free()
SSL_free()

[skip ci]

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5757)

Assert that alpn_selected is NULL before we assign it

The alpn_selected value in the session should be NULL before we first
populate it if this is a new session. We assert to make sure it is.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5715)

Don't write out a bad OID

If we don't have OID data for an object then we should fail if we
are asked to encode the ASN.1 for that OID.

Fixes #5723

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5725)

Update CHANGES and NEWS for the new release

Reviewed-by: Richard Levitte <levitte@openssl.org>

test/test_test.c: add CRYPTO_memcmp regression test.

Reviewed-by: Matt Caswell <matt@openssl.org>

pariscid.pl: fix nasty typo in CRYPTO_memcmp.

Comparison was effectively reduced to least significant bits.

CVE-2018-0733

Reviewed-by: Matt Caswell <matt@openssl.org>

Add fuzz corpora file that found the ASN.1 stack depth issue

Reviewed-by: Rich Salz <rsalz@openssl.org>

Limit ASN.1 constructed types recursive definition depth

Constructed types with a recursive definition (such as can be found in
PKCS7) could eventually exceed the stack given malicious input with
excessive recursion. Therefore we limit the stack depth.

CVE-2018-0739

Credit to OSSFuzz for finding this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove QNX support

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5756)

Configurations/README: mention Libs.private in ex_libs description.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Previously this x509 command line was working, restore that

openssl x509 -in server.pem -signkey privkey.pem -out server.pem

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5746)

Fix dsaparam -genkey with DER outform

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5744)

Fix ecparam -genkey with point compression or DER outform

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5741)

Adds multiple checks to avoid buffer over reads

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5675)

Configurations/windows-makefile.tmpl: refine clean targets.

'nmake clean' was leaving some artefacts behind.

Reviewed-by: Richard Levitte <levitte@openssl.org>

test/shlibloadtest.c: make some variables block local

myDSO_dsobyaddr and myDSO_free are only used in a narrow block of
code, and can therefore be made local to that block.  Otherwise, some
compilers may warn that they are unused.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5733)

test/shlibloadtest.c: fix various errors

These errors were hidden because compiling this file didn't get the
macros derived from the dso_scheme attribute, and therefore, some code
never got compiled.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5733)

Include "internal/dso_conf.h" where needed and appropriate

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5733)

Move the handling of dso_scheme to dso_conf.h

The macros resulting from the dso_scheme attribute were defined for
libraries only, but there's a test program that uses the macros as
well.  The easier way is to move the handling of this macro to
crypto/include/internal/dso_conf.h and having the modules that need it
include it.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5733)

Configurations/unix-Makefile.tmpl: fix inconsistencies with resource files

.res files weren't treated consistently.  They weren't included at all
in the build of a shared library, and were named inconsistently; .res
sometimes, and .res.o otherwise.  Now we standardise it to .res.o,
which is the recommended way with GNU tools.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5730)

Configurations/10-main.conf: correct AIX targets.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5713)

Handle partial messages in TLSProxy

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5412)

Fix resource files

Add it to apps as well as libraries.
Fix the copyright year generation.
Thanks to user RTT for pointing this out.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5704)

include/openssl/rand.h: omit intermediate typedef.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4159)

openssl/rand.h: fix formatting.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4159)

Remove presumably temporary left-over file.

Reviewed-by: Rich Salz <rsalz@openssl.org>

man3/OPENSSL_ia32cap.pod: update assembler version requirements.

Reviewed-by: Rich Salz <rsalz@openssl.org>

curve448/field.h: relax alignment, as it doesn't work universally.

Some platforms, cough-DJGPP, fail to compile claiming that requested
alignment is greater than maximum possible. Supposedly original
alignment was result of an attempt to utilize AVX2...

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5708)

bio/bss_log.c: on DJGPP syslog facility is part of sockets library.

In other words no-sock DJGPP build should suppress syslogging.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5708)

o_fopen.c: compensate for e_os.h omission.

At earlier point e_os.h was omitted from a number of headers (in order
to emphasize OS neutrality), but this affected o_fopen.c, which is not
OS-neutral, and contains some DJGPP-specific code.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5708)

Don't call strsignal, just print the signal number.
The strsignal call is not supported by some machines, so avoid its use.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5658)

aes ctr_drbg: add cavs tests

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #5580

DRBG: Use the EVP layer to do AES encryption

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #5580

Unify s_client/s_server srtp profiles option handling

Add missing guards around STRP-related fields
Remove two unneeded global variables: my 2'cents to #4679
Merge definition and instantiation of srpsrvparm global.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4908)

The default conv_form is uncompressed

Fixes #5711

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5712)

Fixes integer underflow with SSL_trace support

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5706)

Document EC_POINT_get_affine_coordinates_*.

In particular, x and y may be NULL, as used in ecdsa_ossl.c. Make use of
this in ecdh_ossl.c as well, to save an otherwise unnecessary temporary.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5532)

Cleanup the s_time command.

Various code-cleanups.
Use SSL_CTX_set_mode(ctx, SSL_MODE_AUTO_RETRY) insead of handling
SSL_ERROR_WANT_READ everywhere.
Turn off the linger option on connected sockets to avoid failure.
Add BIO_set_conn_mode(conn, BIO_SOCK_NODELAY) to improve thruput.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3952)

Convert _meth_get_ functions to const getters

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2181)

Don't wait for dry at the end of a handshake

For DTLS/SCTP we were waiting for a dry event during the call to
tls_finish_handshake(). This function just tidies up various internal
things, and after it completes the handshake is over. I can find no good
reason for waiting for a dry event here, and nothing in RFC6083 suggests
to me that we should need to. More importantly though it seems to be
wrong. It is perfectly possible for a peer to send app data/alerts/new
handshake while we are still cleaning up our handshake. If this happens
then we will never get the dry event and so we cannot continue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5084)

Check for alerts while waiting for a dry event

At a couple of points in a DTLS/SCTP handshake we need to wait for a dry
event before continuing. However if an alert has been sent by the peer
then we will never receive that dry event and an infinite loop results.

This commit changes things so that we attempt to read a message if we
are waiting for a dry event but haven't got one yet. This should never
succeed, but any alerts will be processed.

Fixes #4763

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5084)

Fix stack-use-after-scope

Fixes regression from #5667.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5701)

o_fopen.c,rand/randfile.c: compensate for e_os.h omission.

At earlier point e_os.h was omitted from a number of headers (in order
to emphasize OS neutrality), but this affected o_fopen.c and randfile.c
which are not OS-neutral, and contain some Win32-specific code.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5676)

Add dladdr() for AIX

Although it deviates from the actual prototype of DSO_dsobyaddr(), this
is now ISO C compliant and gcc -Wpedantic accepts the code.

Added DATA segment checking to catch ptrgl virtual addresses. Avoid
memleaks with every AIX/dladdr() call. Removed debug-fprintf()s.
Added test case for DSO_dsobyaddr(), which will eventually call dladdr().
Removed unecessary AIX ifdefs again.

The implementation can only lookup function symbols, no data symbols.
Added PIC-flag to aix*-cc build targets.

As AIX is missing a dladdr() implementation it is currently uncertain our
exit()-handlers can still be called when the application exits. After
dlclose() the whole library might have been unloaded already.

Signed-off-by: Matthias Kraft <makr@gmx.eu>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5668)

Do not cache sessions with zero sid_ctx_length when SSL_VERIFY_PEER

The sid_ctx is something of a "certificate request context" or a
"session ID context" -- something from the application that gives
extra indication of what sort of thing this session is/was for/from.
Without a sid_ctx, we only know that there is a session that we
issued, but it could have come from a number of things, especially
with an external (shared) session cache.  Accordingly, when resuming,
we will hard-error the handshake when presented with a session with
zero-length sid_ctx and SSL_VERIFY_PEER is set -- we simply have no
information about the peer to verify, so the verification must fail.

In order to prevent these future handshake failures, proactively
decline to add the problematic sessions to the session cache.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5175)

Remove mention of link between message digests and public key algorithms.

The comment in EVP_DigestInit.pod is:

> Returns the NID of the public key signing algorithm associated with this
digest. For example EVP_sha1() is associated with RSA so this will return
B<NID_sha1WithRSAEncryption>. Since digests and signature algorithms are no
longer linked this function is only retained for compatibility reasons.

I.e. there is no link anymore.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5682)

aix compat fixes for ocsp.c

WCOREDUMP and vsyslog are not portable

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5657)

Make pkeyutl a bit more user-friendly

Give meaningful error messages when the user incorrectly uses pkeyutl.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3987)

Reduce the verbosity of test_store

The travis logs are going above 4Mb causing the builds to fail. One
test creates excessive output. This change reduces that output by approx
180k.

[extended tests]

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5694)

Fix the OCSP responder mode

Broken by commit 3e3c7c36.

Fixes #5681

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5688)

Prepare for 1.1.1-pre4-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.1-pre3 release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update copyright year

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5689)

crypto/rand/rand_vms.c: include "internal/rand_int.h"

Without it, the RAND_POOL typedef is missing

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5685)

Fix: drbgtest fails when tests are executed in random order

[extended tests]

The test_rand_reseed assumed that the global DRBGs were not used
previously. This assumption is false when the tests are executed
in random order (OPENSSL_TEST_RAND_ORDER). So we uninstantiate
them first and add a test for the first instantiation.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5680)

s_client, s_server: do generic SSL configuration first, specialization after

We did the SSL_CONF_cmd() pass last of all things that could affect
the SSL ctx.  However, the results of this, for example:

    -max_protocol TLSv1.3 -tls1_2

... would mean that the protocol min got set to TLSv1.2 and the
protocol max to TLSv1.3, when they should clearly both be TLSv1.2.

However, if we see the SSL_CONF_cmd() switches as generic and those
internal to s_client and s_server as specialisations, we get something
that makes a little more sense.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5679)

Fix no-sm3/no-sm2 (with strict-warnings)

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5677)

Fix no-sm3 (and no-sm2)

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5677)

Don't generate buildtest_*err.c

The error string header files aren't supposed to be included directly,
so there's no point testing that they can.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5678)

Enhance ssltestlib's create_ssl_ctx_pair to take min and max proto version

Have all test programs using that function specify those versions.
Additionally, have the remaining test programs that use SSL_CTX_new
directly specify at least the maximum protocol version.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5663)