Fix false positives of IS_*() macros for 8-bit ASCII characters

Fixes #5778, #5840

The various IS_*() macros did not work correctly for 8-bit ASCII
characters with the high bit set, because the CVT(a) preprocessor
macro and'ed the given ASCII value with 0x7F, effectively folding
the high value range 128-255 over the low value range 0-127.
As a consequence, some of the IS_*() erroneously returned TRUE.

This commit fixes the issue by adding range checks instead of
cutting off high order bits using a mask. In order avoid multiple
evaluation of macro arguments, most of the implementation was moved
from macros into a static function is_keytype().

Thanks to Румен Петров for reporting and analyzing the UTF-8 parsing
issue #5840.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5903)

Fix the build_all_generated rule to include generated .map, .def and .opt files

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5900)

TLSProxy/Record.pm: remove dead condition and improve readability.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5887)

TLSProxy/Proxy.pm: refine partial packet handling.

Original logic was "if no records found *or* last one is truncated, then
leave complete records in queue." Trouble is that if we don't pass on
complete records and get complete packet in opposite direction, then
queued records will go back to sender. In other words complete records
should always be passed on. [Possible alternative would be to match
direction in reconstruct_record.]

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5887)

apps/{s_client.c|s_socket}.c: omit usleep calls.

Even though removed calls were oiriginally added on Windows, problem
they tried to mitigate is not Windows-specific.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5887)

apps/s_socket.c: disable the Nagle algorithm.

Without TCP_NODELAY alerts risk to be dropped between shutdown and close.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5887)

TLSProxy/Proxy.pm: harmonize inner loop with the way sockets are.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5887)

Configurations/10-main.conf: clean up HP-UX targets and add magic macros.

HP-UX provides sockets symbols with incompatible prototypes under same
name. This caused problems in 64-bit builds. Additional macros force
unambiguous symbols with unambiguous prototypes.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5742)

config: fix hpux64-parisc2-gcc detection.

hpux64-parisc2-gcc is chosen based on gcc's bitness, and it was overriden
unconditionally.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5742)

test/asn1_time_test.c: make it work on 64-bit HP-UX.

HP-UX gmtime fails with ERANGE past 19011213204552Z, so skip some tests.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5742)

Updated to CONTRIBUTING to reflect GitHub, etc.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5889)

Duplicate code refactored

add_attribute_object and add_DN_object have similar code, so move
it into a common function build_data.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4566)

Add a note and better error if using Ed25519/Ed448 in dgst

Fixes #5873

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5880)

Support EVP_PKEY_sign() and EVP_PKEY_verify() for EdDSA

Adding support for these operations for the EdDSA implementations
makes pkeyutl usable for signing/verifying for these algorithms.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5880)

Fix bugs in X509_NAME_ENTRY_set

The wrong "set" field was incremented in the wrong place and would
create a new RDN, not a multi-valued RDN.
RDN inserts would happen after not before.
Prepending an entry to an RDN incorrectly created a new RDN

Anything which built up an X509_NAME could get a messed-up structure,
which would then be "wrong" for anyone using that name.

Thanks to Ingo Schwarze for extensive debugging and the initial
fix (documented in GitHub issue #5870).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/5882)

Set error code if alloc returns NULL

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5886)

Update the genpkey documentation

Fixes #5739

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5800)

Pick a q size consistent with the digest for DSA param generation

There are two undocumented DSA parameter generation options available in
the genpkey command line app:
dsa_paramgen_md and dsa_paramgen_q_bits.

These can also be accessed via the EVP API but only by using
EVP_PKEY_CTX_ctrl() or EVP_PKEY_CTX_ctrl_str() directly. There are no
helper macros for these options.

dsa_paramgen_q_bits sets the length of q in bits (default 160 bits).
dsa_paramgen_md sets the digest that is used during the parameter
generation (default SHA1). In particular the output length of the digest
used must be equal to or greater than the number of bits in q because of
this code:

            if (!EVP_Digest(seed, qsize, md, NULL, evpmd, NULL))
                goto err;
            if (!EVP_Digest(buf, qsize, buf2, NULL, evpmd, NULL))
                goto err;
            for (i = 0; i < qsize; i++)
                md[i] ^= buf2[i];

            /* step 3 */
            md[0] |= 0x80;
            md[qsize - 1] |= 0x01;
            if (!BN_bin2bn(md, qsize, q))
                goto err;

qsize here is the number of bits in q and evpmd is the digest set via
dsa_paramgen_md. md and buf2 are buffers of length SHA256_DIGEST_LENGTH.
buf2 has been filled with qsize bits of random seed data, and md is
uninitialised.

If the output size of evpmd is less than qsize then the line "md[i] ^=
buf2[i]" will be xoring an uninitialised value and the random seed data
together to form the least significant bits of q (and not using the
output of the digest at all for those bits) - which is probably not what
was intended. The same seed is then used as an input to generating p. If
the uninitialised data is actually all zeros (as seems quite likely)
then the least significant bits of q will exactly match the least
significant bits of the seed.

This problem only occurs if you use these undocumented and difficult to
find options and you set the size of q to be greater than the message
digest output size. This is for parameter generation only not key
generation. This scenario is considered highly unlikely and
therefore the security risk of this is considered negligible.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5800)

Don't crash if an unrecognised digest is used with dsa_paramgen_md

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5800)

Document the change in the previous commit about loading the config file

When libssl is initialised it will attempt to load any config file. This
ensures any system_default configuration (as per
https://github.com/openssl/openssl/pull/4848) is used.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5818)

Move the loading of the ssl_conf module to libcrypto

The GOST engine needs to be loaded before we initialise libssl. Otherwise
the GOST ciphersuites are not enabled. However the SSL conf module must
be loaded before we initialise libcrypto. Otherwise we will fail to read
the SSL config from a config file properly.

Another problem is that an application may make use of both libcrypto and
libssl. If it performs libcrypto stuff first and OPENSSL_init_crypto()
is called and loads a config file it will fail if that config file has
any libssl stuff in it.

This commit separates out the loading of the SSL conf module from the
interpretation of its contents. The loading piece doesn't know anything
about SSL so this can be moved to libcrypto. The interpretation of what it
means remains in libssl. This means we can load the SSL conf data before
libssl is there and interpret it when it later becomes available.

Fixes #5809

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5818)

Change rand_pool_add[_end] prototypes to match

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5877)

rand/randfile.c: fix potential resource leak in RAND_load_file.

Found by Coverity.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5834)

TLSProxy/Proxy.pm: switch to dynamic ports and overhaul.

By asking for port 0, you get a free port dynamically assigned by OS.
TLSProxy::Proxy now asks for 0 and asks s_server to do the same. The
s_server's port is reported in "ACCEPT" line, which TLSProxy::Proxy
parses and uses.

Because the server port is now a random affair in TLSProxy::Proxy,
it's no longer possible to change it with the method 'server_port',
and it has become an accessor only. For the sake of orthogonality, so
has the method 'server_addr'.

Remove all fork calls on Windows, as fork is not to be trusted there.
This naturally minimized amount of fork calls on POSIX systems, to 1.

Sink s_server's output to 'perl -ne print' which ensures that output
is written strictly in lines. This keeps TAP parser happy.

Improve synchronization in -naccept +n cases by establishing next
connection to s_server *after* s_client finishes instead of before it
starts.

Improve error handling and clean up some methods.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5843)

openssl s_server: print the accepting address and socket

The line saying ACCEPT is extended with a space followed by the the
address and port combination on which s_server accepts connections.
The address is written in such a way that s_client should be able to
accepts as argument for the '-connect' option.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5843)

Remove ambiguity in rand_pool_add[_end] return value

When these two functions returned zero, it could mean:

1. that an error occured.  In their case, the error is an overflow of
   the pool, i.e. the correct response from the caller would be to
   stop trying to fill the pool.
2. that there isn't enought entropy acquired yet, i.e. the correct
   response from the caller would be to try and add more entropy to
   the pool.

Because of this ambiguity, the returned zero turns out to be useless.
This change makes the returned value more consistent.  1 means the
addition of new entropy was successful, 0 means it wasn't.  To know if
the pool has been filled enough, the caller will have to call some
other function, such as rand_pool_entropy_available().

Fixes #5846

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5876)

Add test/versions to gitignore

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5875)

Fix configuration of TLSv1.3 ciphersuites

Configuration of TLSv1.3 ciphersuites wasn't working in some cases.

Fixes #5740

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5855)

Add some tests for configuring the TLSv1.3 ciphersuites

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5855)

Don't use CPP in Configurations/unix-Makefile.tmpl

We started using $(CPP) instead of $(CC) -E, with the assumption that
CPP would be predefined.  This is, however, not always true, and
rather depends on the 'make' implementation.  Furthermore, on
platforms where CPP=cpp or something else other than '$(CC) -E',
there's a risk that it won't understand machine specific flags that we
pass to it.  So it turns out that trying to use $(CPP) was a mistake,
and we therefore revert that use back to using $(CC) -E directly.

Fixes #5867

Note: this affects config targets that use Alpha, ARM, IA64, MIPS,
s390x or SPARC assembler modules.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5872)

Visual Studio 2017 debug build warning error on 32 bit build

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5799)

Prevent a possible recursion in ERR_get_state and fix the problem that
was pointed out in commit aef84bb4efbddfd95d042f3f5f1d362ed7d4faeb
differently.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5863)

Don't use getenv for critical functions when run as setuid/setgid

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5856)

Fix a bug in ecp_nistp224.c.

felem_neg does not produce an output within the tight bounds suitable
for felem_contract. This affects build configurations which set
enable-ec_nistp_64_gcc_128.

point_double and point_add, in the non-z*_is_zero cases, tolerate and
fix up the wider bounds, so this only affects point_add calls where the
other point is infinity. Thus it only affects the final addition in
arbitrary-point multiplication, giving the wrong y-coordinate. This is a
no-op for ECDH and ECDSA, which only use the x-coordinate of
arbitrary-point operations.

Note: ecp_nistp521.c has the same issue in that the documented
preconditions are violated by the test case. I have not addressed this
in this PR. ecp_nistp521.c does not immediately produce the wrong
answer; felem_contract there appears to be a bit more tolerant than its
documented preconditions. However, I haven't checked the point_add
property above holds. ecp_nistp521.c should either get this same fix, to
be conservative, or have the bounds analysis and comments reworked for
the wider bounds.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5779)

Enabled OneCore Conf for Console Apps (removed nonUniversal API)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5625)

VMS: stricter acquisition of entropy for the pool

Fail harshly (in debug builds) when rand_pool_acquire_entropy isn't
delivering the required amount of entropy.  In release builds, this
produces an error with details.

We also take the opportunity to modernise the types used.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5857)

Set error code on alloc failures

Almost all *alloc failures now set an error code.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5842)

Fix some errors in the mem leaks docs

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5794)

Fix a text canonicalisation bug in CMS

Where a CMS detached signature is used with text content the text goes
through a canonicalisation process first prior to signing or verifying a
signature. This process strips trailing space at the end of lines, converts
line terminators to CRLF and removes additional trailing line terminators
at the end of a file. A bug in the canonicalisation process meant that
some characters, such as form-feed, were incorrectly treated as whitespace
and removed. This is contrary to the specification (RFC5485). This fix
could mean that detached text data signed with an earlier version of
OpenSSL 1.1.0 may fail to verify using the fixed version, or text data
signed with a fixed OpenSSL may fail to verify with an earlier version of
OpenSSL 1.1.0. A workaround is to only verify the canonicalised text data
and use the "-binary" flag (for the "cms" command line application) or set
the SMIME_BINARY/PKCS7_BINARY/CMS_BINARY flags (if using CMS_verify()).

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5790)

Change the "offset too large" message to more generic wording

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

Remove an unnecessary cast in the param to BUF_MEM_grow

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

Fix range checks with -offset and -length in asn1parse

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

Use strtol instead of atoi in asn1parse

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

Improve diagnostics for invalid arguments in asn1parse -strparse

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

Fix a crash in the asn1parse command

Thanks to Sem Voigtländer for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

Use gnu_printf format attribute to minimise MinGW warnings

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5839)

Prepare for 1.1.1-pre5-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.1-pre4 release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix CHANGES

Fix the last release version number in CHANGES

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5852)

Update copyright year

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5851)

Use the private RNG for data that is not public

Reviewed-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Fixes: #4641
GH: #4665

Documentation typo fix in EVP_EncryptInit.pod

Remove duplicate declaration of `EVP_CIPHER_key_length` in the synopsis.

CLA: trivial

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5792)

fix typo in x509 documentation

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5829)

Remove import/use of File::Spec::Function

It looks like the usage of these functions were removed in
in commit 0a4edb931b883b9973721ae012e60c028387dd50 ("Unified - adapt
the generation of cpuid, uplink and buildinf to use GENERATE").

This commit removes the import/use of File::Spec::Functions module as it
is no longer needed by crypto/build.info.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5832)

Windows builds: remove over-quotation of LIBZ macro

The LIBZ macro definition was already quoted in BASE_windows, then got
quotified once more in windows-makefile.tmpl.  That's a bit too much
quotations, ending up with the compiler being asked to define the
macro |"LIBZ=\"ZLIB1\""| (no, not the macro LIBZ with the value
"ZLIB1").  This is solved by removing the extra quoting in
BASE_windows.

Along with this, change the quotation of macro definitions and include
file specification, so we end up with things like -I"QuotedPath" and
-D"Macro=\"some weird value\"" rather than "-IQuotedPath" and
"-DMacro=\"some weird value\"".

Fixes #5827

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5833)

Make sure we use a nonce when a nonce is required

If a nonce is required and the get_nonce callback is NULL, request 50%
more entropy following NIST SP800-90Ar1 section 9.1.

Reviewed-by: Dr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
GH: #5503

Update documentation on how to close a connection

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
GH: #5823

Fix minor typo in comment in Configure

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5825)

Remove -Wmisleading-indentation from gcc devteam warning options
because this one is enabled by default anyways

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5817)

Fix a gcc-8 warning -Wcast-function-type

Casting to the generic function type "void (*)(void)"
prevents the warning.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/5816)

Refuse to run test_cipherlist unless shared library matches build

test/cipherlist_test.c is an internal consistency check, and therefore
requires that the shared library it runs against matches what it was
built for.  test/recipes/test_cipherlist.t is made to refuse running
unless library version and build version match.

This adds a helper program test/versions.c, that simply displays the
library and the build version.

Partially fixes #5751

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5753)

(cherry picked from commit cde87deafa7486f26bdf954867a6d72ca4ea06e7)

Fix a possible crash in BN_from_montgomery_word

Thanks to Darovskikh Andrei for for reporting this issue.

Fixes: #5785

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5793)

Revert commit 4a56d9a2

We have been unable to trace the contributor of that code to gain their
agreement for the licence change so the code has to be removed.

This commit reverts that contribution. The contribution had no functional
impact so the original way of doing things is still valid. However the
surrounding code has changed significantly so that the exact code as it
was orignally cannot be used. This commit uses the original code as a basis,
but rewrites it to use the PACKET API.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5808)

GOST MAC algorithms don't support EVP_PKEY_new_raw_private_key()

We should use the old EVP_PKEY_new_mac_key() instead.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5808)

Update trace code to know about GOST ciphersuites

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5808)

Remove some code

This commit removes the contribution of a user that we cannot
trace to gain their consent for the licence change.

I also cleaned up the return/error-return flow a bit.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5811)

Add documentation for the RAND_DRBG API

The RAND_DRBG API was added in PR #5462 and modified by PR #5547.
This commit adds the corresponding documention.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5461)

EVP,KDF: Add more error code along some return 0 in ...
methods :
 - EVP_PBE_scrypt
 - EVP_PKEY_meth_add0
 - EVP_PKEY_meth_new
 - EVP_PKEY_CTX_dup

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5783)

BIO_s_accept.pod: add documentation for the new macros

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5007)

Document BIO_{get,set}_conn_ip_family macros.

Add BIO_get_conn_ip_family and BIO_set_conn_ip_family macros to
util/private.num and document them in BIO_s_connect.pod.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5007)

Update the BIO_s_connect documentation after the BIO refactoring.

The BIO was refactored in 417be660e1cd21a2ee085569ff98b0c4249b5416,
but the manual wasn't fully updated to reflect some of the changes.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5007)

Document RAND_DRBG fork-safety locking model

Add some more exposition on why unlocked access to the global rand_fork_count
is safe, and provide a comment for the struct rand_drbg_st fork_count field.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/4110)

Configurations/10-main.conf: fix VC-noCE-common template.

picker() is type agnostic, but its output consumer is not. Or rather
it doesn't work if picker() picks nothing when consumer expects
array. So ensure array is returned when array is expected.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5770)

Configure: harmonize syntax.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5770)

Fix setbuf use for VMS C

The VMS C RTL has setbuf() working for short pointers only, probably
the FILE pointer will always be in P0 (the lower 4GB).  Fortunately,
this only generates a warning about possible data loss (doesn't apply
in this case) that we can simply turn off.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5789)

.travis.yml: with fast fuzz testing, there is no point avoiding it

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5776)

Faster fuzz test: teach the fuzz test programs to handle directories

Instead of invoking the fuzz test programs once for every corpora
file, we invoke them once for each directory of corpora files.  This
dramatically reduces the number of program invikations, as well as the
time 99-test_fuzz.t takes to complete.

fuzz/test-corpus.c was enhanced to handle directories as well as
regular files.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5776)

Revert "util/dofile.pl: only quote stuff that actually needs quoting"

This wasn't a good solution, too many things depend on the quotes being
there consistently.

This reverts commit 49cd47eaababc8c57871b929080fc1357e2ad7b8.

Fixes #5772

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5773)

crypto/e_aes.c: use S390X_AES_FC macro

... to compute s390x aes function code from keylength.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5250)

crypto/evp/e_aes.c: add size_t casts to increase readability

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5250)

s390x assembly pack: add KMF code path for aes-cfb/cfb8

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5250)

s390x assembly pack: add KMO code path for aes-ofb

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5250)

s390x assembly pack: add KM code path for aes-ecb

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5250)

Rewrite the X509->alert mapping code

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5780)

Remove some code

This commit removes the contribution of a user that we cannot trace to
gain their consent for the licence change.

After this commit the various IS_*() macros in the auto-generated file
conf_def.h may incorrectly return true if the supplied character has its
most significant bit set. The IS_*() macros should be able to correctly
handle 8-bit characters. Note that UTF-8 support is not a requirement.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5767)

Tolerate a Certificate using a non-supported group on server side

If a server has been configured to use an ECDSA certificate, we should
allow it regardless of whether the server's own supported groups list
includes the certificate's group.

Fixes #2033

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5601)

Revert "Temporarily disable some tests that hang"

This reverts commit 37a385956461ab526ecea2739a8a40364a8db259.

These tests should now be fixed by commit e6e9170d6.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5765)

Allow NULL for some _free routines.

Based on the description in https://github.com/openssl/openssl/pull/5757,
this re-implements the "allow NULL to be passed" behavior of a number of
xxx_free routines.  I also fixed up some egregious formatting errors
that were nearby.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5761)

o_time.c: use gmtime_s with MSVC
ts/ts_rsp_sign.c: change to OPENSSL_gmtime.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5720)

doc/man1/x509.pod: corrected "S/MIME signing" requirements

When the "certificate purpose" is checked and KeyUsage extension is present,
either 'digitalSignature' or 'nonRepudiation' is accepted.

Manual page corrected to reflect the above.

Signed-off-by: Konstantin Shemyak <konstantin@shemyak.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5727)

statem/statem_clnt.c: omit redundant check in tls_construct_client_hello.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5729)

rand/randfile.c: permit non-regular files in RAND_load_file.

Apparently applications rely on RAND_load_file's ability to work with
non-regular files, customarily with /dev/urandom, so that the ban was
not exactly appropriate.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5737)

Configure: make LIST command work with dynamic 15-android.conf.

This is quick-n-dirty ad-hoc solution, the problem asks for more
elegant one...

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5755)

Temporarily disable some tests that hang

The previous commit causes some tests to hang so we temporarily disable them.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5757)

Remove some code for a contributor that we cannot find

This removes some code because we cannot trace the original contributor
to get their agreement for the licence change (original commit e03ddfae).

After this change there will be numerous failures in the test cases until
someone rewrites the missing code.

All *_free functions should accept a NULL parameter. After this change
the following *_free functions will fail if a NULL parameter is passed:

BIO_ACCEPT_free()
BIO_CONNECT_free()
BN_BLINDING_free()
BN_CTX_free()
BN_MONT_CTX_free()
BN_RECP_CTX_free()
BUF_MEM_free()
COMP_CTX_free()
ERR_STATE_free()
TXT_DB_free()
X509_STORE_free()
ssl3_free()
ssl_cert_free()
SSL_SESSION_free()
SSL_free()

[skip ci]

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5757)

Assert that alpn_selected is NULL before we assign it

The alpn_selected value in the session should be NULL before we first
populate it if this is a new session. We assert to make sure it is.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5715)

Don't write out a bad OID

If we don't have OID data for an object then we should fail if we
are asked to encode the ASN.1 for that OID.

Fixes #5723

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5725)

Update CHANGES and NEWS for the new release

Reviewed-by: Richard Levitte <levitte@openssl.org>

test/test_test.c: add CRYPTO_memcmp regression test.

Reviewed-by: Matt Caswell <matt@openssl.org>

pariscid.pl: fix nasty typo in CRYPTO_memcmp.

Comparison was effectively reduced to least significant bits.

CVE-2018-0733

Reviewed-by: Matt Caswell <matt@openssl.org>