util/dofile.pl: only quote stuff that actually needs quoting

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5533)

(cherry picked from commit 49cd47eaababc8c57871b929080fc1357e2ad7b8)

opensslconf.h.in: Use all the "openssl_api_defines"

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5533)

(cherry picked from commit cd15cb4d21fce81c94bc16f991c4bb1c73e71bfe)

Avoid unconditional store in CRYPTO_malloc.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5372)
(cherry picked from commit 41aede863b76202347c2d5e2c2666428084f9203)

d2i_X509.pod: clarify usage of the 'pp' function parameter

The 'pp' function parameters of d2i_TYPE() and i2d_TYPE() are referenced
in the DESCRIPTION section as 'in' resp. 'out'. This commit renames the
references to 'ppin' resp. 'ppout' and adds an explaining sentence.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5365)

Fix credit for SRP code

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5504)

Windows makefile: Don't quote generator arguments

Rely on the build.info constructor to do the right thing.

Fixes #5500

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5501)

(cherry picked from commit 1c9858d0d013184ff756d063022161b1853a9cbf)

mem_sec.c: portability fixup.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5493)

(cherry picked from commit 014cc4b27a7f8ed0cf23a3c9d1fdbf44e41b7993)

Fix a possible memory leak in engine_table_register

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5489)

(cherry picked from commit 55a7f77d72930f9aee1a51e0af9658b2728be127)

Fixed a typo in a man page

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5498)
(cherry picked from commit c03dc6427cb8d15ddce735b5e5beef606bdea51d)

Fix typo in ASN1_STRING_length doc

CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5458)

Test the result of CMS_RecipientInfo_ktri_get0_algs() before using its output in rsa_cms_encrypt().

CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4966)

(cherry picked from commit 178989b4a3ed714fa169cae5865c75f156ec9fdb)

Always use adr with __thumb2__.

Thumb2 addresses are a bit a mess, depending on whether a label is
interpreted as a function pointer value (for use with BX and BLX) or as
a program counter value (for use with PC-relative addressing). Clang's
integrated assembler mis-assembles this code. See
https://crbug.com/124610#c54 for details.

Instead, use the ADR pseudo-instruction which has clear semantics and
should be supported by every assembler that handles the OpenSSL Thumb2
code. (In other files, the ADR vs SUB conditionals are based on
__thumb2__ already. For some reason, this one is based on __APPLE__, I'm
guessing to deal with an older version of clang assembler.)

It's unclear to me which of clang or binutils is "correct" or if this is
even a well-defined notion beyond "whatever binutils does". But I will
note that https://github.com/openssl/openssl/pull/4669 suggests binutils
has also changed behavior around this before.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5431)

(cherry picked from commit 8a5d8bc4bc1e835b62d988ad63454540be83d862)

bio_b64.c: prevent base64 filter BIO from decoding out-of-bound data

Fixes #5405, #1381

The base64 filter BIO reads its input in chunks of B64_BLOCK_SIZE bytes.
When processing input in PEM format it can happen in rare cases that

- the trailing PEM marker crosses the boundary of a chunk, and
- the beginning of the following chunk contains valid base64 encoded data.

This happened in issue #5405, where the PEM marker was split into
"-----END CER" and "TIFICATE-----" at the end of the first chunk.

The decoding of the first chunk terminated correctly at the '-' character,
which is treated as an EOF marker, and b64_read() returned. However,
when called the second time, b64_read() read the next chunk and interpreted
the string "TIFICATE" as valid base64 encoded data, adding 6 extra bytes
'4c 81 48 08 04 c4'.

This patch restores the assignment of the error code to 'ctx->cont', which
was deleted accidentally in commit 5562cfaca4f3 and which prevents b64_read()
from reading additional data on subsequent calls.

This issue was observed and reported by Annie Yousar.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5422)

mem_sec.c: relax POSIX requirement.

Even though mlock(2) was standardized in POSIX.1-2001, vendors did
implement it prior that point.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5460)

(cherry picked from commit 5839185cdd9b339ff741da437d964a25e72a3fb6)

test/recipes/80-test_pkcs12.t: handle lack of Win32::API.

So far check for availability of Win32::API served as implicit check
for $^O being MSWin32. Reportedly it's not safe assumption, and check
for MSWin32 has to be explicit.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5416)

(cherry picked from commit d4c499f562c1ab7ec7773c3987fc4dce7662a805)

Fix some bugs with the cfb1 bitsize handling

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5426)

(cherry picked from commit 604e591ed75eff9296c21ee5fe93f3e9ec246094)

Update EC_POINT_new.pod

CLA: trivial
fix typo:
EC_point2buf => EC_POINT_point2buf

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5367)

(cherry picked from commit 6f4b929af0b7dfb6a4d420fc9a8e25016f616c10)

do_body: fix heap-use-after-free.

The memory pointed to by the 'push' is freed by the
X509_NAME_ENTRY_free() in do_body(). The second time
it is referenced to (indirectly) in certify_cert:X509_REQ_free().

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4698)

X509V3_EXT_add_nconf_sk, X509v3_add_ext: fix errors handling

X509v3_add_ext: free 'sk' if the memory pointed to by it
was malloc-ed inside this function.
X509V3_EXT_add_nconf_sk: return an error if X509v3_add_ext() fails.
This prevents use of a freed memory in do_body:sk_X509_EXTENSION_num().

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4698)

Sanity check the ticket length before using key name/IV

This could in theory result in an overread - but due to the over allocation
of the underlying buffer does not represent a security issue.

Thanks to Fedor Indutny for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5415)

Add X509_get0_authority_key_id() function

This function makes it easier to retrieve a reference to the
authority key identifier (akid->keyid) inside a certificate.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5271)

(cherry picked from commit b383aa2081467e8d49c3362d295da7bd5cb4e1d8)

test_ssl_old: avoid empty strings for flags

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5396)

(cherry picked from commit 62930b2ecf6ce308fd2ae9ee3e34ace8ba698aac)

OpenSSL::Test::quotify: put quotes around empty arguments

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5396)

(cherry picked from commit 5845f7de8165f35829e49acd41f6a2fc3698f221)

Check the return code from ASN1_TIME_diff()

The function can fail so we should check the return code.

Found by Coverity

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5339)

(cherry picked from commit 0d502c3511ab3b1c8acb129fb3af836727b8092d)

The function X509_gmtime_adj() can fail

Check for a failure and free a_tm as appropriate.

Found by Coverity

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5339)

(cherry picked from commit bc2a0dd283c0f61df572b8c2aaf3bfc2dd4b7571)

Avoid fragile aliasing of SHA224/384 update/final

This is purported to save a few cycles, but makes the code less
obvious and more brittle, and in fact breaks on platforms where for
ABI continuity reasons there is a SHA2 implementation in libc, and
so EVP needs to call those to avoid conflicts.

A sufficiently good optimizer could simply generate the same entry
points for:

        foo(...) { ... }
    and
        bar(...) { return foo(...); }

but, even without that, the different is negligible, with the
"winner" varying from run to run (openssl speed -evp sha384):

    Old:
    type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes 16384 bytes
    sha384           28864.28k   117362.62k   266469.21k   483258.03k   635144.87k 649123.16k

    New:
    type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes 16384 bytes
    sha384           30055.18k   120725.98k   272057.26k   482847.40k   634585.09k 650308.27k

Reviewed-by: Rich Salz <rsalz@openssl.org>

Configure: if a file is generated, never assume it's in the source dir

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5355)

(cherry picked from commit 9b7e82f8d939ca6894f941268b219da55f069b26)

Fix bug in BIO_f_linebuffer()

In BIO_f_linebuffer, this would cause an error:

    BIO_write(bio, "1\n", 1);

I.e. there's a \n just after the part of the string that we currently
ask to get written.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5353)

Copy dlls into fuzz directory

This should fix the recent AppVeyor failures.

[extended tests]

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4171)

(cherry picked from commit 30bb02597df9a79b46bae2ddc9885a35286ed205)

Add the target 'build_all_generated'

This new target is used to build all generated files and only that.
This can be used to prepare everything that requires things like perl
for a system that lacks perl and then move everything to that system
and do the rest of the build there.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3695)

(cherry picked from commit 9b03b91b84b64c086081b87bd0a2c7d3612af6c0)

Remove code that prints "<SPACES/NULS>" in hexdumps
when the data block ends with SPACEs or NULs.

The problem is, you can't see if the data ends
with SPACE or NUL or a combination of both.

This can happen for instance with
openssl rsautl -decrypt -hexdump

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5332)

Copy name string in BIO_meth_new

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5319)

Avoid leaking peername data via accept BIOs

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit 4e0752535eb87b9aab4cf193f4422b5801ab7b32)

Swap the check in ssl3_write_pending to avoid using
the possibly indeterminate pointer value in wpend_buf.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5307)

Don't calculate the Finished MAC twice

In <= TLSv1.2 a Finished message always comes immediately after a CCS
except in the case of NPN where there is an additional message between
the CCS and Finished. Historically we always calculated the Finished MAC
when we processed the CCS. However to deal with NPN we also calculated it
when we receive the Finished message. Really this should only have been
done if we hand negotiated NPN.

This simplifies the code to only calculate the MAC when we receive the
Finished. In 1.1.1 we need to do it this way anyway because there is no
CCS (except in middlebox compat mode) in TLSv1.3.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5286)

util/mkdef.pl: Fix incomplete cherry-pick

The cherry pick that resulted in 65de3f1657d8a3bdb7c48063931a3c619817c921
was incomplete.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5283)

util/mkdef.pl: use better array in search of 'DEPRECATEDIN_'

%disabled_algorithms isn't necessarily initialised with the "algos"
'DEPRECATEDIN_1_1_0' etc.  However, we know that @known_algorithms has
them all, so use that to find them instead.

Fixes #5157
(where this was reported)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5282)

(cherry picked from commit b53fdad0e4350ba49812c50305686ee5a6239111)

util/mkdef.pl: Trust configdata.pm

This script kept its own database of disablable algorithms, which is a
maintenance problem, as it's not always perfectly in sync with what
Configure does.  However, we do have all the data in configdata.pm,
produced by Configure, so let's use that instead.

Also, make sure to parse the *err.h header files, as they contain
function declarations that might not be present elsewhere.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5157)

(cherry picked from commit 54f3b7d2f5a313e5c702f75ee030f8a08e6bf6aa)

Make all private functions in e_afalg.c static

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5273)

Remove unused file

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5267)
(cherry picked from commit a3a5d1b73db46274e2cdedaad42fda5ce5cfb2da)

Fix timing leak in BN_from_montgomery_word.

BN_from_montgomery_word doesn't have a constant memory access pattern.
Replace the pointer trick with a constant-time select. There is, of
course, still the bn_correct_top leak pervasive in BIGNUM itself.

See also https://boringssl-review.googlesource.com/22904 from BoringSSL.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/5228)

(cherry picked from commit f345b1f39d9b4e4c9ef07e7522e9b2a870c9ca09)

Don't leak the exponent bit width in BN_mod_exp_mont_consttime.

The exponent here is one of d, dmp1, or dmq1 for RSA. This value and its
bit length are both secret. The only public upper bound is the bit width
of the corresponding modulus (RSA n, p, and q, respectively).

Although BN_num_bits is constant-time (sort of; see bn_correct_top notes
in preceding patch), this does not fix the root problem, which is that
the windows are based on the minimal bit width, not the upper bound. We
could use BN_num_bits(m), but BN_mod_exp_mont_consttime is public API
and may be called with larger exponents. Instead, use all top*BN_BITS2
bits in the BIGNUM. This is still sensitive to the long-standing
bn_correct_top leak, but we need to fix that regardless.

This may cause us to do a handful of extra multiplications for RSA keys
which are just above a whole number of words, but that is not a standard
RSA key size.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5154)

(cherry picked from commit 39eeb64f59ff838f976ad305de7d15747d47a41c)

Make BN_num_bits_word constant-time.

(This patch was written by Andy Polyakov. I only wrote the commit
message. Mistakes in the analysis are my fault.)

BN_num_bits, by way of BN_num_bits_word, currently leaks the
most-significant word of its argument via branching and memory access
pattern.

BN_num_bits is called on RSA prime factors in various places. These have
public bit lengths, but all bits beyond the high bit are secret. This
fully resolves those cases.

There are a few places where BN_num_bits is called on an input where the
bit length is also secret. This does *not* fully resolve those cases as
we still only look at the top word. Today, that is guaranteed to be
non-zero, but only because of the long-standing bn_correct_top timing
leak. Once that is fixed, a constant-time BN_num_bits on such inputs
must count bits on each word.

Instead, those cases should not call BN_num_bits at all. In particular,
BN_mod_exp_mont_consttime uses the exponent bit width to pick windows,
but it should be using the maximum bit width. The next patch will fix
this.

Thanks to Dinghao Wu, Danfeng Zhang, Shuai Wang, Pei Wang, and Xiao Liu
for reporting this issue.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5154)

(cherry picked from commit 972c87dfc7e765bd28a4964519c362f0d3a58ca4)

Add OPENSSL_VERSION_AT_LEAST

added macro to create version number
use the macro to build OPENSSL_VERSION_AT_LEAST(maj,min,fix) so that
customers of libssl (such as ruby-openssl) do not need to be so aware of
openssl version numbers.
includes updates to ssl(7) and OPENSSL_VERSION_NUMBER(3) man page

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5212)
(cherry picked from commit 3c5a61dd0f9d9a9eac098419bcaf47d1c296ca81)

Remove "dummy" BIO create and destroy functions

They aren't needed if all they do is set bio->init = 1 and zero other
fields that are already zeroed

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5223)

(cherry picked from commit 94f1c9379c3ed4b088718b35d0dd807d619d50c5)

BIO: at the end of BIO_new, declare the BIO inited if no create method present

Without this, every BIO implementation is forced to have a create
method, just to set bio->init = 1.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5223)

(cherry picked from commit 7f55808fe723c146428415a470ed42331548007b)

Update CHANGES with info about SSL_OP_NO_RENGOTIATION

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4901)

Add the SSL_OP_NO_RENEGOTIATION option to 1.1.0

This is based on a heavily modified version of commit db0f35dda by Todd
Short from the master branch.

We are adding this because it used to be possible to disable reneg using
the flag SSL3_FLAGS_NO_RENEGOTIATE_CIPHERS in 1.0.2. This is no longer
possible because of the opacity work.

A point to note about this is that if an application built against new
1.1.0 headers (that know about the new option SSL_OP_NO_RENEGOTIATION
option) is run using an older version of 1.1.0 (that doesn't know about
the option) then the option will be accepted but nothing will happen, i.e.
renegotiation will not be prevented. There's probably not much we can do
about that.

Fixes #4739

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4901)

Make sure we check an incoming reneg ClientHello in DTLS

In TLS we have a check to make sure an incoming reneg ClientHello is
acceptable. The equivalent check is missing in the DTLS code. This means
that if a client does not signal the ability to handle secure reneg in the
initial handshake, then a subsequent reneg handshake should be rejected by
the server. In the DTLS case the reneg was being allowed if the the 2nd
ClientHello had a renegotiation_info extension. This is incorrect.

While incorrect, this does not represent a security issue because if
the renegotiation_info extension is present in the second ClientHello it
also has to be *correct*. Therefore this will only work if both the client
and server believe they are renegotiating, and both know the previous
Finished result. This is not the case in an insecure rengotiation attack.

I have also tidied up the check in the TLS code and given a better check
for determining whether we are renegotiating or not.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5191)

Fix strict-warnings build on FreeBSD

The cryptodev engine is only available for OpenBSD and FreeBSD,
but for the OS version-specific checks the OpenBSD macro is not
defined on FreeBSD.  When building with -Werror and -Wundef (enabled
by strict-warnings), the FreeBSD build fails:

crypto/engine/eng_cryptodev.c:47:7: error: 'OpenBSD' is not defined,
evaluates to 0
      [-Werror,-Wundef]
\# if (OpenBSD >= 200112) || ((__FreeBSD_version >= 470101 &&
\# __FreeBSD_versi...
      ^

The reverse case would be true on OpenBSD (__FreeBSD_version would
not be defined), but since the boolean will short-circuit and this
code is only executed on OpenBSD and FreeBSD, and the line is
already pretty long, leave that out for now.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5160)

Don't define OPENSSL_NO_ERR for the command line

It's already in opensslconf.h, which is included where this is relevant.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5188)

(cherry picked from commit 711a8b999e7a0733f349e7f330fa3fa013e995fd)

Remove $no_sse2, as it's just a 'copy' of $disabled{sse2}

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5188)

(cherry picked from commit 0d59958c4999a456531835d0b61ba4eaecceaf86)

Don't break testing when runnins as root

The rehash test broke the test if run by root.  Instead, just skip the
check that requires non-root to be worth it.

Fixes #4387

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5184)

(cherry picked from commit 98ade24200f127a158b60bc736390c587cdd3dfb)

VMS config.com: better handling of arguments

Most of all, this change preserves casing a bit better

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5180)

(cherry picked from commit d4deecc203bf3a7bcc5f31529692f5b668c0c2bc)

Fix some style nits in commit eee8a40

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5173)

(cherry picked from commit df05f155a69b96b174e0e47bc689bf729de684b6)

Check # of arguments for remaining commands.

Backport of https://github.com/openssl/openssl/pull/4201

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5162)

Make the s_server command listen on IPv6 only when requested

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5152)

(cherry picked from commit eee8a40aa5e06841eed6fa8eb4f6109238d59aea)

Fix setting of IPV6_V6ONLY on Windows

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5156)

Fix BN doc

Backport from https://github.com/openssl/openssl/pull/5141

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5151)

Fix signature of min/max proto getter

The getters for min and max proto version wrongly passed NULL instead of
0 as third argument to SSL_ctrl() and SSL_CTX_ctrl(). The third argument
is not used, but the error results in a compiler warning:

warning: passing argument 3 of ‘SSL_CTX_ctrl’ makes integer from pointer without a cast [-Wint-conversion]
int v = SSL_CTX_get_max_proto_version(self->ctx);

See https://github.com/openssl/openssl/pull/4364

Signed-off-by: Christian Heimes <christian@python.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5128)

(cherry picked from commit 1f82eba718f758757a439e717ea7a5aa8be55f8d)

Fix SSL_CTX_get_{min,max}_proto_version integer conversion warning

When using the SSL_CTX_get_min_min_version macro while compiling with
-Wall, my compiler rightfully complains about this construction:

warning: passing argument 3 of ‘SSL_CTX_ctrl’ makes integer from
pointer without a cast [-Wint-conversion]

These macro's should use 0, instead of NULL, for the third argument,
like most other SSL_CTX_ctrl 'get' wrappers do.

CLA: trivial
Signed-off-by: Steffan Karger <steffan.karger@fox-it.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5099)

(cherry picked from commit 78d0afaa69c2ec662f6c97b47473c2c6a69d636c)

Fix error-path memory leak in asn_mime.c

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5142)

(cherry picked from commit a26dd465b21d8def440c16b6bd90227b03e12e02)

Configure: ensure that a DEPEND generates the correct inclusion directory

We incorrectly assumed that explicit dependencies meant that the
source directory would be added for inclusion.  However, if the
dependent file is generated, it's stored in the build directory, and
that should be used for inclusion rather than the source directory.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5153)

(cherry picked from commit e431bcfabd72163b0ee89674e2face26a349ba9c)

Configure: let INCLUDEs set on binaries "trickle down" to the objects

This ensures that only one set of includes is associated with each
object file, reagardless of where it's used.

For example, if apps/build.info has this:

    SOURCE[openssl]=foo.c
    INCLUDE[openssl]=.. ../include

and test/build.info has this:

    SOURCE[footest]=../apps/foo.c
    INCLUDE[footest]=../include

The inclusion directories used for apps/foo.o would differ depending
on which program's dependencies get generated first in the build file.

With this change, all those INCLUDEs get combined into one set of
inclusion directories tied to the object file.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5153)

(cherry picked from commit 1b5ad51fc9b29d8893d5224f00bb3360f8aca465)

Have EVP_PKEY_asn1_find_str() work more like EVP_PKEY_asn1_find()

EVP_PKEY_asn1_find_str() would search through standard asn1 methods
first, then those added by the application, which EVP_PKEY_asn1_find()
worked the other way around.  Also, EVP_PKEY_asn1_find_str() didn't
handle aliases.

This change brings EVP_PKEY_asn1_find_str() closer to EVP_PKEY_asn1_find().

Fixes #5086

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5137)

(cherry picked from commit 3bf0c3fe31d5339524dae671064cc5fe9e4bda38)

Revert "EVP_PKEY_asn1_add0(): Check that this method isn't already registered"

This reverts commit d85722d31ac9ff0dc54c06cdc8d125acf56ca27a.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5137)

(cherry picked from commit 7203c94e98c9fa76b0859c25b723b2bde4a9059b)

Don't add $(EX_LIBS) to libssl.pc's Libs.private

Since libssl requires libcrypto and libcrypto.pc already has
Libs.private set exactly the same, there's no reason to repeat it in
libssl.pc.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5135)

Add anything specifying a threads library to ex_libs

Even -pthread gets treated that way.  The reason to do this is so it
ends up in 'Libs.private' in libcrypto.pc.

Fixes #3884

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5135)

Don't attempt to use X25519 for ECDSA in speed

Fixes #5090

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5097)

(cherry picked from commit da1a1b5ac417c4e38f7717082d69cb5f0ceda6b9)

Don't crash on a missing Subject in index.txt

An index.txt entry which has an empty Subject name field will cause ca
to crash. Therefore check it when we load it to make sure its not empty.

Fixes #5109

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5114)

(cherry picked from commit 1e05c6d07ff963107286d028f6778d2ccc863a9a)

Don't allow an empty Subject when creating a Certificate

Misconfiguration (e.g. an empty policy section in the config file) can
lead to an empty Subject. Since certificates should have unique Subjects
this should not be allowed.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5114)

(cherry picked from commit e505f1e86874acfd98826d64c53bf2ddfd9c1399)

Add a configure option to opt-out secure memory

./config -DOPENSSL_NO_SECURE_MEMORY

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5113)

(cherry picked from commit 154d8c132fbe22a248f95e95ef21f1840451da62)

Update the license end year

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5121)

(cherry picked from commit 7c24f9d21cddd2bb30167153b05168fee7e3cf0b)

Enable TLSProxy tests on Windows

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5096)

Set OPENSSL_ENGINES for Windows

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5108)

(cherry picked from commit 7a6ce9039cb6690aae3cca01f18daab16b8e30d7)

Create one permanent proxy socket per TLSProxy::Proxy instance

On Windows, we sometimes see a behavior with SO_REUSEADDR where there
remains lingering listening sockets on the same address and port as a
newly created one.

To avoid this scenario, we don't create a new proxy port for each new
client run.  Instead, we create one proxy socket when the proxy object
is created, and close it when destroying that object.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5095)

(cherry picked from commit c7454e1af74b1b99f3f47f782a6ac484c4c55b7f)

Only implement secure malloc if _POSIX_VERSION allows

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5060)

(cherry picked from commit e44c7d02ddac975ec6abff7901e77a0c37f9949d)

The Cygwin gcc doesn't define _WIN32, don't pretend it does

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5060)

(cherry picked from commit 4d3c278c1a4f940e098edac6de3d370b94d1bb8b)

Simplify Cygwin checks, part 1

Because OPENSSL_SYS_CYGWIN will keep OPENSSL_SYS_UNIX defined, there's
no point having checks of this form:

    #if (defined(OPENSSL_SYS_UNIX) || defined(OPENSSL_SYS_CYGWIN))

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5060)

(cherry picked from commit 6ce66ce073b19af46fd94f4eefdb7c4772d7dd6f)

Cygwin is POSIX, don't say it isn't

More to the point, Cygwin is a POSIX API.  In our library, the use of
a POSIX API is marked by defining the macro OPENSSL_SYS_UNIX.
Therefore, that macro shouldn't be undefined when building for Cygwin.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5060)

(cherry picked from commit 33cebb9c0dd6115bc357b5097f7db82b64b10919)

TLSProxy::Proxy: Don't use ReuseAddr on Windows

On Windows, we sometimes see a behavior with SO_REUSEADDR where there
remains lingering listening sockets on the same address and port as a
newly created one.

An easy solution is not to use ReuseAddr on Windows.

Thanks Bernd Edlinger for the suggestion.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5103)

(cherry picked from commit e02d5886636095c26a8bff1bf8344bd0bba7ccff)

TLSProxy::Proxy: don't waste time redirecting STDOUT and STDERR

On Windows, it seems that doing so in a forked (pseudo-)process
sometimes affects the parent, and thereby hides all the results that
are supposed to be seen by the running test framework (the "ok" and
"not ok" lines).

It turns out that our redirection isn't necessary, as the test
framework seems to swallow it all in non-verbose mode anyway.
It's possible that we did need this at some point, but the framework
has undergone some refinement since then...

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5100)

(cherry picked from commit 825038d8c36075488922617e48fc4782afe871d1)

TLSProxy::Proxy:  If we don't support IPv6, force IPv4

We use the first we can of the following IO::Socket modules to create
sockets:

- IO::Socket::INET6
- IO::Socket::IP
- IO::Socket::INET

The last of them doesn't support IPv6, so if that's the one available,
we must force the s_client and s_server processes to use IPv4.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5092)

(cherry picked from commit c44bab084ed139b5e5c3f6b318bf1bf65e04cfa5)

Extend timeout for TLSProxy

I received this error from a TLSProxy test:

Failed to start up server (localhost,4443): Transport endpoint is not
connected

So, extend the timeout before we give up trying to connect to the server.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5088)

(cherry picked from commit 82992a4130e668c14ff66fca26170629b23c4556)

Revert BN_copy() flag copy semantics change

Commit 9f9442918a changed the semantics of BN_copy() to additionally
copy the BN_FLG_CONSTTIME flag if it is set. This turns out to be
ill advised as it has unintended consequences. For example calling
BN_mod_inverse_no_branch() can sometimes return a result with the flag
set and sometimes not as a result. This can lead to later failures if we
go down code branches that do not support constant time, but check for
the presence of the flag.

The original commit was made due to an issue in BN_MOD_CTX_set(). The
original PR fixed the problem in that function, but it was changed in
review to fix it in BN_copy() instead. The solution seems to be to revert
the BN_copy() change and go back to the originally proposed way.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/5080)

(cherry picked from commit 7d461736f7bd3af3c2f266f8541034ecf6f41ed9)

Fix intermittent Windows and Cygwin failures in s_server

The same kind of failure that has already been observed on the
s_client can sometimes also be observed on s_server, so we need to add
the same kind of 50ms delay as was previously added on s_client.

Ref: git commit cb2e10f257a464c6b475b321dd9e4769df84dbf6:

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5079)

(cherry picked from commit d17bdfc26d87097c2c80854fd521b6ecebdf5ebb)

Fix intermittent Cygwin failures in s_client

This was identified for Windows almost two years ago for VC and
msys/mingw.  It seems that Cygwin suffers from the same issue, and
since Cygwin doesn't define OPENSSL_SYS_WINDOWS, we need to make a
special case to have a 50ms pause before closing the TLS connection.

Ref: git commit cb2e10f257a464c6b475b321dd9e4769df84dbf6

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5074)

(cherry picked from commit 27da42d85212f80fc0db02b1cedf1d49fbfb8c43)

Use constant value 1 instead of SHUT_WR in do_server

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5072)

(cherry picked from commit 803141f6100ca8d756928328a262e1dc52daef2d)

Explicitly shut the socket down in s_client

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5072)

(cherry picked from commit 26ec943e020c0db6a25e6d155ba318270eff0fd7)

Catch SIGPIPE in TLSProxy::Proxy::clientstart

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5071)

(cherry picked from commit 438e57a43f096d37bf316869b98780e24596a5e8)

Make data argument const in SSL_dane_tlsa_add

The data argument of SSL_dane_tlsa_add is used read-only, so it
should be const.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5056)

(cherry picked from commit a41a6120cdcb7e883481bc1bed55e7157c9255c4)

Tolerate DTLS alerts with an incorrect version number

In the case of a protocol version alert being sent by a peer the record
version number may not be what we are expecting. In DTLS records with an
unexpected version number are silently discarded. This probably isn't
appropriate for alerts, so we tolerate a mismatch in the minor version
number.

This resolves an issue reported on openssl-users where an OpenSSL server
chose DTLS1.0 but the client was DTLS1.2 only and sent a protocol_version
alert with a 1.2 record number. This was silently ignored by the server.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5018)

(cherry picked from commit 08455bc9b0e69ed5f25c16fc30cc2db57cdca842)

Configure: try to make sure every config target name is unique

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5047)

(cherry picked from commit 225f980d22dfe66aa3945c21609042c32284e61c)

Update copyright years on all files merged since Jan 1st 2018

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5039)

Various small build improvements on mkdef.pl, progs.pl

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4994)

(cherry picked from commit ab307dc645502d025c2760160062d5032da56bf7)

Document OPENSSL_ENGINES environment variable

In apps/engine.pod and crypto/ENGINE_add.pod

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4956)

(cherry picked from commit c81c38cb2786dd308256c55ce8d5337e6a3f14ce)

Fix spelling: adroideabi -> androideabi

CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5029)

(cherry picked from commit bebef673607b9701bc21a0f7b7dbf92434af2588)

Clean up uClinux targets

The uClinux targets included some attributes that would result in
circular references of CFLAGS and LDCLAGS.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5034)

(cherry picked from commit f720194cbc174806fe1ce9c349edd249d8076c02)

CHANGES: Document the removal of OS390-Unix

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5031)

(cherry picked from commit 4b9e91b02bee0dd114260513cb6a3f28261124b7)

Add fingerprint text, remove MD5

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4906)
(Cherry-picked from commit 9422d45de2b70cabec5f6e6a5c812e0647e6d3ab)