Fix race condition in TLSProxy

Normally TLSProxy waits for the s_server process to finish before
continuing. However in cases where serverconnects > 1 we need to keep the
s_server process around for a later test so we continue immediately. This
means that TAP test output can end up being printed to stdout at the same
time as s_server is printing stuff. This confuses the test runner and can
cause spurious test failures. This commit introduces a small delay in cases
where serverconnects > 1 in order to give s_server enough time to finish
what it was doing before we continue to the next test.

Fixes #4129

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4661)

Remove 4 broken macros from ocsp.h

There were 4 macros in ocsp.h that have not worked since 1.1.0 because
they attempt to access the internals of an opaque structure.

For OCSP_REQUEST_sign() applications should use OCSP_request_sign() instead.
For OCSP_BASICRESP_sign() applications should use OCSP_basic_sign() instead.
For OCSP_REQUEST_verify() applications should use OCSP_request_verify()
instead.
For OCSP_BASICRESP_verify() applications should use OCSP_basic_verify()
instead.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4635)

(cherry picked from commit 9f5671c7e9f30dfa53b1a2b553f234c2761ceb66)

Consolidate the locations where we have our internal perl modules

Instead of having perl modules under test/testlib and util,
consolidate them all to be inside util/perl.

(this is an adaptation of the part of #4069 that wasn't included in #4666)

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4667)

Perl: Use our own globbing wrapper rather than File::Glob::glob

File::Glob::glob is deprecated, it's use generates this kind of
message:

    File::Glob::glob() will disappear in perl 5.30. Use File::Glob::bsd_glob() instead. at ../master/Configure line 277.

The first idea was to use a construction that makes the caller glob()
use File::Glob::bsd_glob().  That turned out not to work well
everywhere, so instead, we make our own wrapper, OpenSSL::Glob and use
that.

Fixes #4636

(this is an adaptation of #4040 and part of #4069, for 1.1.0)

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4666)

aes/asm/{aes-armv4|bsaes-armv7}.pl: make it work with binutils-2.29.

It's not clear if it's a feature or bug, but binutils-2.29[.1]
interprets 'adr' instruction with Thumb2 code reference differently,
in a way that affects calculation of addresses of constants' tables.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4669)

(cherry picked from commit b82acc3c1a7f304c9df31841753a0fa76b5b3cda)

Spelling doc #3580
Duplicated tests descriptions

Backport of #3580 to 1.1.0
plus a few other typo fixes found at fligth.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4645)

Add error handling in dsa_main and ASN1_i2d_bio.

CLA: trivial

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4600)

(cherry picked from commit a6f622bc99ffdc7b34199babb9d200b24a7a6431)

Check return value of OBJ_nid2obj in dsa_pub_encode.

CLA: trivial

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4600)

(cherry picked from commit 7760384b403a61824c43cc767a11cd22abfa9e49)

Travis: if "make update" created a diff, please show it

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4652)

(cherry picked from commit d7948767556e68378b75196841b3d32dd70d169a)

Prepare for 1.1.0h-dev

Reviewed-by: Andy Polyakov <appro@openssl.org>

Prepare for 1.1.0g release

Reviewed-by: Andy Polyakov <appro@openssl.org>

Update CHANGES and NEWS for new release

Reviewed-by: Andy Polyakov <appro@openssl.org>

bn/asm/x86_64-mont5.pl: fix carry bug in bn_sqrx8x_internal.

Credit to OSS-Fuzz for finding this.

CVE-2017-3736

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix small but important regression

In OpenSSL pre 1.1.0, 'openssl x509 -CAkeyformat engine' was possible
and supported.  In 1.1.0, a small typo ('F' instead of 'f') removed
that possibility.  This restores the pre 1.1.0 behavior.

Fixes #4366

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4643)

(cherry picked from commit bd6eba79d70677f891f1bb55b6f5bc5602c47cbc)

Address a timing side channel whereby it is possible to determine some

information about the length of the scalar used in ECDSA operations
from a large number (2^32) of signatures.

This doesn't rate as a CVE because:

* For the non-constant time code, there are easier ways to extract
  more information.

* For the constant time code, it requires a significant number of signatures
  to leak a small amount of information.

Thanks to Neals Fournaise, Eliane Jaulmes and Jean-Rene Reinhard for
reporting this issue.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4576)

(cherry picked from commit 4a089bbdf11f9e231cc68f42bba934c954d81a49)

 Address a timing side channel whereby it is possible to determine some

information about the length of a value used in DSA operations from
a large number of signatures.

This doesn't rate as a CVE because:

* For the non-constant time code, there are easier ways to extract
  more information.

* For the constant time code, it requires a significant number of signatures
  to leak a small amount of information.

Thanks to Neals Fournaise, Eliane Jaulmes and Jean-Rene Reinhard for
reporting this issue.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4576)

(cherry picked from commit c0caa945f6ef30363e0d01d75155f20248403df4)

Travis: Add a docs checking job

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4631)

docs: assign section 7 where appropriate

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4633)

doc/crypto/OPENSSL_secure_malloc: add missing names

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4633)

docs: fixup OpenSSL version style

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4633)

Adapt util/find-doc-nits back to 1.1.0

This version was a direct port from 1.1.1-dev, which has a different
source structure for the docs.  Adjustment done.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4633)

Fix EVP_PKEY_ASN1_METHOD manual

Missing names slipped through

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4630)

afalg: Fix kernel version check

The check should reject kernel versions < 4.1.0, not <= 4.1.0.

The issue was spotted on OpenSUSE 42.1 Leap, since its linux/version.h
header advertises 4.1.0.

CLA: trivial
Fixes: 7f458a48 ("ALG: Add AFALG engine")
Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4618)

afalg: Use eventfd2 syscall instead of eventfd

The eventfd syscall is deprecated and is not available on aarch64, causing
build to fail:

    engines/e_afalg.c: In function 'eventfd':
    engines/e_afalg.c:108:20: error: '__NR_eventfd' undeclared (first use in this function)
         return syscall(__NR_eventfd, n);
                        ^

Instead, switch to the newer eventfd2 syscall, which is supposed to be
supported by all architectures.

This kind of issues would be avoided by simply using the eventfd(2)
wrapper from the libc, but there must be subtle reasons not to...

Tested on a aarch64 system running OpenSUSE Leap 42.1 (gcc118 from
https://cfarm.tetaneutral.net/machines/list/ ) and also cross-compiling
for aarch64 with LEDE (kernel 4.9).

This properly fixes #1685.

CLA: trivial
Fixes: 7f458a48 ("ALG: Add AFALG engine")
Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4618)

EVP_PKEY_asn1_add0(): Check that this method isn't already registered

No two public key ASN.1 methods with the same pkey_id can be
registered at the same time.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4620)

Document EVP_PKEY_ASN1_METHOD and associated functions

[skip ci]

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4620)

Only reset the ctx when a cipher is given

This restores the 1.0.2 behaviour

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Benjamin Kaduk <bkaduk@akamai.com>
GH: #4613
(cherry picked from commit ffd23209933ea0ad5543f15ca6303d63d8dac826)

Add missing paren.

Thanks to Remi Gacogne for pointing this out.
Also indented the two macro bodies

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4608)

x509v3/v3_utl.c: avoid double-free.

Thanks to David Benjamin for spotting this.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4532)

(cherry picked from commit 432f8688bb72e21939845ac7a69359ca718c6676)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4514)

crypto/x509v3/v3_utl.c: fix Coverity problems.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4492)

(cherry picked from commit 32f3b98d1302d4c0950dc1bf94b50269b6edbd95)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4514)

Don't use strcasecmp and strncasecmp for IA5 strings

The functions strcasecmp() and strncasecmp() will use locale specific rules
when performing comparison. This could cause some problems in certain
locales. For example in the Turkish locale an 'I' character is not the
uppercase version of 'i'. However IA5 strings should not use locale specific
rules, i.e. for an IA5 string 'I' is uppercase 'i' even if using the
Turkish locale.

This fixes a bug in name constraints checking reported by Thomas Pornin
(NCCGroup).

This is not considered a security issue because it would require both a
Turkish locale (or other locale with similar issues) and malfeasance by
a trusted name-constrained CA for a certificate to pass name constraints
in error. The constraints also have to be for excluded sub-trees which are
extremely rare. Failure to match permitted subtrees is a bug, not a
vulnerability.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4569)

(cherry picked from commit 9cde5f81222fd491d6d56eb8f37ab9c40a26f745)

Fix doc-nits in doc/man3/DEFINE_STACK_OF.pod

<compar> to <compare> to match the var name in function prototype

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4559)

(cherry picked from commit d9c989fe3f137580ee627c91e01245e78b0b41ff)

doc/man3/d2i_X509.pod: add {d2i,i2d}_DSA_PUBKEY in NAME section

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4584)

(cherry picked from commit 82d89ef72515ad3d78c0160641faf30b8b024dda)

asn1_item_embed_new(): if locking failed, don't call asn1_item_embed_free()

asn1_item_embed_free() will try unlocking and fail in this case, and
since the new item was just allocated on the heap, free it directly
with OPENSSL_free() instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4579)

(cherry picked from commit fe6fcd31546db1ab019e55edd15c953c5b358559)

asn1_item_embed_new(): don't free an embedded item

The previous change with this intention didn't quite do it.  An
embedded item must not be freed itself, but might potentially contain
non-embedded elements, which must be freed.

So instead of calling ASN1_item_ex_free(), where we can't pass the
embed flag, we call asn1_item_embed_free() directly.

This changes asn1_item_embed_free() from being a static function to
being a private non-static function.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4579)

(cherry picked from commit 03996c19c30575c48b254f10625d24f86058605b)

Don't make any changes to the lhash structure if we are going to fail

The lhash expand() function can fail if realloc fails. The previous
implementation made changes to the structure and then attempted to do a
realloc. If the realloc failed then it attempted to undo the changes it
had just made. Unfortunately changes to lh->p were not undone correctly,
ultimately causing subsequent expand() calls to increment num_nodes to a
value higher than num_alloc_nodes, which can cause out-of-bounds reads/
writes. This is not considered a security issue because an attacker cannot
cause realloc to fail.

This commit moves the realloc call to near the beginning of the function
before any other changes are made to the lhash structure. That way if a
failure occurs we can immediately fail without having to undo anything.

Thanks to Pavel Kopyl (Samsung) for reporting this issue.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4550)

(cherry picked from commit 4ce8bebcca90a1f8a3347be29df7a501043d4464)

Fix memory leak in GENERAL_NAME_set0_othername.

CLA: trivial

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4544)

(cherry picked from commit 04761b557a53f026630dd5916b2b8522d94579db)

asn1_item_embed_new(): don't free an embedded item

An embedded item wasn't allocated separately on the heap, so don't
free it as if it was.

Issue discovered by Pavel Kopyl

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4572)

(cherry picked from commit 590bbdfdf43b97abf8817f506f8ab46687d1eadd)

Correct value for BN_security_bits()

The function BN_security_bits() uses the values from SP800-57 to assign
security bit values for different FF key sizes. However the value for 192
security bits is wrong. SP800-57 has it as 7680 but the code had it as
7690.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4546)

(cherry picked from commit c9fe362303fc54ff19bde7511475f28663f7d554)

s390x assembly pack: define OPENSSL_s390xcap_P in s390xcap.c

Remove all .comm definitions from the asm modules.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4563)

ECDSA_* is deprecated. EC_KEY_* is used instead

CLA: trivial

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Ben Laurie <ben@links.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4561)
(cherry picked from commit 9b02dc97e4963969da69675a871dbe80e6d31cda)

Additional name for all commands

Add openssl-foo as a name for the openssl "foo" command.
Addresses an issue found by a usability study to be published.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4553)

(cherry picked from commit 3f2181e6fadea9e7ad8810b3f170fd0b2154e8b8)

s390x assembly pack: remove capability double-checking.

An instruction's QUERY function is executed at initialization, iff the required
MSA level is installed. Therefore, it is sufficient to check the bits returned
by the QUERY functions. The MSA level does not have to be checked at every
function call.
crypto/aes/asm/aes-s390x.pl: The AES key schedule must be computed if the
required KM or KMC function codes are not available. Formally, the availability
of a KMC function code does not imply the availability of the corresponding KM
function code.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4501)

(cherry picked from commit af1d638730bdfad85a7fa8c3f157b2828eda7c1d)

crypto/aes/asm/aes-s390x.pl: fix $softonly=1 code path.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4501)

(cherry picked from commit 4c5100ce7d66ccff48d6435c1761b5e3281de61f)

Update RAND_load_file return value.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4537)

(cherry picked from commit fe7a4d7c4c8148f732bc47ef7585f4aa41b7391a)

Backport key redirection test from master branch

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4520)

Skip ssl-tests/19-mac-then-encrypt.conf for no-tls1_2

The second set of tests in that configuration uses the AES-SHA256
ciphers, which are only available for TLS 1.2.  Thus, when TLS 1.2
is disabled, there are no ciphers available and the handshake fails
with an internal error.  Apply the same treatment as for
13-fragmentation.conf, which uses the same ciphers.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4526)

Document EVP_PKEY_set1_engine()

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4503)

(cherry picked from commit 8e826a339f8cda20a4311fa88a1de782972cf40d)

make update

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4503)

Add EVP_PKEY_set1_engine() function.

Add an ENGINE to EVP_PKEY structure which can be used for cryptographic
operations: this will typically be used by an HSM key to redirect calls
to a custom EVP_PKEY_METHOD.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4503)

(cherry picked from commit d19b01ad79f9e2aac5c87496b5ca5f80016daeb7)

Fix memory leak on lookup failure

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4503)

(cherry picked from commit 918a27facd3558444c69b1edbedb49478e82dff5)

Don't ignore passed ENGINE.

If we are passed an ENGINE to use in int_ctx_new e.g. via EVP_PKEY_CTX_new()
use it instead of the default.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4503)

(cherry picked from commit c2976edf4b22691d8bebb0e3ca2db18b3d0c71c6)

Ensure we test all parameters for BN_FLG_CONSTTIME

RSA_setup_blinding() calls BN_BLINDING_create_param() which later calls
BN_mod_exp() as follows:

BN_mod_exp(ret->A, ret->A, ret->e, ret->mod, ctx)

ret->mod will have BN_FLG_CONSTTIME set, but ret->e does not. In
BN_mod_exp() we only test the third param for the existence of this flag.
We should test all the inputs.

Thanks to Samuel Weiser (samuel.weiser@iaik.tugraz.at) for reporting this
issue.

This typically only happens once at key load, so this is unlikely to be
exploitable in any real scenario.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4477)

(cherry picked from commit e913d11f444e0b46ec1ebbf3340813693f4d869d)

Reduce the things we ignore in test/

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4507)

(cherry picked from commit d2068e34d1e6b19daa6aba32bc7c6393699c9371)

Use the possibility to have test results in a different directory

RESULT_D can be used to provide a separate directory for test results.
Let's use that to separate them from other files.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4507)

(cherry picked from commit 41f571e10c31cd58aada3cfde3be6a8a94cea64a)

Fix util/perl/OpenSSL/Test.pm input variable overwrite

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4507)

(cherry picked from commit 9b9a8a712d64e35a337b22869288f246b5580c73)

Fix parameter name, for common aesthetics and to silence IDE warnings.

CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4494)

Fix util/find-doc-nits to correctly parse function signature typedefs

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4499)

(cherry picked from commit 0ed78e78007bb74e48e6f59fa2388bb244153bf0)

Correct some typedef documentation

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4499)

(cherry picked from commit 5bf6d418034a246bd3680d648c22e2c4500a3e0a)

Fix doc for i2d/d2i private/public key

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4491)

(cherry picked from commit 24b0be11b061f36d30ccccdf9d34edf270be4c2f)

doc/apps/openssl.pod: Add missing commands and links

Fixes #4471 and more

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4473)

Test mac-then-encrypt

Verify that the encrypt-then-mac negotiation is handled
correctly. Additionally, when compiled with no-asm, this test ensures
coverage for the constant-time MAC copying code in
ssl3_cbc_copy_mac. The proxy-based CBC padding test covers that as
well but it's nevertheless better to have an explicit handshake test
for mac-then-encrypt.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(cherry picked from commit b3618f44a7b8504bfb0a64e8a33e6b8e56d4d516)

Add SSL_OP_NO_ENCRYPT_THEN_MAC

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit cde6145ba19a2fce039cf054a89e49f67c623c59)

Remove an incorrect comment

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4456)

(cherry picked from commit 786b4df402ce57e375012401a02ad7a6696b90c2)

Configurations/windows-makefile.tmpl: canonicalise configured paths

This avoids issues that can come with an ending backslash, among other.

Fixes #4458

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4461)

(cherry picked from commit dc6a62d5d5de905776433ab8ab6b1b2fffaae1ea)

Fix the return type of felem_is_zero_int which should be int.
Change argument type of xxxelem_is_zero_int to const void*
to avoid the need of type casts.

Fixes #4413

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4450)

(cherry picked from commit c55b786a8911cef41f890735ba5fde79e116e055)

recipes/25-test_verify.t: reformat.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4411)

Guard against DoS in name constraints handling.

This guards against the name constraints check consuming large amounts
of CPU time when certificates in the presented chain contain an
excessive number of names (specifically subject email names or subject
alternative DNS names) and/or name constraints.

Name constraints checking compares the names presented in a certificate
against the name constraints included in a certificate higher up in the
chain using two nested for loops.

Move the name constraints check so that it happens after signature
verification so peers cannot exploit this using a chain with invalid
signatures. Also impose a hard limit on the number of name constraints
check loop iterations to further mitigate the issue.

Thanks to NCC for finding this issue. Fix written by Martin Kreichgauer.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4411)

(cherry picked from commit 8545051c3652bce7bb962afcb6879c4a6288bc67)

Resolved conflicts:
crypto/x509v3/v3_ncons.c
test/recipes/25-test_verify.t

Added const-time flag to DSA key decoding to avoid potential leak of privkey

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4440)

(cherry picked from commit 6364475a990449ef33fc270ac00472f7210220f2)

doc: note that the BN_new() initialises the BIGNUM

BN_new() and BN_secure_new() not only allocate memory, but also
initialise it to deterministic value - 0.

Document that behaviour to make it explicit

backport from #4438

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4442)

Allow DH_set0_key with only private key.

The pub_key field for DH isn't actually used in DH_compute_key at all.
(Note the peer public key is passed in as as BIGNUM.) It's mostly there
so the caller may extract it from DH_generate_key. It doesn't
particularly need to be present if filling in a DH from external
parameters.

The check in DH_set0_key conflicts with adding OpenSSL 1.1.0 to Node.
Their public API is a thin wrapper over the old OpenSSL one:
https://nodejs.org/api/crypto.html#crypto_class_diffiehellman

They have separate setPrivateKey and setPublicKey methods, so the public
key may be set last or not at all. In 1.0.2, either worked fine since
operations on DH objects generally didn't use the public key.  (Like
with OpenSSL, Node's setPublicKey method is also largely a no-op, but so
it goes.) In 1.1.0, DH_set0_key prevents create a private-key-only DH
object.

(cherry picked from commit d58ad9a2a287d1c0bc99ba63c997eed88cc161b5)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4425)

BN_copy now propagates BN_FLG_CONSTTIME

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4377)

(cherry picked from commit 9f9442918aeaed5dc2442d81ab8d29fe3e1fb906)

Fixed error in propagating BN_FLG_CONSTTIME flag through BN_MONT_CTX_set, which could lead to information disclosure on RSA primes p and q.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4377)

(cherry picked from commit 3de81a5912041a70884cf4e52e7213f3b5dfa747)

Make sure that a cert with extensions gets version number 2 (v3)

Fixes #4419

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4420)

(cherry picked from commit 4881d849da23528e19b7312f963d28916d9804b1)

Fix 'key' option in s_server can be in ENGINE keyform

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4405)

(cherry picked from commit 75c445e49bb3d22afe72b28ae67945a9f67091f6)

Remove dhparam from SSL_CONF list.

Avoid duplicate assertion by removing dhparam from SSL_CONF parameter list:
dhparam is handled manually by s_server.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4408)

Reenable s_server -dhparam option

This option was lost when converting to a table-driven option parser
in commit 7e1b7485706c2b11091b5fa897fe496a2faa56cc.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4398)

(cherry picked from commit 51ac82702dc91cabd3dbf890d8f65b285282c0ce)

Fix overflow in c2i_ASN1_BIT_STRING.

c2i_ASN1_BIT_STRING takes length as a long but uses it as an int.  Check
bounds before doing so. Previously, excessively large inputs to the
function could write a single byte outside the target buffer. (This is
unreachable as asn1_ex_c2i already uses int for the length.)

Thanks to NCC for finding this issue. Fix written by Martin Kreichgauer.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4385)

(cherry picked from commit 6b1c8204b33aaedb7df7a009c241412839aaf950)

Null pointer used.
Address coverity report of null pointer being dereferenced.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4381)

(cherry picked from commit 9be34ee5c8576539a929d5b396ad071aed525f43)

Provide getters for min/max proto version

OpenSSL 1.1.0 made SSL_CTX and SSL structs opaque and introduced a new
API to set the minimum and maximum protocol version for SSL_CTX with
TLS_method(). Add getters to introspect the configured versions:

  int SSL_CTX_get_min_proto_version(SSL_CTX *ctx);
  int SSL_CTX_get_max_proto_version(SSL_CTX *ctx);
  int SSL_get_min_proto_version(SSL *ssl);
  int SSL_get_max_proto_version(SSL *ssl);

NOTE: The getters do not resolv the version in case when the minimum or
maxium version are configured as '0' (meaning auto-select lowest and
highst version number).

Signed-off-by: Christian Heimes <christian@python.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(cherry picked from commit 3edabd3ccb7aac89af5a63cfb2378e33a8be05d7)

Updated for new manual page location and TLS 1.3.

(Merged from https://github.com/openssl/openssl/pull/4376)

Error out when forcing an unsupported TLS version

If the result of a SSL_{CTX_,}set_{min,max}_proto_version() call
leaves the min and max version identical, and support for that version
is compiled out of the library, return an error.  Such an object has
no hope of successfully completing a handshake, and this error may
be easier to decipher than the resulting handshake failure.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit c8feba723a33e15201009d716d9ead02e653dfe6)

Updated the cherry-pick to not reference TLS1_3_VERSION, which does
not exist on this branch.

(Merged from https://github.com/openssl/openssl/pull/4376)

Disable the EGD seeding meachanism when stdio is disabled

crypto/rand/rand_egd.c makes extensive use of stdio functions.  When
they are disabled, it makes sense to disable egd as well.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4358)

(cherry picked from commit 15a1bd0ab2950671686cea51f4218c8f3d92fad9)

Update rsautl.pod for typo

Fixes the typo
CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4354)

(cherry picked from commit f70c22eb23763c6dce050293cc1b9a0a234d72b2)

Allow an endpoint to read the alert data before closing the socket

If an alert gets sent and then we close the connection immediately with
data still in the input buffer then a TCP-RST gets sent. Some OSs
immediately abandon data in their input buffer if a TCP-RST is received -
meaning the alert data itself gets ditched. Sending a TCP-FIN before the
TCP-RST seems to avoid this.

This was causing test failures in MSYS2 builds.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4333)

(cherry picked from commit bac6abe18d28373e0d2d0666c411020404197337)

Fix error handling/cleanup

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4326)

(cherry picked from commit 180794c54e98ae467c4ebced3737e1ede03e320a)

Add checks for alloc failing.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4341)

(cherry picked from commit d3c3dfc5778ab2cca0d25c5959c8b814a334addb)

Fix OpenSSL::Test::Utils::config to actualy load the config data

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4319)

(cherry picked from commit 607f4d564f9540cda6cf5b127f2414625a11741a)

Configure: base compiler-specific decisions on pre-defines.

The commit subject is a bit misleading in sense that decisions affect
only gcc and gcc-alikes, like clang, recent icc...

This is back-port of 54cf3b981afcbbd3754c8ba1114ab6a658d86c08, GH#4281.

Reviewed-by: Rich Salz <rsalz@openssl.org>

crypto/cryptlib.c: mask more capability bits upon FXSR bit flip.

OPENSSL_ia32cap.pod discusses possibility to disable operations on
XMM register bank. This formally means that this flag has to be checked
in combination with other flags. But it customarily isn't. But instead
of chasing all the cases we can flip more bits together with FXSR one.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4303)

(cherry picked from commit 6e5a853bda24e8aece325ecf5aa68b8ea832e414)

util/mkdef.pl: handle line terminators correctly

When parsing the header files, mkdef.pl didn't clear the line
terminator properly.  In most cases, this didn't matter, but there
were moments when this caused parsing errors (such as CRLFs in certain
cases).

Fixes #4267

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4304)

(cherry picked from commit e66b62b86e7725bdace0f24a76baa61db9c763f8)

Fixed address family test error for AF_UNIX in BIO_ADDR_make

CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4298)

(cherry picked from commit 177503752b24299cc97ccf07062a3b79c4f28899)

Avoid out-of-bounds read

Fixes CVE 2017-3735

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4276)

(cherry picked from commit b23171744b01e473ebbfd6edad70c1c3825ffbcd)

Remove NO_DIRENT; it isn't used anywhere

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4261)

(cherry picked from commit 32c1356302e74dfa5e8bd2d7002c18d91a323b70)

Clear secret stack values after use in curve25519.c

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4242)

(cherry picked from commit 78f1e4d0b063e17c9700f2aceecaca03bfa434f3)

NO_SYS_TYPES_H isn't defined anywhere, stop using it as a guard

This is a vestige from pre-1.1.0 OpenSSL

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4256)

(cherry picked from commit b379fe6cd046b9dd8a62309dcbaded763e2d4187)

Fix description of how to report a bug in INSTALL

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4259)

(cherry picked from commit 0a8ddc17f57691c8e2e2446c4126fb4133d07d21)

Clarify the meaning of no-stdio in INSTALL

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4259)

(cherry picked from commit 219b4643e40ada993730c55ae2c09815f89b4a2d)

Add documentation for SRTP functions

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4205)

(cherry picked from commit 3733ce61a2a0933bf7b04d9a14bfe3ac22bb8a0d)

Correct GCM docs.

Fix GCM documentation: the tag does not have to be supplied before
decrypting any data any more.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4231)

Tweak wording to be more clear.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4234)
(cherry picked from commit a130950df92abf7dd787b000403da02af8f41c2d)