Fixed error in propagating BN_FLG_CONSTTIME flag through BN_MONT_CTX_set, which could lead to information disclosure on RSA primes p and q.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4377)

(cherry picked from commit 3de81a5912041a70884cf4e52e7213f3b5dfa747)

Make sure that a cert with extensions gets version number 2 (v3)

Fixes #4419

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4420)

(cherry picked from commit 4881d849da23528e19b7312f963d28916d9804b1)

Fix 'key' option in s_server can be in ENGINE keyform

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4405)

(cherry picked from commit 75c445e49bb3d22afe72b28ae67945a9f67091f6)

Remove dhparam from SSL_CONF list.

Avoid duplicate assertion by removing dhparam from SSL_CONF parameter list:
dhparam is handled manually by s_server.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4408)

Reenable s_server -dhparam option

This option was lost when converting to a table-driven option parser
in commit 7e1b7485706c2b11091b5fa897fe496a2faa56cc.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4398)

(cherry picked from commit 51ac82702dc91cabd3dbf890d8f65b285282c0ce)

Fix overflow in c2i_ASN1_BIT_STRING.

c2i_ASN1_BIT_STRING takes length as a long but uses it as an int.  Check
bounds before doing so. Previously, excessively large inputs to the
function could write a single byte outside the target buffer. (This is
unreachable as asn1_ex_c2i already uses int for the length.)

Thanks to NCC for finding this issue. Fix written by Martin Kreichgauer.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4385)

(cherry picked from commit 6b1c8204b33aaedb7df7a009c241412839aaf950)

Null pointer used.
Address coverity report of null pointer being dereferenced.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4381)

(cherry picked from commit 9be34ee5c8576539a929d5b396ad071aed525f43)

Provide getters for min/max proto version

OpenSSL 1.1.0 made SSL_CTX and SSL structs opaque and introduced a new
API to set the minimum and maximum protocol version for SSL_CTX with
TLS_method(). Add getters to introspect the configured versions:

  int SSL_CTX_get_min_proto_version(SSL_CTX *ctx);
  int SSL_CTX_get_max_proto_version(SSL_CTX *ctx);
  int SSL_get_min_proto_version(SSL *ssl);
  int SSL_get_max_proto_version(SSL *ssl);

NOTE: The getters do not resolv the version in case when the minimum or
maxium version are configured as '0' (meaning auto-select lowest and
highst version number).

Signed-off-by: Christian Heimes <christian@python.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(cherry picked from commit 3edabd3ccb7aac89af5a63cfb2378e33a8be05d7)

Updated for new manual page location and TLS 1.3.

(Merged from https://github.com/openssl/openssl/pull/4376)

Error out when forcing an unsupported TLS version

If the result of a SSL_{CTX_,}set_{min,max}_proto_version() call
leaves the min and max version identical, and support for that version
is compiled out of the library, return an error.  Such an object has
no hope of successfully completing a handshake, and this error may
be easier to decipher than the resulting handshake failure.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit c8feba723a33e15201009d716d9ead02e653dfe6)

Updated the cherry-pick to not reference TLS1_3_VERSION, which does
not exist on this branch.

(Merged from https://github.com/openssl/openssl/pull/4376)

Disable the EGD seeding meachanism when stdio is disabled

crypto/rand/rand_egd.c makes extensive use of stdio functions.  When
they are disabled, it makes sense to disable egd as well.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4358)

(cherry picked from commit 15a1bd0ab2950671686cea51f4218c8f3d92fad9)

Update rsautl.pod for typo

Fixes the typo
CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4354)

(cherry picked from commit f70c22eb23763c6dce050293cc1b9a0a234d72b2)

Allow an endpoint to read the alert data before closing the socket

If an alert gets sent and then we close the connection immediately with
data still in the input buffer then a TCP-RST gets sent. Some OSs
immediately abandon data in their input buffer if a TCP-RST is received -
meaning the alert data itself gets ditched. Sending a TCP-FIN before the
TCP-RST seems to avoid this.

This was causing test failures in MSYS2 builds.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4333)

(cherry picked from commit bac6abe18d28373e0d2d0666c411020404197337)

Fix error handling/cleanup

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4326)

(cherry picked from commit 180794c54e98ae467c4ebced3737e1ede03e320a)

Add checks for alloc failing.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4341)

(cherry picked from commit d3c3dfc5778ab2cca0d25c5959c8b814a334addb)

Fix OpenSSL::Test::Utils::config to actualy load the config data

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4319)

(cherry picked from commit 607f4d564f9540cda6cf5b127f2414625a11741a)

Configure: base compiler-specific decisions on pre-defines.

The commit subject is a bit misleading in sense that decisions affect
only gcc and gcc-alikes, like clang, recent icc...

This is back-port of 54cf3b981afcbbd3754c8ba1114ab6a658d86c08, GH#4281.

Reviewed-by: Rich Salz <rsalz@openssl.org>

crypto/cryptlib.c: mask more capability bits upon FXSR bit flip.

OPENSSL_ia32cap.pod discusses possibility to disable operations on
XMM register bank. This formally means that this flag has to be checked
in combination with other flags. But it customarily isn't. But instead
of chasing all the cases we can flip more bits together with FXSR one.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4303)

(cherry picked from commit 6e5a853bda24e8aece325ecf5aa68b8ea832e414)

util/mkdef.pl: handle line terminators correctly

When parsing the header files, mkdef.pl didn't clear the line
terminator properly.  In most cases, this didn't matter, but there
were moments when this caused parsing errors (such as CRLFs in certain
cases).

Fixes #4267

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4304)

(cherry picked from commit e66b62b86e7725bdace0f24a76baa61db9c763f8)

Fixed address family test error for AF_UNIX in BIO_ADDR_make

CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4298)

(cherry picked from commit 177503752b24299cc97ccf07062a3b79c4f28899)

Avoid out-of-bounds read

Fixes CVE 2017-3735

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4276)

(cherry picked from commit b23171744b01e473ebbfd6edad70c1c3825ffbcd)

Remove NO_DIRENT; it isn't used anywhere

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4261)

(cherry picked from commit 32c1356302e74dfa5e8bd2d7002c18d91a323b70)

Clear secret stack values after use in curve25519.c

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4242)

(cherry picked from commit 78f1e4d0b063e17c9700f2aceecaca03bfa434f3)

NO_SYS_TYPES_H isn't defined anywhere, stop using it as a guard

This is a vestige from pre-1.1.0 OpenSSL

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4256)

(cherry picked from commit b379fe6cd046b9dd8a62309dcbaded763e2d4187)

Fix description of how to report a bug in INSTALL

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4259)

(cherry picked from commit 0a8ddc17f57691c8e2e2446c4126fb4133d07d21)

Clarify the meaning of no-stdio in INSTALL

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4259)

(cherry picked from commit 219b4643e40ada993730c55ae2c09815f89b4a2d)

Add documentation for SRTP functions

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4205)

(cherry picked from commit 3733ce61a2a0933bf7b04d9a14bfe3ac22bb8a0d)

Correct GCM docs.

Fix GCM documentation: the tag does not have to be supplied before
decrypting any data any more.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4231)

Tweak wording to be more clear.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4234)
(cherry picked from commit a130950df92abf7dd787b000403da02af8f41c2d)

Fix ctype arguments.

Cast arguments to the various ctype functions to unsigned char to match their
documentation.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4203)

(cherry picked from commit 00dfbaad88a69ed8294d6039bf5f7d722f72bf39)

evp_test.c: Add PrivPubKeyPair negative tests

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3246)

evp_test.c: Add PrivPubKeyPair test

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3246)

Addressed build failure because of missing #ifdef AF_UNIX guard
CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4067)

(cherry picked from commit 326eaa941e03a8922a3789ccab0d134c63d05c92)

Add a comment on expectations in the "tar" target

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4179)

(cherry picked from commit 77a9c26e03ccfec8d16985bce79e95eb6dc2dd2e)

Prepare tarball in dist directory

We changed directory to the wrong directory.
This change also separates the preparation phase from the tarball
building phase.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4179)

(cherry picked from commit 17c84aa763b1d69c5446542bf9b4e2f642c570f2)

Turn on error sensitivity in the "tar" target

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4179)

(cherry picked from commit 34a5b7d727204eb990acd44899d457245ac94d7c)

err/err.c: fix "wraparound" bug in ERR_set_error_data.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit d3d880ce01cfaf0091f46a2f6b5bd146d47a93e7)

Clear outputs in PKCS12_parse error handling.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4145)

(cherry picked from commit 524fdd515569e12047ddb29ba4c7f19706aacc98)

Fix OCSP_basic_verify() cert chain construction in case bs->certs is NULL

Now the certs arg is not any more neglected when building the signer cert chain.
Added case to test/recipes/80-test_ocsp.t proving fix for 3-level CA hierarchy.

See also http://rt.openssl.org/Ticket/Display.html?id=4620

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4124)

(cherry picked from commit 121738d1cbfffa704eef4073510f13b419e6f08d)

Reorder extensions to put SigAlgs last

WebSphere application server cannot handle having an empty
extension (e.g. EMS/EtM) as the last extension in a client hello.
This moves the SigAlgs extension last (before any padding) for TLSv1.2
to avoid this issue.

Force the padding extension to a minimum length of 1.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3927)

no-ec2m fixes

Fix warning and don't use binary field certificate for ECDH CMS
key only test.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4134)

(cherry picked from commit ed5c7ea250657796517fef035e162b20eb8d3c7f)

Add alternative CMS P-256 cert

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4134)

(cherry picked from commit 1aee92bf0f3fe243192fb5440f7c9789d5a08c67)

Add missing HTML tag in www_body in s_server.c

In the generated HTML document, the `<pre>` tag is not closed. This patch
also has a trivial code-style improvement, unrelated to the bug fix.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4088)

(cherry picked from commit 1a9f5cf0d58629ab8972f50e937d8ab78bf27b6f)

Support CMS decrypt without a certificate for all key types

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4115)

(cherry picked from commit 3f1d1704f215dc11e1fefbb6ecdcb2a08c3a65db)

Add test for ECDH CMS key only

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4115)

(cherry picked from commit 5d09b003c080d81ff6adfb6c54be5c018a2ba294)

Avoid surpising password dialog in X509 file lookup.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4111)

(cherry picked from commit db854bb14a7010712cfc02861731399b1b587474)

Fix typo in files in crypto folder

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Andy Polyakov <appro@openssl.org>
GH: #4093
(cherry picked from commit c9a41d7dd631a69b73bea8af714a3a8b872b8309)

Revert "Perl: Use File::Glob::bsd_glob rather than File::Glob::glob"

This needs more change that what is appropriate for the 1.1.0 branch.

This reverts commit 0401110073cd392602855f9b72af2ebec7909625.

Reviewed-by: Andy Polyakov <appro@openssl.org>

remove horrible pragma macro and remove __owur from SSL_CTX_add_session() declaration

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4014)

(cherry picked from commit 5bd05e579994c756cd994b5e0ff5f395aae6bfff)

Remove the obsolete misleading comment and code related to it.

The comment "The following should not return 1, otherwise, things
are very strange" is from the very first commit of OpenSSL. The
really meaning of the comment is if the identical session can be
found from internal cache after calling get_session_cb but not
found before calling get_session_cb, it is just strange.

The value 1 was originated from the old doc of SSLeay, reversed
from the actual return value of SSL_CTX_add_session().

Anyway either return value of SSL_CTX_add_session() should not
interrupt the session resumption process. So the checking of
return value of SSL_CTX_add_session() is not necessary.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4014)

(cherry picked from commit e29bb83479cc567b4bb414dc55148ec06a30a115)

Add EC key generation paragraph in doc/HOWTO/keys.txt

Seems this documentation is not dead, so add this missing part

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4037)

(cherry picked from commit 003ef7ef9ad84bfb12ae1f42c41cdf08111f499f)

RSA_get0_ functions permit NULL parameters

Document that the RSA_get0_ functions permit a NULL BIGNUM **. Those output parameters are ignored.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4064)

(cherry picked from commit 07c54e598ce8a15c08abcfcae939bdf8f017dae3)

Fix an information leak in the RSA padding check code.
The memory blocks contain secret data and must be
cleared before returning to the system heap.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4062)

(cherry picked from commit e670db0183079b5f6325ce2abd9d785e0f966890)

Fix errors in SSL_state_string_long

TLS_ST_SR_NEXT_PROTO means "SSLv3/TLS read next proto"
Fix typo in the message for TLS_ST_SW_CERT_STATUS

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4054)

(cherry picked from commit f978f2b8af576ed1d9409de440b5c1f97ac0e7ab)

app_isdir() cleanup

I think it's better to use `GetFileAttributes` to obtain the attributes
of a file than `FindFirstFile`. If the input name contains `*`, this
function should return failure rather than check whether the first match
happens to be a file or a directory.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/3991)

(cherry picked from commit 5bd051a0dcd4f04bc9ea197f74b34612b3fcca84)

Add some test coverage for OPENSSL_secure_clear_free

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4048)

Implement the CRYPTO_secure_clear_free function.
Use OPENSSL_secure_clear_free for secure mem BIOs
and X25519 private keys.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4048)

Clean password buffer on stack for PEM_read_bio_PrivateKey
and d2i_PKCS8PrivateKey_bio before it goes out of scope.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4047)

(cherry picked from commit 02fd47c8b0930dff9b188fd13bfb9da5e59444a8)

Fix rsa -check option

original problem: if a private key is invaild, nothing outputted.

the error filter in apps/rsa.c is not working any more.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4043)

(cherry picked from commit 03883e7e168b0f5ef52a516eeb86346b767e0298)

Fix a reference nit in doc

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4036)

(cherry picked from commit dbd007d7d2cae4891936aed55949b55b776b97ec)

Perl: Use File::Glob::bsd_glob rather than File::Glob::glob

File::Glob::glob is deprecated, it's use generates this kind of
message:

    File::Glob::glob() will disappear in perl 5.30. Use File::Glob::bsd_glob() instead. at ../master/Configure line 277.

So instead, use a construction that makes the caller glob() use
File::Glob::bsd_glob().

Note that we're still excluding VMS, as it's directory specs use '['
and ']', which have a different meaning with bsd_glob and would need
some extra quoting.  This might change, but later.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4040)

(cherry picked from commit 102c9e1296b656c4049c1110abc8a52b43bd0dcf)

Fix comment typo.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4023)

(cherry picked from commit d67e755418b62fb451ec221c126c9935a06ea63b)

Fix async engine pause dead lock in error case.

In 'crypto/rand/ossl_rand.c', a call to
'ASYNC_unblock_pause()' is missing in an error case.

CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(cherry picked from commit e4b16013e9b3d19241d3ba0bb0875f0d70d93509)

(Merged from https://github.com/openssl/openssl/pull/4024)

Remove resolved TODO

Fixed in 5b8fa431ae8eb5a18ba913494119e394230d4b70

[ci skip]

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3924)

Fix potential use-after-free and memory leak

In function wait_for_async(), allocated async fds is freed if
`SSL_get_all_async_fds` fails, but later `fds` is used. Interestingly,
it is not freed when everything succeeds.

Rewrite the FD set loop to make it more readable and to not modify the allocated
pointer so it can be freed.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/3992)

(cherry picked from commit 0a3452520fe4cd6871ae8b7c4199c6d5d4efe912)

x86_64 assembly pack: "optimize" for Knights Landing.

"Optimize" is in quotes because it's rather a "salvage operation"
for now. Idea is to identify processor capability flags that
drive Knights Landing to suboptimial code paths and mask them.
Two flags were identified, XSAVE and ADCX/ADOX. Former affects
choice of AES-NI code path specific for Silvermont (Knights Landing
is of Silvermont "ancestry"). And 64-bit ADCX/ADOX instructions are
effectively mishandled at decode time. In both cases we are looking
at ~2x improvement.

Hardware used for benchmarking courtesy of Atos, experiments run by
Romain Dolbeau <romain.dolbeau@atos.net>. Kudos!

This is minimalistic backpoint of 64d92d74985ebb3d0be58a9718f9e080a14a8e7f

Thanks to David Benjamin for spotting typo in Knights Landing detection!

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4006)

aes/asm/aesni-sha*-x86_64.pl: add SHAEXT performance results.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/3898)

(cherry picked from commit 1843787173da9b07029d0863e236107b1dd4fdd7)

evp/e_aes_cbc_hmac_sha256.c: give SHAEXT right priority.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/3898)

(cherry picked from commit d0f6eb1d8c84165c383a677266cfae9c0b162781)

schlock global variable needs to be volatile

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4000)

(cherry picked from commit e0de4dd5a2b0c0dc27e6a6ab01fabe374d657d23)

Fix nid assignment in ASN1_STRING_TABLE_add

CLA: trivial

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3934)

(cherry picked from commit 386e9169c35718436ba038dff93711d2db73fa6a)

test/recipes/80-test_tsa.t: Don't trust 'OPENSSL_CONF'

There's a case when the environment variable OPENSSL_CONF is
useless...  when cross compiling for mingw and your wine environment
has an environment variable OPENSSL_CONF.  The latter will override
anything that's given when starting wine and there make the use of
that environment variable useless in our tests.

Therefore, we should not trust it, and use explicit '-config' options
instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3994)

(cherry picked from commit 83e0d090b196f70a9f25ff505d5813745585ef85)

Simplify Makefile.shared

Makefile.shared was designed to figure out static library names,
shared library names, library version compatibility, import library
names and the like on its own.  This was a design for pre-1.1.0
OpenSSL because the main Makefile didn't have all that knowledge.

With 1.1.0, the situation isn't the same, a lot more knowledge is
included in the main Makefile, and while Makefile.shared did things
right most of the time (there are some corner cases, such as the
choice of .sl or .so as DSO extension on some HPUX versions), there's
still an inherent fragility when one has to keep an eye on
Makefile.shared to make sure it produces what the main Makefile
produces.

This change simplifies Makefile.shared by removing all its
"intelligence" and have it depend entirely on the input from the main
Makefile instead.  That way, all the naming is driven from
configuration data.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3983)

(cherry picked from commit d07abe13a7955296da756d3f5032a276ac3d47ee)

Fix const correctness of EC_KEY_METHOD_get_*

Changes the EC_KEY_METHOD_get_* family to not need a EC_KEY_METHOD* as
its first parameter, but a const EC_KEY_METHOD*, which is entirely
sufficient.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
GH: #3985
(cherry picked from commit 4e9b720e90ec154c9708139e96ec0ff8e2796c82)

Remove some dead code

The intention of the removed code was to check if the previous operation
carried. However this does not work. The "mask" value always ends up being
a constant and is all ones - thus it has no effect. This check is no longer
required because of the previous commit.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3832)

(cherry picked from commit d5475e319575a45b20f560bdfae56cbfb165cb01)

Fix undefined behaviour in e_aes_cbc_hmac_sha256.c and e_aes_cbc_hmac_sha1.c

In TLS mode of operation the padding value "pad" is obtained along with the
maximum possible padding value "maxpad". If pad > maxpad then the data is
invalid. However we must continue anyway because this is constant time code.

We calculate the payload length like this:

    inp_len = len - (SHA_DIGEST_LENGTH + pad + 1);

However if pad is invalid then inp_len ends up -ve (actually large +ve
because it is a size_t).

Later we do this:

    /* verify HMAC */
    out += inp_len;
    len -= inp_len;

This ends up with "out" pointing before the buffer which is undefined
behaviour. Next we calculate "p" like this:

    unsigned char *p =
        out + len - 1 - maxpad - SHA256_DIGEST_LENGTH;

Because of the "out + len" term the -ve inp_len value is cancelled out
so "p" points to valid memory (although technically the pointer arithmetic
is undefined behaviour again).

We only ever then dereference "p" and never "out" directly so there is
never an invalid read based on the bad pointer - so there is no security
issue.

This commit fixes the undefined behaviour by ensuring we use maxpad in
place of pad, if the supplied pad is invalid.

With thanks to Brian Carpenter for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3832)

(cherry picked from commit 335d0a4646981c9d96b62811bcfd69a96a1a67d9)

RSA_padding_check_PKCS1_type_2 is not constant time.

This is an inherent weakness of the padding mode. We can't make the
implementation constant time (see the comments in rsa_pk1.c), so add a
warning to the docs.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix bogus use of BIO_sock_should_retry.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3948)

(cherry picked from commit daaaa3cb7e506466b38de995b3e5149f4045bdff)

Fix gcc-7 warnings about missing fall thru comments.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3936)

Update PR#3925

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3925)

Retry SSL_read on ERROR_WANT_READ.
This resolves the retry issue in general, but also the specific case where a TLS 1.3 server sends a post-handshake NewSessionTicket message prior to appdata.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3925)

Fix crash in BUF_MEM_grow_clean.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3896)

(cherry picked from commit e1ca9e1f6db97052a0ebea6591f323b12b1e0020)

Fix cipher_compare

Unsigned overflow.  Found by Brian Carpenter

Fixes #3889

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3890)

(cherry picked from commit a7ff57965b81ce4fd73a18266ce29abf6b909fdb)

test/run_tests.pl: Make sure to exit with a code that's understood universally

TAP::Parser::Aggregator::has_errors may return any number, not just 0
and 1.  With Perl on VMS, any number from 2 and on is interpreted as a
VMS status, the 3 lower bits are the encoded severity (1 = SUCCESS,
for example), so depending on what has_errors returns, a test failure
might be interpreted as a success.  Therefore, it's better to make
sure the exit code is 0 or 1, nothing else (they are special on VMS,
and mean SUCCESS or FAILURE, to match Unix conventions).

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3880)

(cherry picked from commit 4549ed12ec3337313c14815438fa9aee88bf1359)

test/recipes/90-test_shlibload.t: Make sure to handle library renames

VMS renames our libraries to fit VMS conventions.  This must be accounted
for when we want to load them.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3880)

(cherry picked from commit bfa3480f7609351563ac36dddd7c64e97aa6f446)

VMS: When running a sub-MMS, make sure to give it the main MMS' qualifiers

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3880)

(cherry picked from commit 984cf15eb5faac8e328d1ba4a623b1777eb82de1)

Add echo for end of each build phase

Ported GH #3842 to 1.1.0 branch

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3844)

Fix travis clang-3.9 builds

Something environmental changed in travis so that it started preferring
the ubuntu clang-3.9 version instead of the llvm.org one. This breaks the
sanitiser based builds. This change forces travis to de-prioritise the
ubuntu clang packages.

[extended tests]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3854)

Fix small UI issues

- in EVP_read_pw_string_min(), the return value from UI_add_* wasn't
  properly checked
- in UI_process(), |state| was never made NULL, which means an error
  when closing the session wouldn't be accurately reported.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/3849)

(cherry picked from commit b96dba9e5ec7afc355be1eab915f69c8c0d51741)

Avoid possible memleak in X509_policy_check()

When tree_calculate_user_set() fails, a jump to error failed to
deallocate a possibly allocated |auth_nodes|.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/3850)

(cherry picked from commit 67f060acefae34d820ccdb2f560d86ed10633500)

Correct documentation for UI_get0_result_string

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3824)

When apps_startup() fails, exit with a failure code and a message

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3816)

(cherry picked from commit f2da4a4917eae1bf66290e1bd8ccd3db69d3fe63)

util/mkdef.pl: Make symbol version processing Linux only

For Windows, we care which way it is, the resulting file is just a pile
of symbols.  For VMS, we really need to care about the numeric ordering,
and getting the symbols sorted by symbol version too didn't agree with
that.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3804)

(cherry picked from commit 0e288c2af2f24121ebd5f0c58912d9429915c02a)

util/mkdef.pl: Add UNIX as a platform

This allows us to guard Unix specific functions with
#ifndef / #ifdef OPENSSL_SYS_UNIX

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3804)

(cherry picked from commit 9c06cf04ee9aa6682e6cc635aeb453ac12e641c1)

tsget.in: remove call of WWW::Curl::Easy::global_cleanup

This function is undocumented, but similarly named functions (such as
'curl_global_cleanup') are documented as internals that should not be
called by scripts.

Fixes #3765

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3768)

(cherry picked from commit 6544a91cefe817156461e57a4538c3fe7f621075)

Fix OBJ_create() to tolerate a NULL sn and ln

In 1.0.2 and before OBJ_create() allowed the sn or ln parameter to be NULL.
Commit 52832e47 changed that so that it crashed if they were NULL.

This was causing problems with the built-in config oid module. If a long
name was provided OBJ_create() is initially called with a NULL ln and
therefore causes a crash.

Fixes #3733

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3753)

(cherry picked from commit f13615c5b828aeb8e3d9bf2545c803633d1c684f)

Fix the names of ChaCha20-Poly1305 cipher suites in t1_trce.c.

Per RFC 7905, the cipher suite names end in "_SHA256". The original
implementation targeted the -03 draft, but there was a -04 draft right
before the RFC was published to make the names consistent.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3748)

(cherry picked from commit 32bbf777d0de7b0be90170b69fe9290096065fc9)

Add documentation for the SSL_export_keying_material() function

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3736)

Fix DTLS failure when used in a build which has SCTP enabled

The value of BIO_CTRL_DGRAM_SET_PEEK_MODE was clashing with the value for
BIO_CTRL_DGRAM_SCTP_SET_IN_HANDSHAKE. In an SCTP enabled build
BIO_CTRL_DGRAM_SCTP_SET_IN_HANDSHAKE was used unconditionally with
the reasoning that it would be ignored if SCTP wasn't in use. Unfortunately
due to this clash, this wasn't the case. The BIO ended up going into peek
mode and was continually reading the same data over and over - throwing it
away as a replay.

Fixes #3723

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3724)

(cherry picked from commit 9924087573cfbc8d2bc97088f36d1a81ca00cda3)

Don't fail the connection in SSLv3 if server selects ECDHE

ECDHE is not properly defined for SSLv3. Commit fe55c4a2 prevented ECDHE
from being selected in that protocol. However, historically, servers do
still select ECDHE anyway so that commit causes interoperability problems.
Clients that previously worked when talking to an SSLv3 server could now
fail.

This commit introduces an exception which enables a client to continue in
SSLv3 if the server selected ECDHE.

(cherry picked from commit 8af91fd9d08487e0dffb6ccac5f42633c964f3f0)

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3734)

Add parentheses around macro argument of OSSL_NELEM.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3039)

(cherry picked from commit 26dc47f3c44c7fb4488d1becaf997737486f4922)

Remove duplicates from clang_devteam_warnings

Since the clang_devteam_warnings are appended to the gcc_devteam_warnings
when strict-warnings are requested, any items present in both the gcc
and clang variables will be duplicated in the cflags used for clang builds.
Remove the extra copy from the clang-specific flags in favor of the
gcc_devteam_warnings that are used for all strict-warnings builds.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3239)
(cherry picked from commit 96db26919d5caff2db6340354a026f56dc6f09da)

[extended tests]