Fix potential use-after-free and memory leak

In function wait_for_async(), allocated async fds is freed if
`SSL_get_all_async_fds` fails, but later `fds` is used. Interestingly,
it is not freed when everything succeeds.

Rewrite the FD set loop to make it more readable and to not modify the allocated
pointer so it can be freed.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/3992)

(cherry picked from commit 0a3452520fe4cd6871ae8b7c4199c6d5d4efe912)

x86_64 assembly pack: "optimize" for Knights Landing.

"Optimize" is in quotes because it's rather a "salvage operation"
for now. Idea is to identify processor capability flags that
drive Knights Landing to suboptimial code paths and mask them.
Two flags were identified, XSAVE and ADCX/ADOX. Former affects
choice of AES-NI code path specific for Silvermont (Knights Landing
is of Silvermont "ancestry"). And 64-bit ADCX/ADOX instructions are
effectively mishandled at decode time. In both cases we are looking
at ~2x improvement.

Hardware used for benchmarking courtesy of Atos, experiments run by
Romain Dolbeau <romain.dolbeau@atos.net>. Kudos!

This is minimalistic backpoint of 64d92d74985ebb3d0be58a9718f9e080a14a8e7f

Thanks to David Benjamin for spotting typo in Knights Landing detection!

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4006)

aes/asm/aesni-sha*-x86_64.pl: add SHAEXT performance results.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/3898)

(cherry picked from commit 1843787173da9b07029d0863e236107b1dd4fdd7)

evp/e_aes_cbc_hmac_sha256.c: give SHAEXT right priority.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/3898)

(cherry picked from commit d0f6eb1d8c84165c383a677266cfae9c0b162781)

schlock global variable needs to be volatile

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4000)

(cherry picked from commit e0de4dd5a2b0c0dc27e6a6ab01fabe374d657d23)

Fix nid assignment in ASN1_STRING_TABLE_add

CLA: trivial

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3934)

(cherry picked from commit 386e9169c35718436ba038dff93711d2db73fa6a)

test/recipes/80-test_tsa.t: Don't trust 'OPENSSL_CONF'

There's a case when the environment variable OPENSSL_CONF is
useless...  when cross compiling for mingw and your wine environment
has an environment variable OPENSSL_CONF.  The latter will override
anything that's given when starting wine and there make the use of
that environment variable useless in our tests.

Therefore, we should not trust it, and use explicit '-config' options
instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3994)

(cherry picked from commit 83e0d090b196f70a9f25ff505d5813745585ef85)

Simplify Makefile.shared

Makefile.shared was designed to figure out static library names,
shared library names, library version compatibility, import library
names and the like on its own.  This was a design for pre-1.1.0
OpenSSL because the main Makefile didn't have all that knowledge.

With 1.1.0, the situation isn't the same, a lot more knowledge is
included in the main Makefile, and while Makefile.shared did things
right most of the time (there are some corner cases, such as the
choice of .sl or .so as DSO extension on some HPUX versions), there's
still an inherent fragility when one has to keep an eye on
Makefile.shared to make sure it produces what the main Makefile
produces.

This change simplifies Makefile.shared by removing all its
"intelligence" and have it depend entirely on the input from the main
Makefile instead.  That way, all the naming is driven from
configuration data.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3983)

(cherry picked from commit d07abe13a7955296da756d3f5032a276ac3d47ee)

Fix const correctness of EC_KEY_METHOD_get_*

Changes the EC_KEY_METHOD_get_* family to not need a EC_KEY_METHOD* as
its first parameter, but a const EC_KEY_METHOD*, which is entirely
sufficient.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
GH: #3985
(cherry picked from commit 4e9b720e90ec154c9708139e96ec0ff8e2796c82)

Remove some dead code

The intention of the removed code was to check if the previous operation
carried. However this does not work. The "mask" value always ends up being
a constant and is all ones - thus it has no effect. This check is no longer
required because of the previous commit.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3832)

(cherry picked from commit d5475e319575a45b20f560bdfae56cbfb165cb01)

Fix undefined behaviour in e_aes_cbc_hmac_sha256.c and e_aes_cbc_hmac_sha1.c

In TLS mode of operation the padding value "pad" is obtained along with the
maximum possible padding value "maxpad". If pad > maxpad then the data is
invalid. However we must continue anyway because this is constant time code.

We calculate the payload length like this:

    inp_len = len - (SHA_DIGEST_LENGTH + pad + 1);

However if pad is invalid then inp_len ends up -ve (actually large +ve
because it is a size_t).

Later we do this:

    /* verify HMAC */
    out += inp_len;
    len -= inp_len;

This ends up with "out" pointing before the buffer which is undefined
behaviour. Next we calculate "p" like this:

    unsigned char *p =
        out + len - 1 - maxpad - SHA256_DIGEST_LENGTH;

Because of the "out + len" term the -ve inp_len value is cancelled out
so "p" points to valid memory (although technically the pointer arithmetic
is undefined behaviour again).

We only ever then dereference "p" and never "out" directly so there is
never an invalid read based on the bad pointer - so there is no security
issue.

This commit fixes the undefined behaviour by ensuring we use maxpad in
place of pad, if the supplied pad is invalid.

With thanks to Brian Carpenter for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3832)

(cherry picked from commit 335d0a4646981c9d96b62811bcfd69a96a1a67d9)

RSA_padding_check_PKCS1_type_2 is not constant time.

This is an inherent weakness of the padding mode. We can't make the
implementation constant time (see the comments in rsa_pk1.c), so add a
warning to the docs.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix bogus use of BIO_sock_should_retry.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3948)

(cherry picked from commit daaaa3cb7e506466b38de995b3e5149f4045bdff)

Fix gcc-7 warnings about missing fall thru comments.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3936)

Update PR#3925

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3925)

Retry SSL_read on ERROR_WANT_READ.
This resolves the retry issue in general, but also the specific case where a TLS 1.3 server sends a post-handshake NewSessionTicket message prior to appdata.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3925)

Fix crash in BUF_MEM_grow_clean.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3896)

(cherry picked from commit e1ca9e1f6db97052a0ebea6591f323b12b1e0020)

Fix cipher_compare

Unsigned overflow.  Found by Brian Carpenter

Fixes #3889

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3890)

(cherry picked from commit a7ff57965b81ce4fd73a18266ce29abf6b909fdb)

test/run_tests.pl: Make sure to exit with a code that's understood universally

TAP::Parser::Aggregator::has_errors may return any number, not just 0
and 1.  With Perl on VMS, any number from 2 and on is interpreted as a
VMS status, the 3 lower bits are the encoded severity (1 = SUCCESS,
for example), so depending on what has_errors returns, a test failure
might be interpreted as a success.  Therefore, it's better to make
sure the exit code is 0 or 1, nothing else (they are special on VMS,
and mean SUCCESS or FAILURE, to match Unix conventions).

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3880)

(cherry picked from commit 4549ed12ec3337313c14815438fa9aee88bf1359)

test/recipes/90-test_shlibload.t: Make sure to handle library renames

VMS renames our libraries to fit VMS conventions.  This must be accounted
for when we want to load them.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3880)

(cherry picked from commit bfa3480f7609351563ac36dddd7c64e97aa6f446)

VMS: When running a sub-MMS, make sure to give it the main MMS' qualifiers

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3880)

(cherry picked from commit 984cf15eb5faac8e328d1ba4a623b1777eb82de1)

Add echo for end of each build phase

Ported GH #3842 to 1.1.0 branch

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3844)

Fix travis clang-3.9 builds

Something environmental changed in travis so that it started preferring
the ubuntu clang-3.9 version instead of the llvm.org one. This breaks the
sanitiser based builds. This change forces travis to de-prioritise the
ubuntu clang packages.

[extended tests]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3854)

Fix small UI issues

- in EVP_read_pw_string_min(), the return value from UI_add_* wasn't
  properly checked
- in UI_process(), |state| was never made NULL, which means an error
  when closing the session wouldn't be accurately reported.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/3849)

(cherry picked from commit b96dba9e5ec7afc355be1eab915f69c8c0d51741)

Avoid possible memleak in X509_policy_check()

When tree_calculate_user_set() fails, a jump to error failed to
deallocate a possibly allocated |auth_nodes|.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/3850)

(cherry picked from commit 67f060acefae34d820ccdb2f560d86ed10633500)

Correct documentation for UI_get0_result_string

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3824)

When apps_startup() fails, exit with a failure code and a message

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3816)

(cherry picked from commit f2da4a4917eae1bf66290e1bd8ccd3db69d3fe63)

util/mkdef.pl: Make symbol version processing Linux only

For Windows, we care which way it is, the resulting file is just a pile
of symbols.  For VMS, we really need to care about the numeric ordering,
and getting the symbols sorted by symbol version too didn't agree with
that.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3804)

(cherry picked from commit 0e288c2af2f24121ebd5f0c58912d9429915c02a)

util/mkdef.pl: Add UNIX as a platform

This allows us to guard Unix specific functions with
#ifndef / #ifdef OPENSSL_SYS_UNIX

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3804)

(cherry picked from commit 9c06cf04ee9aa6682e6cc635aeb453ac12e641c1)

tsget.in: remove call of WWW::Curl::Easy::global_cleanup

This function is undocumented, but similarly named functions (such as
'curl_global_cleanup') are documented as internals that should not be
called by scripts.

Fixes #3765

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3768)

(cherry picked from commit 6544a91cefe817156461e57a4538c3fe7f621075)

Fix OBJ_create() to tolerate a NULL sn and ln

In 1.0.2 and before OBJ_create() allowed the sn or ln parameter to be NULL.
Commit 52832e47 changed that so that it crashed if they were NULL.

This was causing problems with the built-in config oid module. If a long
name was provided OBJ_create() is initially called with a NULL ln and
therefore causes a crash.

Fixes #3733

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3753)

(cherry picked from commit f13615c5b828aeb8e3d9bf2545c803633d1c684f)

Fix the names of ChaCha20-Poly1305 cipher suites in t1_trce.c.

Per RFC 7905, the cipher suite names end in "_SHA256". The original
implementation targeted the -03 draft, but there was a -04 draft right
before the RFC was published to make the names consistent.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3748)

(cherry picked from commit 32bbf777d0de7b0be90170b69fe9290096065fc9)

Add documentation for the SSL_export_keying_material() function

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3736)

Fix DTLS failure when used in a build which has SCTP enabled

The value of BIO_CTRL_DGRAM_SET_PEEK_MODE was clashing with the value for
BIO_CTRL_DGRAM_SCTP_SET_IN_HANDSHAKE. In an SCTP enabled build
BIO_CTRL_DGRAM_SCTP_SET_IN_HANDSHAKE was used unconditionally with
the reasoning that it would be ignored if SCTP wasn't in use. Unfortunately
due to this clash, this wasn't the case. The BIO ended up going into peek
mode and was continually reading the same data over and over - throwing it
away as a replay.

Fixes #3723

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3724)

(cherry picked from commit 9924087573cfbc8d2bc97088f36d1a81ca00cda3)

Don't fail the connection in SSLv3 if server selects ECDHE

ECDHE is not properly defined for SSLv3. Commit fe55c4a2 prevented ECDHE
from being selected in that protocol. However, historically, servers do
still select ECDHE anyway so that commit causes interoperability problems.
Clients that previously worked when talking to an SSLv3 server could now
fail.

This commit introduces an exception which enables a client to continue in
SSLv3 if the server selected ECDHE.

(cherry picked from commit 8af91fd9d08487e0dffb6ccac5f42633c964f3f0)

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3734)

Add parentheses around macro argument of OSSL_NELEM.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3039)

(cherry picked from commit 26dc47f3c44c7fb4488d1becaf997737486f4922)

Remove duplicates from clang_devteam_warnings

Since the clang_devteam_warnings are appended to the gcc_devteam_warnings
when strict-warnings are requested, any items present in both the gcc
and clang variables will be duplicated in the cflags used for clang builds.
Remove the extra copy from the clang-specific flags in favor of the
gcc_devteam_warnings that are used for all strict-warnings builds.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3239)
(cherry picked from commit 96db26919d5caff2db6340354a026f56dc6f09da)

[extended tests]

Address some -Wold-style-declaration warnings

gcc's -Wextra pulls in -Wold-style-declaration, which triggers when a
declaration has a storage-class specifier as a non-initial qualifier.
The ISO C formal grammar requires the storage-class to be the first
component of the declaration, if present.

Seeint as the register storage-class specifier does not really have any effect
anymore with modern compilers, remove it entirely while we're here, instead of
fixing up the order.

Interestingly, the gcc devteam warnings do not pull in -Wextra, though
the clang ones do.

[extended tests]

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3239)
(cherry picked from commit f44903a428cc63ce88bfba26e8e4e2e9b21f058d)

Add -Wextra to gcc devteam warnings

clang already has it; let's flip the switch and deal with the fallout.
Exclude -Wunused-parameter, as we have many places where we keep unused
parameters to conform to a uniform vtable-like interface.
Also exclude -Wmissing-field-initializers; it's okay to rely on
the standard-mandated behavior of filling out with 0/NULL.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3239)
(cherry picked from commit 560ad13c74fe6967991a2429d90eeeba815d1f9e)

Fix the fall-out in 04-test_bioprint.t

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3712)
(cherry picked from commit 3ac6d5ee5372b05aa90cc5c44efbde01bd669e9e)

Fix the error handling in ERR_get_state:

- Ignoring the return code of ossl_init_thread_start created a memory leak.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3712)
(cherry picked from commit af6de400b49c011600cfd557319d1142da6e5cbd)

Remove non-accurate description in Configure script

For DES and 3DES based ciphers are also enabled by this option.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3707)
(cherry picked from commit edcdf38bd09f77160f0ec3e5bdd9d9525daf6f25)

Remove a pointless "#if 0" block from BN_mul.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3683)

(cherry picked from commit 93a8b3ba793c769a3634e56642dac55a8d44023f)

Add apps/progs.h to gitignore

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3692)
(cherry picked from commit 0bb4847b67b9eaa1123abf99e077d66ad54c7616)

Build apps/progs.h dynamically

Because apps/progs.h isn't configuration agnostic, it's not at all
suited for 'make update' or being versioned, so change it to be
dynamically generated.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3688)

(cherry picked from commit 6a74806ed741db24df3a7155e4bc11fb4ba9bc2a)

.travis.yml: Detect if 'make update' updated something

If it did, it really is something that should be checked in, and should
therefore make a CI build fail.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3686)

(cherry picked from commit 46e5b661d435b11652b90cd9e06cbf6606d3b61a)

Fix crash in ecdh_simple_compute_key.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3671)
(cherry picked from commit abea494cf75061650deecf584adc2cd293ce322d)

Fix a possible crash in dsa_builtin_paramgen2.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3675)
(cherry picked from commit fb0a64126b8c11a6961dfa1323c3602b591af7df)

Fix another possible crash in rsa_ossl_mod_exp.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3675)
(cherry picked from commit 5625567f9c7daaa2e2689647e10e4c5d7370718f)

Fix possible crash in X931 code.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3675)
(cherry picked from commit 5419dadd4bd1f7abbfa23326ca766d2c143f257c)

Fix ex_data and session_dup issues

Code was added in commit b3c31a65 that overwrote the last ex_data value
using CRYPTO_dup_ex_data() causing a memory leak, and potentially
confusing the ex_data dup() callback.

In ssl_session_dup(), fix error handling (properly reference and up-ref
shared data) and new-up the ex_data before calling CRYPTO_dup_ex_data();
all other structures that dup ex_data have the destination ex_data new'd
before the dup.

Fix up some of the ex_data documentation.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3625)

Fix a possible crash in the error handling.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3672)

(cherry picked from commit 4fc426b78964b3d234cb7b1b6112c9b80e16a13a)

Fix a memleak in ec_copy_parameters.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3666)
(cherry picked from commit 188a9bd950837c70661aa6849894e4e02d129031)

Fix memleak in EVP_DigestSignFinal/VerifyFinal.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3658)
(cherry picked from commit 19546246cf44d30043fb17d1899b2c325924ac8b)

Remove needless type casting.

CLA: trivial

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3627)

(cherry picked from commit a020f54c25985fc83e809daa15a3920731d39612)

Fix possible usage of NULL pointers in apps/spkac.c

Check return value of NETSCAPE_SPKI_new() and
NETSCAPE_SPKI_b64_encode(), and also clean up coding style incidentally.

Signed-off-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3647)
(cherry picked from commit f2582f08d5167ee84b7b313fd1435fe91ee44880)

Fix speed command for alternation of ciphers and digests.

CLA: trivial

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3487)

(cherry picked from commit 9ae4e664da0692f27bfe0d1a34db29ed815203c8)

fix broken implementations of GOST ciphersuites

removed the unnecessary upper bracket
add !SSL_USE_SIGALGS to check for broken implementations of GOST
client signature (signature without length field)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3588)

Remove stale note from s_server.pod

Modern browsers are now, well, pretty modern.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3644)
(cherry picked from commit 36c438514db71eba3e8062fef7869b9211630a19)

Ignore -named_curve auto value to improve backwards compatibility

Fixes #3490

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3518)
(cherry picked from commit 1c7aa0dbf16c3389bbedd13391bb653e7a189603)

Fix a read off the end of the input buffer

when building with OPENSSL_SMALL_FOOTPRINT defined.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3533)

(cherry picked from commit 0b20ad127ce86b05a854f31d51d91312c86ccc74)

Use memset to clear SRP_CTX instead of NULL and zero assignments

This uses memset() to clear all of the SRP_CTX when free'ing or
initializing it as well as in error paths instead of having a series
of NULL and zero assignments as it is safer.

It also changes SSL_SRP_CTX_init() to reset all the SRP_CTX to zero
in case or error, previously it could retain pointers to freed
memory, potentially leading to a double free.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3467)

(cherry picked from commit 135976b3dd24e674c202c20b5746fc04ebb1fc1a)

Make SRP_CTX.info ownership and lifetime be the same as SRP_CTX.login.

Ownership and lifetime rules of SRP_CTX.info are confusing and different
from those of SRP_CTX.login, making it difficult to use correctly.
This makes the ownership and lifetime be the same as those of SRP_CTX.login,
thet is a copy is made when setting it and is freed when SRP_CTX is freed.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3467)
(cherry picked from commit e655f5494100d93307726b23f4718ead0cadc0c3)

Windows: rearrange programs cleanup

The list of programs hit nmake's maximum line length, so we split up the
line in smaller chunks.

Fixes #3634

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3636)
(cherry picked from commit edef840f23b31066df4333995f544ae0f32d3e09)

Add a lock around the OBJ_NAME table

Various initialization functions modify this table, which can cause heap
corruption in the absence of external synchronization.

Some stats are modified from OPENSSL_LH_retrieve, where callers aren't
expecting to have to take out an exclusive lock. Switch to using atomic
operations for those stats.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3525)
(cherry picked from commit be606c013d31847718ceb5d97c567988a771c2e5)

Document default client -psk_identity

Document that -psk is required to use PSK cipher

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3607)
(cherry picked from commit 9d772829c9e4f202460acb43f9e073841a7cb9db)

ec/asm/ecp_nistz256-x86_64.pl: minor sqr_montx cleanup.

Drop some redundant instructions in reduction in ecp_nistz256_sqr_montx.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit 8fc063dcc9668589fd95533d25932396d60987f9)

Only release thread-local key if we created it.

Thanks to Jan Alexander Steffens for finding the bug and confirming the
fix.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3592)
(cherry picked from commit 73bc53708c386c1ea85941d345721e23dc61c05c)

Add text pointing to full change list.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3606)
(cherry picked from commit 01dfaa08b1960049f91485f2e5eec6c6bd03db39)

Do not document SSL_CTX_set1_cert_store()

It does not exist on this branch.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3602)

Fix coding style in apps/passwd file

Reformat some indents and braces based on OpenSSL coding style spec.

Signed-off-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3586)
(cherry picked from commit 72d8b823bbe749da528f386408541ae1daa644c9)

Wait longer for the server in TLSProxy to start

In a recent PR (#3566) it seems that TLSProxy gave up trying to connect to
the server process too quickly. This meant the test failed even though the
server *did* eventually start. Currently we try 3 times to connect with a
0.1 second pause between each attempt. That is probably too aggressive.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3587)
(cherry picked from commit 142463c9375efdcdaa4c504f6aaf5bfd61d5ba99)

Add stricter checking in NAME section

Require a comma between every name and a single space before the dash

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3559)
(cherry picked from commit 2bcb232ebeb155c6f1241deb84a26ab23176f866)

Make default_method mostly compile-time

Document thread-safety issues

Cherry-pick from 076fc55527a1499391fa6de109c8387895199ee9 but
keeps the RSA_null method.

Reviewed-by: Geoff Thorpe <geoff@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3146)

modes/ocb128.c: address undefined behaviour warning.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3526)

(cherry picked from commit 14bb100b6cc4c875473af6659c8368198e898050)

Fix a Proxy race condition

Issue #3562 describes a problem where a race condition can occur in the
Proxy such that a test "ok" line can appear in the middle of other text
causing the test harness to miss it. The issue is that we do not wait for
the client process to finish after the test is complete, so that process may
continue to write data to stdout/stderr at the same time that the test
harness does.

This commit fixes TLSProxy so that we always wait for the client process to
finish before continuing.

Fixes #3562

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3567)
(cherry picked from commit b72668a0d3586ee2560f0536c43e18991a4cfc6f)

Fix inconsistent check of UNSAFE_LEGACY_RENEGOTIATION

The check for SSL3_FLAGS_ALLOW_UNSAFE_LEGACY_RENEGOTIATION is
inconsistent. Most places check SSL->options, one place is checking
SSL_CTX->options; fix that.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
GH: #3523
(cherry picked from commit dffdcc773ac0a294b1ce620131cb8d7401da9408)

Fix use of "can_load()" in run_tests.pl.

CLA: Trivial

Fixes #3563.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3564)
(cherry picked from commit 418bb7b31bb7cfca6e419a5aa7bf161784f61059)

Add missing commas in pod files

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #3557
(cherry picked from commit 6061f80b5c55f03f9604d793cc06917b105bc4bc)

Document that HMAC() with a NULL md is not thread safe

Fixes #3541

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3554)

Prepare for 1.1.0g-dev

Reviewed-by: Stephen Henson <steve@openssl.org>

Prepare for 1.1.0f release

Reviewed-by: Stephen Henson <steve@openssl.org>

Update CHANGES and NEWS for new release

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3547)

Document that PKCS#12 functions assume UTF-8 for passwords

Part of issue #3531

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3535)
(cherry picked from commit cff85f39e438830cb5bc8bf7cb2ed6458670c5be)

Clarify what character encoding is used in the returned UI strings

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3532)
(cherry picked from commit 789d6dddecbc231f5c858d203aab318cf81676fd)

Fix typo in doc/man3/EVP_EncrypInit.pod

In the example section.

CLA: trivial

Signed-off-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3520)
(cherry picked from commit 719b289d62d32fe45226e8bc5b4fb0d76f6a1b5d)

[1.1.0 backport] set entry type on SCTs from X.509 and OCSP extensions

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3519)

Fix infinite loops in secure memory allocation.

Remove assertion when mmap() fails.
Only give the 1<<31 limit test as an example.

Fix the small arena test to just check for the symptom of the infinite
loop (i.e. initialized set on failure), rather than the actual infinite
loop. This avoids some valgrind errors.

Backport of:
PR #3512 commit fee423bb68869de02fceaceefbc847e98213574b
PR #3510 commit a486561b691d6293a901b412172ca0c6d1ffc0dc
PR #3455 commit c8e89d58a5d44b9dd657d6d13a5a10d1d4d30733
PR #3449 commit 7031ddac94d0ae616d1b0670263a9265ce672cd2

Issue 1:

sh.bittable_size is a size_t but i is and int, which can result in
freelist == -1 if sh.bittable_size exceeds an int.

This seems to result in an OPENSSL_assert due to invalid allocation
size, so maybe that is "ok."

Worse, if sh.bittable_size is exactly 1<<31, then this becomes an
infinite loop (because 1<<31 is a negative int, so it can be shifted
right forever and sticks at -1).

Issue 2:

CRYPTO_secure_malloc_init() sets secure_mem_initialized=1 even when
sh_init() returns 0.

If sh_init() fails, we end up with secure_mem_initialized=1 but
sh.minsize=0. If you then call secure_malloc(), which then calls,
sh_malloc(), this then enters an infite loop since 0 << anything will
never be larger than size.

Issue 3:

That same sh_malloc loop will loop forever for a size greater
than size_t/2 because i will proceed (assuming sh.minsize=16):
i=16, 32, 64, ..., size_t/8, size_t/4, size_t/2, 0, 0, 0, 0, ....
This sequence will never be larger than "size".

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3453)

Fix srp app missing NULL termination with password callback

The password_callback() function does not necessarily NULL terminate
the password buffer, the caller must use the returned length but the
srp app uses this function as if it was doing NULL termination.

This made the -passin and -passout options of "openssl srp"
fail inexpicably and randomly or even crash.

Fixed by enlarging the buffer by one, so that the maximum password length
remains unchanged, and adding NULL termination upon return.

[Rearrange code for coding style compliance in process.]

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3475)

(cherry picked from commit 0e83981d61fc435f42d4bb4d774272b69556b7bc)

test/run_tests.pl: don't mask test failures.

Switch to TAP::Harness inadvertently masked test failures.
Test::Harness::runtests was terminating with non-zero exit code in case
of failure[s], while TAP::Harness apparently holds caller responsible
for doing so.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

-inkey can be an identifier, not just a file

update pkcs12, smime, ts apps.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3507)
(cherry picked from commit 48b5352212d8c68f7fd071ca9f38822b7e954c5a)

Fix endless loop on srp app when listing users

With the -list option the srp app loops on the main while() endlessly,
whether users were given on the command line or not. The loop should
be stopped when in list mode and there are no more users.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3489)

(cherry picked from commit 5ec3210fc106ecc6badb48db6cfb1493a7607763)

Fix ASN1_TIME_to_generalizedtime to take a const ASN1_TIME

Fixes #1526

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3360)
(cherry picked from commit 9bfeeef8ee2220339e601a028fa991c30d296ed4)

Make SSL_is_server() accept a const SSL

Fixes #1526

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3360)
(cherry picked from commit 6944311688015ad293bd788ce78f3226738ebf00)

Remove notification settings from appveyor.yml

Notifications can be (and should be) configured on account basis on
the CI web site.  This avoids getting emails to openssl-commits for
personal accounts that also build OpenSSL stuff.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3484)
(cherry picked from commit 7a94f5b0f7c878b1056a08f659ce23aa97bfa3ad)

INSTALL: Remind people to read more if they added configuration options

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3469)
(cherry picked from commit 62f218cb8d31851935b8113a2a2236493b3510cc)

INSTALL: clarify a bit more how Configure treats "unknown" options

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3468)
(cherry picked from commit a3cb4cfc6af3f5fc1cd81ccd264daaa79d1c0a46)

Clean up SSL_OP_* a bit

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3439)
(cherry picked from commit 80a2fc4100daf6f1001eee33ef2f9b9eee05bedf)

Clean away needless VMS check

BIO_socket_ioctl is only implemented on VMS for VMS version 7.0 and
up, but since we only support version 7.1 and up, there's no need to
check the VMS version.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3448)
(cherry picked from commit b57f0c598bde43e147a886c9ffb0d6fdb3141d72)

Cleanup - use e_os2.h rather than stdint.h

Not exactly everywhere, but in those source files where stdint.h is
included conditionally, or where it will be eventually

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3447)
(cherry picked from commit 74a011ebb5e9028ef18982d737a434a8ff926a95)