On VMS, massage the fetch file names to remove the generation number

The generation number is ';nnn' at the end of the file name fetched
with readdir().  Because rehash checks for specific extensions and
doesn't expect an additional generation number, the easiest is to
massage the received file name early by simply removing the generation
number.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2717)
(cherry picked from commit 39aceac320a1561d50c7d71ac2560aec7ab8eddb)

Let the output from 'openssl enc -ciphers' go to stdout

Also, don't exit with an error code

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2716)
(cherry picked from commit 341de5f1997d21b60cee69be656f1ae709bccdac)

Fix typo, should be && rather than &

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2689)
(cherry picked from commit 50799f3558981eac0482d3ea77b21c58b56d4871)

Fix typo, missing ||

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2707)
(cherry picked from commit 6eb837583758506607f538fe2a3dd87925e4e69d)

Iterate over EC_GROUP's poly array in a safe way

Prevent that memory beyond the last element is accessed if every element
of group->poly[] is non-zero

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2689)

(cherry picked from commit 57f48f939ed5d3119e3c691ea0a8a3ac2f4a1a9e)

Make "openssl rehash" work on VMS 8.3 and up

A spelling error prevented it from building correctly.
Furthermore, we need to be more careful when to add a / at the end
of the dirname and when not.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2706)
(cherry picked from commit 5c80e2af3a7d8aa5129a1668c286c1464983e1ac)

Have the directory reader use the Unix API on VMS

opendir(), readdir() and closedir() have been available on VMS since
version 7.0.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2707)
(cherry picked from commit d8eaaf15356e1559f0f669b430b0d22b3514f8f0)

Add some more consistency checks in tls_decrypt_ticket.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2704)
(cherry picked from commit 79020b27beff060d02830870fdfd821fe8cbd439)

Fix i2d_SSL_SESSION pp output parameter should point to end of asn1 data.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2607)
(cherry picked from commit a0179d0afb621a0875ddcfd939719a9628ac4444)

Fix memory leak in pkcs12 -export

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2676)
(cherry picked from commit 1b8f19379a521ec11ce37e12316dd3edc0acfb82)

Fix some more memory leaks with TXT_DB_insert.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2684)
(cherry picked from commit 0fbaef9e64fa10446aff805791befaa2b967e322)

Fix a few memleaks in TXT_DB.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2684)
(cherry picked from commit 9ad52c562a93c9a57ae3024e54c575430753244c)

Prevent OOB in SRP base64 code.

Change size comparison from > (GT) to >= (GTE) to ensure an additional
byte of output buffer, to prevent OOB reads/writes later in the function
Reject input strings larger than 2GB
Detect invalid output buffer size and return early

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2672)

(cherry picked from commit ecca16632a73bb80ee27cdec8a97f6def0a4714d)

Removed ugly size_t less than zero check.

CLA: trivial.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2674)

(cherry picked from commit 5e1f879ab5a2bfdf2d58222f965f93fe1b511ce7)

Ensure minsize >= sizeof(SH_LIST)

The sh_add_to_list function will overwrite subsequent slots in the free list
for small allocations.  This causes a segmentation fault if the writes goes
off the end of the secure memory.  I've not investigated if this problem
can overwrite memory without the segmentation fault, but it seems likely.

This fix limits the minsize to the sizeof of the SH_LIST structure (which
also has a side effect of properly aligning the pointers).

The alternative would be to return an error if minsize is too small.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2657)

(cherry picked from commit 70e14ffbaf6a67dab56c24cae01f1248cf3f1e77)

Don't call memcpy if len is zero.

Prevent undefined behavior in CRYPTO_cbc128_encrypt: calling this function
with the 'len' parameter being 0 would result in a memcpy where the source
and destination parameters are the same, which is undefined behavior.
Do same for AES_ige_encrypt.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2671)

(cherry picked from commit b1498c98f3fb5b8a340acc9ce20b0fd5346294e5)

VMS fix of test/recipes/80-test_ssl_new.t

On VMS, file names with more than one period get all but the last get
escaped with a ^, so 21-key-update.conf.in becomes 21-key-update^.conf.in
That means that %conf_dependent_tests and %skip become useless unless
we massage the file names that are used as indexes.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2678)
(cherry picked from commit d89f66412ba5168e7d6fd9dd88619d927d716f55)

If all versions of a proto are disabled, disabled the proto as well

For example, 'no-dtls1 no-dtls1_2' will imply 'no-dtls'

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2670)
(cherry picked from commit 343a7467c270c54a8e1c85e88e807a1c2e0b6127)

Fix a slightly confusing if condition in a2i_ASN1_INTEGER.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2640)
(cherry picked from commit aa402e2ba408254c052b5750b14e7f01e48bced1)

Fix test_x509_store

Don't run this test unless 'openssl rehash' works properly.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2664)
(cherry picked from commit 73540f4729bb856ab066c6e7a57513a97e3ca36f)

Add a test of the X509_STORE / X509_LOOKUP API

Fortunately, "openssl verify" makes good use of that API

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2652)
(cherry picked from commit bb0f7eca75b8da1538c08c1f5be1bb7ea8f40638)

test/README: clarify test number groups

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2652)
(cherry picked from commit 532e7b36d9622ac06a96fb3557b5bc16016e5ca8)

Fix a mem leak in ssl_test_ctx.c

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2650)
(cherry picked from commit d605fc3a0ce4103ca6660904795bf1209cdb55b7)

Don't run MSBLOB conversion tests when RSA or DSA are disabled

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2174)
(cherry picked from commit d8594555ffaf98ada08b26ce3d1138f16bc029c5)

Change req_check_len error message

it also accepts 20 bytes, but states 'less than' in the error message

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2648)
(cherry picked from commit 0cb8c9d85e9d5690670d6f1f02e8ccc756520210)

Use _WIN32 over WIN32 for preprocessor conditional

The intent seems to be that the WIN32 symbol is for things that are a direct
byproduct of being a windows-variant configuration and should be used for
feature en/disablement on windows systems.  Use of the _WIN32 symbol is more
widespread, being used to implement platform portability of more generic code.

We do define WIN32 in some situations in e_os.h, but that is not included
universally.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2642)
(cherry picked from commit ac879ed62a19f3c878f7be3020a1b93cc77f4b38)

Prepare for 1.1.0f-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.0e release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update CHANGES and NEWS for new release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove an OPENSSL_assert() and replace with a soft assert and check

Following on from CVE-2017-3733, this removes the OPENSSL_assert() check
that failed and replaces it with a soft assert, and an explicit check of
value with an error return if it fails.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't change the state of the ETM flags until CCS processing

Changing the ciphersuite during a renegotiation can result in a crash
leading to a DoS attack. ETM has not been implemented in 1.1.0 for DTLS
so this is TLS only.

The problem is caused by changing the flag indicating whether to use ETM
or not immediately on negotiation of ETM, rather than at CCS. Therefore,
during a renegotiation, if the ETM state is changing (usually due to a
change of ciphersuite), then an error/crash will occur.

Due to the fact that there are separate CCS messages for read and write
we actually now need two flags to determine whether to use ETM or not.

CVE-2017-3733

Reviewed-by: Richard Levitte <levitte@openssl.org>

Provide a test for the Encrypt-Then-Mac renegotiation crash

Changing the ciphersuite during a renegotiation can result in a crash
leading to a DoS attack. ETM has not been implemented in 1.1.0 for DTLS
so this is TLS only.

This commit provides a test for the issue.

CVE-2017-3733

Reviewed-by: Richard Levitte <levitte@openssl.org>

Properly zero cipher_data for ChaCha20-Poly1305 on cleanup

Fix a typo. Probably this has not been found because EVP_CIPHER_CTX is
smaller than EVP_CHACHA_AEAD_CTX and heap overflow does not occur.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2294)
(cherry picked from commit a8f957686675194d786b41f6e1f7c48bb85723ec)

crypto/armcap.c: short-circuit processor capability probe in iOS builds.

Capability probing by catching SIGILL appears to be problematic
on iOS. But since Apple universe is "monocultural", it's actually
possible to simply set pre-defined processor capability mask.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2617)

(cherry picked from commit 8653e78f4319b23d60239f9557d8c1e1d23be1a5)

ARMv4 assembly pack: harmonize Thumb-ification of iOS build.

Three modules were left behind in a285992763f3961f69a8d86bf7dfff020a08cef9.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2617)

(cherry picked from commit c93f06c12f10c07cea935abd78a07a037e27f155)

Rework error handling of custom_ext_meth_add towards strong exception safety.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2636)
(cherry picked from commit ed874fac6399d5064d6eb8fe2022b918aeaf75af)

Fix a few typos

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2571)
(cherry picked from commit 7e12cdb52e3f4beff050caeecf3634870bb9a7c4)

Remove obsolete comment

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1613)
(cherry picked from commit 7c120357e5ef434c8a7d1d1c3ba4f2a33266374e)

Use TLSEXT_KEYNAME_LENGTH in tls_decrypt_ticket.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2618)
(cherry picked from commit 57b0d651f052ed86528da916397acbcce035fb21)

Prevent allocations of size 0 in sh_init.

which are not possible with the default OPENSSL_zalloc, but are possible if
the user has installed their own allocator using CRYPTO_set_mem_functions. If
the 0-allocations succeeds, the secure heap code will later access
(at least) the first byte of that space, which is technically an OOB
access. This could lead to problems with some custom allocators that only
return a valid pointer for subsequent free()-ing, and do not expect that
the pointer is actually dereferenced.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2605)
(cherry picked from commit 7f07149d25f8d7e00e9350ff2f064a4d25c1a13d)

Make -xcert work again.

When a certificate is prepended update the list pointer.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2628)
(cherry picked from commit 52f4840cb237cc37cad5eac8328828cf3d3e1049)

Add no-ec build

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2626)
(cherry picked from commit b4568b04c7cd425103ac8f1603682e8da2044238)

mem leak on error path and error propagation fix

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2559)
(cherry picked from commit e0670973d5c0b837eb5a9f1670e47107f466fbc7)

Further improvements to ASYNC_WAIT_CTX_clear_fd

Remove call to cleanup function
Use only one loop to find previous element

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2581)
(cherry picked from commit 219aa86cb04e1bfc9c156fab18da2f767502afb2)

Remove fd from the list when the engine clears the wait context before pause

This fixes the num of fds added/removed returned by ASYNC_WAIT_CTX_get_changed_fds

Previously, the numbers were not consistent with the fds actually written in
the buffers since the fds that have been both added and removed are explicitly
ignored in the loop.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2581)
(cherry picked from commit f89dd6738a0ec2b6cfb05a3cc5fa38843dc27d2f)

Add test to show wrong behavior of ASYNC_WAIT_CTX

This happens when a fd is added and then immediately removed from the
ASYNC_WAIT_CTX before pausing the job.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2581)
(cherry picked from commit f44e63644d29e5908be52b7896d5031a5cf460eb)

DES keys are not 7 days long.

CLA: trivial
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2604)

(cherry picked from commit 4fd7b54dc224930a0ce6dd67b35c598c5072857c)

test_rehash does nothing, have it do something

test/recipes/40-test_rehash.t uses test files from certs/demo, which
doesn't exist any longer.  Have it use PEM files from test/ instead.

Because rehash wants only one certificate or CRL per file, we must
also filter those PEM files to produce test files with a single object
each.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2594)
(cherry picked from commit 4bbd8a5daaa810c487f684971c0339a1d7c15da9)

Restore EVP_CIPH_FLAG_LENGTH_BITS working properly

EVP_CIPH_FLAG_LENGTH_BITS flag for CFB1 has been broken with the
introduction of the is_partially_overlapping() check that did not take
it into the account (treating number of bits passed as bytes). This
remedies that and allows this flag to work as intended.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1942)
(cherry picked from commit 64846096b18340b9a39ddd29a7a0e23c56f22959)

Don't read uninitialised data for short session IDs.

While it's always safe to read |SSL_MAX_SSL_SESSION_ID_LENGTH| bytes
from an |SSL_SESSION|'s |session_id| array, the hash function would do
so with without considering if all those bytes had been written to.

This change checks |session_id_length| before possibly reading
uninitialised memory. Since the result of the hash function was already
attacker controlled, and since a lookup of a short session ID will
always fail, it doesn't appear that this is anything more than a clean
up.

In particular, |ssl_get_prev_session| uses a stack-allocated placeholder
|SSL_SESSION| as a lookup key, so the |session_id| array may be
uninitialised.

This was originally found with libFuzzer and MSan in
https://boringssl.googlesource.com/boringssl/+/e976e4349d693b4bbb97e1694f45be5a1b22c8c7,
then by Robert Swiecki with honggfuzz and MSan here. Thanks to both.

Reviewed-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2583)
(cherry picked from commit bd5d27c1c6d3f83464ddf5124f18a2cac2cbb37f)

Fix a typo in the X509_get0_subject_key_id() documentation

Fixes a copy&paste error

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2568)
(cherry picked from commit fbc9eeaaa32ba1416d6cb2794201f440bbaeb629)

Centralize documentation about config file location

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2569)
(cherry picked from commit e9681f8314c64c6802b11997c471bd763de38c8c)

Remove unused variable

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2566)
(cherry picked from commit a19a6c8179faa3da0dedaaf2effae385cf7dd65d)

Fix a crash in EVP_CIPHER_CTX_cleanup due to cipher_data may be NULL
or EVP_CTRL_INIT/EVP_CTRL_COPY was not called or failed.
If that happens in EVP_CipherInit_ex/EVP_CIPHER_CTX_copy set cipher = NULL,
aes_gcm_cleanup should check that gctx != NULL before calling OPENSSL_cleanse.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2562)
(cherry picked from commit 273a0218e65f1737cdbb0ef65a5ddebd601e6bef)

Fix parsing of serial# in req

Reported by Jakub Wilk.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2556)
(cherry picked from commit be4c82aa767998ce2a5717fc895482052373f1b1)

Doc fix

Reported by Alexander Köppe

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2557)
(cherry picked from commit bb6c5e7f6956c5cd1049136d79e631ca8338fc7b)

Fix a crash with malformed user notice policy numbers

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2352)
(cherry picked from commit fe4075f25962dbd302d856c11ac47adb84edc9ca)

Combined patch against OpenSSL_1_1_0-stable branch for the following issues:

Fixed a memory leak in ASN1_digest and ASN1_item_digest.
Reworked error handling in asn1_item_embed_new.
Fixed error handling in int_ctx_new and EVP_PKEY_CTX_dup.
Fixed a memory leak in CRYPTO_free_ex_data.
Reworked error handing in x509_name_ex_d2i, x509_name_encode and x509_name_canon.
Check for null pointer in tls_process_cert_verify.

Fixes #2103 #2104 #2105 #2109 #2111 #2115

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2163)

Majority rules, use session_ctx vs initial_ctx

session_ctx and initial_ctx are aliases of each other, and with the
opaque data structures, there's no need to keep both around. Since
there were more references of session_ctx, replace all instances of
initial_ctx with session_ctx.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2340)

Grouped data declarations [skip ci]
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit bc1dba209533f2033a4de0d93380fc0f485e6f7e)

Removed tab spaces.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit 5d5eed4456ebc035893eedbcc4e32a9d065cecb3)

Corrections according the review comments.

Updated indentations according project rules, renamed file-local define to the shorter version - USE_RWLOCK, fixed declaration after the if statement in CRYPTO_THREAD_lock_new().
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit ec93a2921f6128ac931466ae171fba92a0eab45d)

Compile fix on platforms with missing pthread_rwlock_t.

Fix compilation on platforms with missing pthread_rwlock_t implementation by replacing it with pthread_mutex_t. An example of such platform can be Android OS 2.0 - 2.1, API level 5 (Eclair), Android NDK platform - android-5 where pthread_rwlock_t is not implemented and is missing in pthread.h.

In case of missing pthread_rwlock_t implementation CRYPTO_RWLOCK will work as exclusive lock in write-only mode of pthread_rwlock_t lock.

The implementation based on pthread_mutex_t must be using PTHREAD_MUTEX_RECURSIVE mode to be compatible with recursive behavior of pthread_rwlock_rdlock.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit 2accf3f7e013c3d02312afc27cc2edbd1f149db3)

remove test/.rnd on make clean

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2344)
(cherry picked from commit 122fa088524571a3b60ebf301873f69afdac8f7a)

bn: fix occurance of negative zero in BN_rshift1()

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit 0a2dcb6990dacc94337f746f4f4a6dfac1fbeac4)

bn: fix occurances of negative zero

The BIGNUM behaviour is supposed to be "consistent" when going into and
out of APIs, where "consistent" means 'top' is set minimally and that
'neg' (negative) is not set if the BIGNUM is zero (which is iff 'top' is
zero, due to the previous point).

The BN_DEBUG testing (make test) caught the cases that this patch
corrects.

Note, bn_correct_top() could have been used instead, but that is intended
for where 'top' is expected to (sometimes) require adjustment after direct
word-array manipulation, and so is heavier-weight. Here, we are just
catching the negative-zero case, so we test and correct for that
explicitly, in-place.

Change-Id: Iddefbd3c28a13d935648932beebcc765d5b85ae7
Signed-off-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1672)
(cherry picked from commit 38d1b3cc0271008b8bd130a2c4b442775b028a08)

bn: catch negative zero as an error

Change-Id: I5ab72ad0aae9069b47d5b7b7b9e25bd1b7afa251
Signed-off-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1672)
(cherry picked from commit 2fc9b36a96ccd77cbd9ecfb3a3cdaa7ad2ca305e)

bn: fix BN_DEBUG + BN_DEBUG_RAND support

Couple of updates to make this code work properly again;
* use OPENSSL_assert() instead of assert() (and #include <assert.h>)
* the circular-dependency-avoidance uses RAND_bytes() (not pseudo)

Change-Id: Iefb5a9dd73f71fd81c1268495c54a64378955354
Signed-off-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1672)
(cherry picked from commit 0b50ac1a0fe907f4effcf3f2f36dac32523938c5)

Fix faulty free

On error, i2o_SCT_signature() and i2o_SCT() free a pointer that may
have wandered off from the start of the allocated block (not currently
true for i2o_SCT_signature(), but has that potential as the code may
change.  To avoid this, save away the start of the allocated block and
free that instead.

Thanks to Guido Vranken for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2323)
(cherry picked from commit d85d3c993e322d3e4c3f00be2910faa8c55b40e3)

test/evp_test.c: If no algorithm was specified, don't try to check for DES

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2319)
(cherry picked from commit 31b69e9a26c5b127ce273bc5834b9e13e5e25556)

Add a couple of test to check CRL fingerprint

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2314)
(cherry picked from commit 929860d0e6112f5c7766d9ea036c3f8bd8d3d719)

Document what EXFLAG_SET is for in x509v3.h

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2314)
(cherry picked from commit 2d60c923141e7853c268364f26195343a5e995bf)

X509_CRL_digest() - ensure precomputed sha1 hash before returning it

X509_CRL_digest() didn't check if the precomputed sha1 hash was actually
present.  This also makes sure there's an appropriate flag to check.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2314)
(cherry picked from commit 6195848b2eea627c47f74b63eb2ba3dc3d5b6436)

Correct pointer to be freed

The pointer that was freed in the SSLv2 section of ssl_bytes_to_cipher_list
may have stepped up from its allocated position.  Use a pointer that is
guaranteed to point at the start of the allocated block instead.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2312)
(cherry picked from commit 63414e64e66e376654e993ac966e3b2f9d849d3b)

Travis: run on Trusty with clang 3.9 (1.1.0 branch)

See https://github.com/travis-ci/travis-ci/issues/6460 for context on the changes to wine install.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add server signature algorithm bug test.

Add a client authentication signature algorithm to simple
ssl test and a server signature algorithm. Since we don't
do client auth this should have no effect. However if we
use client auth signature algorithms by mistake this will
abort the handshake with a no shared signature algorithms
error.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2297)

Use correct signature algorithm list when sending or checking.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2297)

Prepare for 1.1.0e-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.0d release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update CHANGES and NEWS for new release

Reviewed-by: Richard Levitte <levitte@openssl.org>

bn/asm/x86_64-mont5.pl: fix carry bug in bn_sqr8x_internal.

CVE-2017-3732

Reviewed-by: Rich Salz <rsalz@openssl.org>

Document DH_check_params()

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Better check of DH parameters in TLS data

When the client reads DH parameters from the TLS stream, we only
checked that they all are non-zero.  This change updates the check to
use DH_check_params()

DH_check_params() is a new function for light weight checking of the p
and g parameters:

    check that p is odd
    check that 1 < g < p - 1

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

crypto/evp: harden AEAD ciphers.

Originally a crash in 32-bit build was reported CHACHA20-POLY1305
cipher. The crash is triggered by truncated packet and is result
of excessive hashing to the edge of accessible memory. Since hash
operation is read-only it is not considered to be exploitable
beyond a DoS condition. Other ciphers were hardened.

Thanks to Robert Święcki for report.

CVE-2017-3731

Reviewed-by: Rich Salz <rsalz@openssl.org>

crypto/evp: harden RC4_MD5 cipher.

Originally a crash in 32-bit build was reported CHACHA20-POLY1305
cipher. The crash is triggered by truncated packet and is result
of excessive hashing to the edge of accessible memory (or bogus
MAC value is produced if x86 MD5 assembly module is involved). Since
hash operation is read-only it is not considered to be exploitable
beyond a DoS condition.

Thanks to Robert Święcki for report.

CVE-2017-3731

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove assert from is_partially_overlapping()

This function is used to validate application supplied parameters. An
assert should be used to check for an error that is internal to OpenSSL.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2275)
(cherry picked from commit b153f0921bea38127de0b9440b0487db3004330d)

Adjust in and in_len instead of donelen

Don't use the temp variable donelen in the non-aad fragmented code path.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2275)
(cherry picked from commit ef055ec536a3c6e698dd9f45b41d57a32366b4fa)

Fix the overlapping check for fragmented "Update" operations

When doing in place encryption the overlapping buffer check can fail
incorrectly where we have done a partial block "Update" operation. This
fixes things to take account of any pending partial blocks.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2275)
(cherry picked from commit 7141ba31969d0b378d08104a51f8f99b9187b9d5)

Update evp_test to make sure passing partial block to "Update" is ok

The previous commit fixed a bug where a partial block had been passed to
an "Update" function and it wasn't properly handled. We should catch this
type of error in evp_test.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2275)
(cherry picked from commit 0b96d77a62d8ac9a45ac1dda47560ced676b5b8d)

Properly handle a partial block in OCB mode

If we have previously been passed a partial block in an "Update" call then
make sure we properly increment the output buffer when we use it.

Fixes #2273

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2275)
(cherry picked from commit 7c12c7b61c5b37c9dff930ccc68421fb7de00271)

Don't use magic numbers in aes_ocb_cipher()

Lots of references to 16 replaced by AES_BLOCK_SIZE. Also a few other style
tweaks in that function

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2275)
(cherry picked from commit 0ba5a9eaa0a6ae7fc25ee70eefc1f4fbdaf09483)

s_client: Better response success check for CONNECT

Instead of looking for "200" and "established" (and failing all other
2xx responses or "Established"), let's look for a line that's not a
header (i.e. doesn't contain a ':') and where the first space is
followed by a '2'.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1664)
(cherry picked from commit ec2a0e60652c0e61e90dde367756c5d92cd882d3)

Replace div-spoiler hack with simpler code, GH#1027,2253.

This is 1.1.0-specific 8f77fab82486c19ab48eee07718e190f76e6ea9a redux.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Cleanup EVP_CIPH/EP_CTRL duplicate defines

Remove duplicate defines from EVP source files.
Most of them were in evp.h, which is always included.
Add new ones evp_int.h
EVP_CIPH_FLAG_TLS1_1_MULTIBLOCK is now always defined in evp.h, so
remove conditionals on it

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2201)
(cherry picked from commit 9d6fcd4295fef7ebc4232aab85718a99d36cc50a)

Fix error handling in compute_key, BN_CTX_get can return NULL

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2132)
(cherry picked from commit 7928ee4d685b727619555bc1ec0aee805f6fc8c4)

Do not overallocate for tmp.ciphers_raw

Well, not as much, at least.

Commit 07afdf3c3ac97af4f2b4eec22a97f7230f8227e0 changed things so
that for SSLv2 format ClientHellos we store the cipher list in the
TLS format, i.e., with two bytes per cipher, to be consistent with
historical behavior.

However, the space allocated for the array still performed the computation
with three bytes per cipher, a needless over-allocation (though a relatively
small one, all things considered).

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2281)
(cherry picked from commit f1429b85c5821e55224e5878da9d0fa420a41f71)

Fix SSL_get0_raw_cipherlist()

SSL_get0_raw_cipherlist() was a little too "raw" in the case of an SSLv2
compat ClientHello. In 1.0.2 and below, during version negotiation, if
we received an SSLv2 compat ClientHello but actually wanted to do SSLv3+
then we would construct a "fake" SSLv3+ ClientHello. This "fake" ClientHello
would have its ciphersuite list converted to the SSLv3+ format. It was
this "fake" raw list that got saved away to later be returned by a call to
SSL_get0_raw_cipherlist().

In 1.1.0+ version negotiation works differently and we process an SSLv2
compat ClientHello directly without the need for an intermediary "fake"
ClientHello. This meant that the raw ciphersuite list being saved was in
the SSLv2 format. Any caller of this function would not expect that and
potentially overread the returned buffer by one byte.

Fixes #2189

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2280)
(cherry picked from commit 07afdf3c3ac97af4f2b4eec22a97f7230f8227e0)

Add missing flags for EVP_chacha20()

ChaCha20 code uses its own custom cipher_data. Add EVP_CIPH_CUSTOM_IV
and EVP_CIPH_ALWAYS_CALL_INIT so that the key and the iv can be set by
different calls of EVP_CipherInit_ex().

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2156)
(cherry picked from commit c83680a04aac7a15e6ac48ed732b8322c4c9d49a)

Fix a ssl session leak due to OOM in lh_SSL_SESSION_insert

- s == NULL can mean c is a new session *or* lh_insert was
  unable to create a hash entry.
- use lh_SSL_SESSION_retrieve to check for this error condition.
- If it happens simply remove the extra reference again.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2138)
(cherry picked from commit 38088ce9934a90d4aea486edbff864f3935342e6)

RAND_egd_bytes: No need to check RAND_status on connection error.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1886)
(cherry picked from commit c2114afc1622ff0113974b3696e557ea8bf7ffb4)