Add AGL's "beer mug" PEM file as another test input

AGL has a history of pointing out the idiosynchronies/laxness of the
openssl PEM parser in amusing ways.  If we want this functionality to
stay present, we should test that it works.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2756)
(cherry picked from commit a00b9560f7ece1e51bd7a8dc6a7ffb7a3d20cf86)

Add test corpus for PEM reading

Generate a fresh certificate and DSA private key in their respective PEM
files.  Modify the resulting ASCII in various ways so as to produce input
files that might be generated by non-openssl programs (openssl always
generates "standard" PEM files, with base64 data in 64-character lines
except for a possible shorter last line).

Exercise various combinations of line lengths, leading/trailing
whitespace, non-base64 characters, comments, and padding, for both
unencrypted and encrypted files.  (We do not have any other test coverage
that uses encrypted files, as far as I can see, and the parser enforces
different rules for the body of encrypted files.)

Add a recipe to parse these test files and verify that they contain the
expected string or are rejected, according to the expected status.
Some of the current behavior is perhaps suboptimal and could be revisited.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2756)
(cherry picked from commit e8cee55718bb9cb957f449fbe7145a77f252bb73)

Code health: make update

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2793)
(cherry picked from commit 77baccd7fac7cf480e3a3981b7deae5ef3b812b9)

Code health: Remove VAX exceptions in util/mkdef.pl

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2793)
(cherry picked from commit 96bc5d03b813a318403d45600e07d6bdcb41d195)

Code health: Remove unused VAX transfer vector for engines

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2793)
(cherry picked from commit 38a322a5f29ae0b4a9bd42233310835487d875ac)

Exdata test was never enabled.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2787)
(cherry picked from commit 629192c1b9f17965e0a6b73229b7b1e004bfbd98)

Fix test_ssl_new when compiled with no-tls1_2 or no-dtls1_2

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2788)
(cherry picked from commit 4d118fe007692de2dd8c5dd084254f8d3b308167)

Update year, wording tweak

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2755)
(cherry picked from commit 6faa3456326afa56ea8c25a0b49239392074e192)

Code cleanup: remove the VMS specific reimplementation of gmtime

This reimplementation was necessary before VMS C V7.1.  Since that's
the minimum version we support in this OpenSSL version, the
reimplementation is no longer needed.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2762)
(cherry picked from commit 9d70ac97d9d8720e6ed280609c844da403b80440)

Adjust the default value of the private key size

...in the man page to reflect the actual default (2048 instead of 512)

CLA: trivial
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2754)

(cherry picked from commit 013bc448672cbc3c9cd154709400c676c2955229)

.travis.yml: limit mingw tests' resource consumption.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2735)

.travis.yml: make package pulls conditional.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2735)

Increase the size of the stack buffer to prevent an overflow.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2721)
(cherry picked from commit 8fce04ee3540ba3039bb66df34ea3f076a599ab9)

.travis.yml: remove osx from build matrix.

Travis OS X utilization and backlog statistics suggest that it became
bottleneck for our integration builds with requests piling up for days
during working days of the week. Suggestion is to remove osx till
capacity is lesser issue.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit e12e903e9ac675d08f9dd0db1f0c1a2049232c21)

Fix potential memory leak in ASN1_TIME_to_generalizedtime()

If ret is allocated, it may be leaked on error.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2666)
(cherry picked from commit 4483e23444fa18034344874ffbe67919207e9e47)

Add -Wundef to strict-warnings

Avoid a -Wundef warning in o_str.c
Avoid a -Wundef warning in testutil.h
Include internal/cryptlib.h before openssl/stack.h
to avoid use of undefined symbol OPENSSL_API_COMPAT.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2709)

Check for the presence of _WIN32 rather than its value.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2727)
(cherry picked from commit 46958a043d51633ed36bcfb13ff048a3381366a6)

In apps/rehash.c, decorate the inclusion of internal/o_dir.h for VMS

The library files are built with symbol names as is, while the
application is built with the default uppercase-all-symbols mode.
That's fine for public APIs, because we have __DECC_INCLUDE_PROLOGUE.H
and __DECC_INCLUDE_EPILOGUE.H automatically telling the compiler how
to treat the public header files.  However, we don't have the same
setup for internal library APIs, since they are usually only used by
the libraries.

Because apps/rehash.c uses a library internal header file, we have to
surround that inclusion with the same kind of pragmas found in
__DECC_INCLUDE_PROLOGUE.H and __DECC_INCLUDE_EPILOGUE.H, or we get
unresolved symbols when building no-shared.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2725)
(cherry picked from commit 2ac915f16218982f48dbc799b8308a07441d2e35)

On VMS, massage the fetch file names to remove the generation number

The generation number is ';nnn' at the end of the file name fetched
with readdir().  Because rehash checks for specific extensions and
doesn't expect an additional generation number, the easiest is to
massage the received file name early by simply removing the generation
number.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2717)
(cherry picked from commit 39aceac320a1561d50c7d71ac2560aec7ab8eddb)

Let the output from 'openssl enc -ciphers' go to stdout

Also, don't exit with an error code

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2716)
(cherry picked from commit 341de5f1997d21b60cee69be656f1ae709bccdac)

Fix typo, should be && rather than &

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2689)
(cherry picked from commit 50799f3558981eac0482d3ea77b21c58b56d4871)

Fix typo, missing ||

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2707)
(cherry picked from commit 6eb837583758506607f538fe2a3dd87925e4e69d)

Iterate over EC_GROUP's poly array in a safe way

Prevent that memory beyond the last element is accessed if every element
of group->poly[] is non-zero

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2689)

(cherry picked from commit 57f48f939ed5d3119e3c691ea0a8a3ac2f4a1a9e)

Make "openssl rehash" work on VMS 8.3 and up

A spelling error prevented it from building correctly.
Furthermore, we need to be more careful when to add a / at the end
of the dirname and when not.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2706)
(cherry picked from commit 5c80e2af3a7d8aa5129a1668c286c1464983e1ac)

Have the directory reader use the Unix API on VMS

opendir(), readdir() and closedir() have been available on VMS since
version 7.0.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2707)
(cherry picked from commit d8eaaf15356e1559f0f669b430b0d22b3514f8f0)

Add some more consistency checks in tls_decrypt_ticket.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2704)
(cherry picked from commit 79020b27beff060d02830870fdfd821fe8cbd439)

Fix i2d_SSL_SESSION pp output parameter should point to end of asn1 data.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2607)
(cherry picked from commit a0179d0afb621a0875ddcfd939719a9628ac4444)

Fix memory leak in pkcs12 -export

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2676)
(cherry picked from commit 1b8f19379a521ec11ce37e12316dd3edc0acfb82)

Fix some more memory leaks with TXT_DB_insert.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2684)
(cherry picked from commit 0fbaef9e64fa10446aff805791befaa2b967e322)

Fix a few memleaks in TXT_DB.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2684)
(cherry picked from commit 9ad52c562a93c9a57ae3024e54c575430753244c)

Prevent OOB in SRP base64 code.

Change size comparison from > (GT) to >= (GTE) to ensure an additional
byte of output buffer, to prevent OOB reads/writes later in the function
Reject input strings larger than 2GB
Detect invalid output buffer size and return early

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2672)

(cherry picked from commit ecca16632a73bb80ee27cdec8a97f6def0a4714d)

Removed ugly size_t less than zero check.

CLA: trivial.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2674)

(cherry picked from commit 5e1f879ab5a2bfdf2d58222f965f93fe1b511ce7)

Ensure minsize >= sizeof(SH_LIST)

The sh_add_to_list function will overwrite subsequent slots in the free list
for small allocations.  This causes a segmentation fault if the writes goes
off the end of the secure memory.  I've not investigated if this problem
can overwrite memory without the segmentation fault, but it seems likely.

This fix limits the minsize to the sizeof of the SH_LIST structure (which
also has a side effect of properly aligning the pointers).

The alternative would be to return an error if minsize is too small.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2657)

(cherry picked from commit 70e14ffbaf6a67dab56c24cae01f1248cf3f1e77)

Don't call memcpy if len is zero.

Prevent undefined behavior in CRYPTO_cbc128_encrypt: calling this function
with the 'len' parameter being 0 would result in a memcpy where the source
and destination parameters are the same, which is undefined behavior.
Do same for AES_ige_encrypt.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2671)

(cherry picked from commit b1498c98f3fb5b8a340acc9ce20b0fd5346294e5)

VMS fix of test/recipes/80-test_ssl_new.t

On VMS, file names with more than one period get all but the last get
escaped with a ^, so 21-key-update.conf.in becomes 21-key-update^.conf.in
That means that %conf_dependent_tests and %skip become useless unless
we massage the file names that are used as indexes.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2678)
(cherry picked from commit d89f66412ba5168e7d6fd9dd88619d927d716f55)

If all versions of a proto are disabled, disabled the proto as well

For example, 'no-dtls1 no-dtls1_2' will imply 'no-dtls'

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2670)
(cherry picked from commit 343a7467c270c54a8e1c85e88e807a1c2e0b6127)

Fix a slightly confusing if condition in a2i_ASN1_INTEGER.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2640)
(cherry picked from commit aa402e2ba408254c052b5750b14e7f01e48bced1)

Fix test_x509_store

Don't run this test unless 'openssl rehash' works properly.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2664)
(cherry picked from commit 73540f4729bb856ab066c6e7a57513a97e3ca36f)

Add a test of the X509_STORE / X509_LOOKUP API

Fortunately, "openssl verify" makes good use of that API

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2652)
(cherry picked from commit bb0f7eca75b8da1538c08c1f5be1bb7ea8f40638)

test/README: clarify test number groups

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2652)
(cherry picked from commit 532e7b36d9622ac06a96fb3557b5bc16016e5ca8)

Fix a mem leak in ssl_test_ctx.c

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2650)
(cherry picked from commit d605fc3a0ce4103ca6660904795bf1209cdb55b7)

Don't run MSBLOB conversion tests when RSA or DSA are disabled

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2174)
(cherry picked from commit d8594555ffaf98ada08b26ce3d1138f16bc029c5)

Change req_check_len error message

it also accepts 20 bytes, but states 'less than' in the error message

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2648)
(cherry picked from commit 0cb8c9d85e9d5690670d6f1f02e8ccc756520210)

Use _WIN32 over WIN32 for preprocessor conditional

The intent seems to be that the WIN32 symbol is for things that are a direct
byproduct of being a windows-variant configuration and should be used for
feature en/disablement on windows systems.  Use of the _WIN32 symbol is more
widespread, being used to implement platform portability of more generic code.

We do define WIN32 in some situations in e_os.h, but that is not included
universally.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2642)
(cherry picked from commit ac879ed62a19f3c878f7be3020a1b93cc77f4b38)

Prepare for 1.1.0f-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.0e release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update CHANGES and NEWS for new release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove an OPENSSL_assert() and replace with a soft assert and check

Following on from CVE-2017-3733, this removes the OPENSSL_assert() check
that failed and replaces it with a soft assert, and an explicit check of
value with an error return if it fails.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't change the state of the ETM flags until CCS processing

Changing the ciphersuite during a renegotiation can result in a crash
leading to a DoS attack. ETM has not been implemented in 1.1.0 for DTLS
so this is TLS only.

The problem is caused by changing the flag indicating whether to use ETM
or not immediately on negotiation of ETM, rather than at CCS. Therefore,
during a renegotiation, if the ETM state is changing (usually due to a
change of ciphersuite), then an error/crash will occur.

Due to the fact that there are separate CCS messages for read and write
we actually now need two flags to determine whether to use ETM or not.

CVE-2017-3733

Reviewed-by: Richard Levitte <levitte@openssl.org>

Provide a test for the Encrypt-Then-Mac renegotiation crash

Changing the ciphersuite during a renegotiation can result in a crash
leading to a DoS attack. ETM has not been implemented in 1.1.0 for DTLS
so this is TLS only.

This commit provides a test for the issue.

CVE-2017-3733

Reviewed-by: Richard Levitte <levitte@openssl.org>

Properly zero cipher_data for ChaCha20-Poly1305 on cleanup

Fix a typo. Probably this has not been found because EVP_CIPHER_CTX is
smaller than EVP_CHACHA_AEAD_CTX and heap overflow does not occur.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2294)
(cherry picked from commit a8f957686675194d786b41f6e1f7c48bb85723ec)

crypto/armcap.c: short-circuit processor capability probe in iOS builds.

Capability probing by catching SIGILL appears to be problematic
on iOS. But since Apple universe is "monocultural", it's actually
possible to simply set pre-defined processor capability mask.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2617)

(cherry picked from commit 8653e78f4319b23d60239f9557d8c1e1d23be1a5)

ARMv4 assembly pack: harmonize Thumb-ification of iOS build.

Three modules were left behind in a285992763f3961f69a8d86bf7dfff020a08cef9.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2617)

(cherry picked from commit c93f06c12f10c07cea935abd78a07a037e27f155)

Rework error handling of custom_ext_meth_add towards strong exception safety.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2636)
(cherry picked from commit ed874fac6399d5064d6eb8fe2022b918aeaf75af)

Fix a few typos

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2571)
(cherry picked from commit 7e12cdb52e3f4beff050caeecf3634870bb9a7c4)

Remove obsolete comment

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1613)
(cherry picked from commit 7c120357e5ef434c8a7d1d1c3ba4f2a33266374e)

Use TLSEXT_KEYNAME_LENGTH in tls_decrypt_ticket.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2618)
(cherry picked from commit 57b0d651f052ed86528da916397acbcce035fb21)

Prevent allocations of size 0 in sh_init.

which are not possible with the default OPENSSL_zalloc, but are possible if
the user has installed their own allocator using CRYPTO_set_mem_functions. If
the 0-allocations succeeds, the secure heap code will later access
(at least) the first byte of that space, which is technically an OOB
access. This could lead to problems with some custom allocators that only
return a valid pointer for subsequent free()-ing, and do not expect that
the pointer is actually dereferenced.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2605)
(cherry picked from commit 7f07149d25f8d7e00e9350ff2f064a4d25c1a13d)

Make -xcert work again.

When a certificate is prepended update the list pointer.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2628)
(cherry picked from commit 52f4840cb237cc37cad5eac8328828cf3d3e1049)

Add no-ec build

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2626)
(cherry picked from commit b4568b04c7cd425103ac8f1603682e8da2044238)

mem leak on error path and error propagation fix

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2559)
(cherry picked from commit e0670973d5c0b837eb5a9f1670e47107f466fbc7)

Further improvements to ASYNC_WAIT_CTX_clear_fd

Remove call to cleanup function
Use only one loop to find previous element

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2581)
(cherry picked from commit 219aa86cb04e1bfc9c156fab18da2f767502afb2)

Remove fd from the list when the engine clears the wait context before pause

This fixes the num of fds added/removed returned by ASYNC_WAIT_CTX_get_changed_fds

Previously, the numbers were not consistent with the fds actually written in
the buffers since the fds that have been both added and removed are explicitly
ignored in the loop.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2581)
(cherry picked from commit f89dd6738a0ec2b6cfb05a3cc5fa38843dc27d2f)

Add test to show wrong behavior of ASYNC_WAIT_CTX

This happens when a fd is added and then immediately removed from the
ASYNC_WAIT_CTX before pausing the job.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2581)
(cherry picked from commit f44e63644d29e5908be52b7896d5031a5cf460eb)

DES keys are not 7 days long.

CLA: trivial
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2604)

(cherry picked from commit 4fd7b54dc224930a0ce6dd67b35c598c5072857c)

test_rehash does nothing, have it do something

test/recipes/40-test_rehash.t uses test files from certs/demo, which
doesn't exist any longer.  Have it use PEM files from test/ instead.

Because rehash wants only one certificate or CRL per file, we must
also filter those PEM files to produce test files with a single object
each.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2594)
(cherry picked from commit 4bbd8a5daaa810c487f684971c0339a1d7c15da9)

Restore EVP_CIPH_FLAG_LENGTH_BITS working properly

EVP_CIPH_FLAG_LENGTH_BITS flag for CFB1 has been broken with the
introduction of the is_partially_overlapping() check that did not take
it into the account (treating number of bits passed as bytes). This
remedies that and allows this flag to work as intended.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1942)
(cherry picked from commit 64846096b18340b9a39ddd29a7a0e23c56f22959)

Don't read uninitialised data for short session IDs.

While it's always safe to read |SSL_MAX_SSL_SESSION_ID_LENGTH| bytes
from an |SSL_SESSION|'s |session_id| array, the hash function would do
so with without considering if all those bytes had been written to.

This change checks |session_id_length| before possibly reading
uninitialised memory. Since the result of the hash function was already
attacker controlled, and since a lookup of a short session ID will
always fail, it doesn't appear that this is anything more than a clean
up.

In particular, |ssl_get_prev_session| uses a stack-allocated placeholder
|SSL_SESSION| as a lookup key, so the |session_id| array may be
uninitialised.

This was originally found with libFuzzer and MSan in
https://boringssl.googlesource.com/boringssl/+/e976e4349d693b4bbb97e1694f45be5a1b22c8c7,
then by Robert Swiecki with honggfuzz and MSan here. Thanks to both.

Reviewed-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2583)
(cherry picked from commit bd5d27c1c6d3f83464ddf5124f18a2cac2cbb37f)

Fix a typo in the X509_get0_subject_key_id() documentation

Fixes a copy&paste error

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2568)
(cherry picked from commit fbc9eeaaa32ba1416d6cb2794201f440bbaeb629)

Centralize documentation about config file location

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2569)
(cherry picked from commit e9681f8314c64c6802b11997c471bd763de38c8c)

Remove unused variable

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2566)
(cherry picked from commit a19a6c8179faa3da0dedaaf2effae385cf7dd65d)

Fix a crash in EVP_CIPHER_CTX_cleanup due to cipher_data may be NULL
or EVP_CTRL_INIT/EVP_CTRL_COPY was not called or failed.
If that happens in EVP_CipherInit_ex/EVP_CIPHER_CTX_copy set cipher = NULL,
aes_gcm_cleanup should check that gctx != NULL before calling OPENSSL_cleanse.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2562)
(cherry picked from commit 273a0218e65f1737cdbb0ef65a5ddebd601e6bef)

Fix parsing of serial# in req

Reported by Jakub Wilk.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2556)
(cherry picked from commit be4c82aa767998ce2a5717fc895482052373f1b1)

Doc fix

Reported by Alexander Köppe

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2557)
(cherry picked from commit bb6c5e7f6956c5cd1049136d79e631ca8338fc7b)

Fix a crash with malformed user notice policy numbers

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2352)
(cherry picked from commit fe4075f25962dbd302d856c11ac47adb84edc9ca)

Combined patch against OpenSSL_1_1_0-stable branch for the following issues:

Fixed a memory leak in ASN1_digest and ASN1_item_digest.
Reworked error handling in asn1_item_embed_new.
Fixed error handling in int_ctx_new and EVP_PKEY_CTX_dup.
Fixed a memory leak in CRYPTO_free_ex_data.
Reworked error handing in x509_name_ex_d2i, x509_name_encode and x509_name_canon.
Check for null pointer in tls_process_cert_verify.

Fixes #2103 #2104 #2105 #2109 #2111 #2115

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2163)

Majority rules, use session_ctx vs initial_ctx

session_ctx and initial_ctx are aliases of each other, and with the
opaque data structures, there's no need to keep both around. Since
there were more references of session_ctx, replace all instances of
initial_ctx with session_ctx.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2340)

Grouped data declarations [skip ci]
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit bc1dba209533f2033a4de0d93380fc0f485e6f7e)

Removed tab spaces.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit 5d5eed4456ebc035893eedbcc4e32a9d065cecb3)

Corrections according the review comments.

Updated indentations according project rules, renamed file-local define to the shorter version - USE_RWLOCK, fixed declaration after the if statement in CRYPTO_THREAD_lock_new().
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit ec93a2921f6128ac931466ae171fba92a0eab45d)

Compile fix on platforms with missing pthread_rwlock_t.

Fix compilation on platforms with missing pthread_rwlock_t implementation by replacing it with pthread_mutex_t. An example of such platform can be Android OS 2.0 - 2.1, API level 5 (Eclair), Android NDK platform - android-5 where pthread_rwlock_t is not implemented and is missing in pthread.h.

In case of missing pthread_rwlock_t implementation CRYPTO_RWLOCK will work as exclusive lock in write-only mode of pthread_rwlock_t lock.

The implementation based on pthread_mutex_t must be using PTHREAD_MUTEX_RECURSIVE mode to be compatible with recursive behavior of pthread_rwlock_rdlock.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit 2accf3f7e013c3d02312afc27cc2edbd1f149db3)

remove test/.rnd on make clean

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2344)
(cherry picked from commit 122fa088524571a3b60ebf301873f69afdac8f7a)

bn: fix occurance of negative zero in BN_rshift1()

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit 0a2dcb6990dacc94337f746f4f4a6dfac1fbeac4)

bn: fix occurances of negative zero

The BIGNUM behaviour is supposed to be "consistent" when going into and
out of APIs, where "consistent" means 'top' is set minimally and that
'neg' (negative) is not set if the BIGNUM is zero (which is iff 'top' is
zero, due to the previous point).

The BN_DEBUG testing (make test) caught the cases that this patch
corrects.

Note, bn_correct_top() could have been used instead, but that is intended
for where 'top' is expected to (sometimes) require adjustment after direct
word-array manipulation, and so is heavier-weight. Here, we are just
catching the negative-zero case, so we test and correct for that
explicitly, in-place.

Change-Id: Iddefbd3c28a13d935648932beebcc765d5b85ae7
Signed-off-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1672)
(cherry picked from commit 38d1b3cc0271008b8bd130a2c4b442775b028a08)

bn: catch negative zero as an error

Change-Id: I5ab72ad0aae9069b47d5b7b7b9e25bd1b7afa251
Signed-off-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1672)
(cherry picked from commit 2fc9b36a96ccd77cbd9ecfb3a3cdaa7ad2ca305e)

bn: fix BN_DEBUG + BN_DEBUG_RAND support

Couple of updates to make this code work properly again;
* use OPENSSL_assert() instead of assert() (and #include <assert.h>)
* the circular-dependency-avoidance uses RAND_bytes() (not pseudo)

Change-Id: Iefb5a9dd73f71fd81c1268495c54a64378955354
Signed-off-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1672)
(cherry picked from commit 0b50ac1a0fe907f4effcf3f2f36dac32523938c5)

Fix faulty free

On error, i2o_SCT_signature() and i2o_SCT() free a pointer that may
have wandered off from the start of the allocated block (not currently
true for i2o_SCT_signature(), but has that potential as the code may
change.  To avoid this, save away the start of the allocated block and
free that instead.

Thanks to Guido Vranken for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2323)
(cherry picked from commit d85d3c993e322d3e4c3f00be2910faa8c55b40e3)

test/evp_test.c: If no algorithm was specified, don't try to check for DES

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2319)
(cherry picked from commit 31b69e9a26c5b127ce273bc5834b9e13e5e25556)

Add a couple of test to check CRL fingerprint

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2314)
(cherry picked from commit 929860d0e6112f5c7766d9ea036c3f8bd8d3d719)

Document what EXFLAG_SET is for in x509v3.h

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2314)
(cherry picked from commit 2d60c923141e7853c268364f26195343a5e995bf)

X509_CRL_digest() - ensure precomputed sha1 hash before returning it

X509_CRL_digest() didn't check if the precomputed sha1 hash was actually
present.  This also makes sure there's an appropriate flag to check.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2314)
(cherry picked from commit 6195848b2eea627c47f74b63eb2ba3dc3d5b6436)

Correct pointer to be freed

The pointer that was freed in the SSLv2 section of ssl_bytes_to_cipher_list
may have stepped up from its allocated position.  Use a pointer that is
guaranteed to point at the start of the allocated block instead.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2312)
(cherry picked from commit 63414e64e66e376654e993ac966e3b2f9d849d3b)

Travis: run on Trusty with clang 3.9 (1.1.0 branch)

See https://github.com/travis-ci/travis-ci/issues/6460 for context on the changes to wine install.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add server signature algorithm bug test.

Add a client authentication signature algorithm to simple
ssl test and a server signature algorithm. Since we don't
do client auth this should have no effect. However if we
use client auth signature algorithms by mistake this will
abort the handshake with a no shared signature algorithms
error.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2297)

Use correct signature algorithm list when sending or checking.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2297)

Prepare for 1.1.0e-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.0d release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update CHANGES and NEWS for new release

Reviewed-by: Richard Levitte <levitte@openssl.org>

bn/asm/x86_64-mont5.pl: fix carry bug in bn_sqr8x_internal.

CVE-2017-3732

Reviewed-by: Rich Salz <rsalz@openssl.org>

Document DH_check_params()

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>