Unix Makefile: Have manual generation use the same perl script as Windows and VMS

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2864)

util/process_docs.pl: make it possible to add a suffix to man docs

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2864)

Remove some duplicate manpage entries

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2860)
(cherry picked from commit 6556519ea0581323f2330684ad2ae81f0448ef52)

Add documentation on platform specific checks

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2851)
(cherry picked from commit 44eb65ce20d673d0332802275d54f6811f448076)

Add a platform specific configuration checker

For each platform, we may need to perform some basic checks to see
that available tools perform as we expect them.

For the moment, the added checkers test that Perl gives the expected
path format.  This should help MingW users to see if they run an
appropriate Perl implementation, for example.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2851)
(cherry picked from commit d192a3aaeb76fc89f8285b4dc938c2bc0c37d0d4)

Fix an endless loop in rsa_builtin_keygen.

And add a test case.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2757)

(cherry picked from commit 697958313ba48c8ebc832ab8f9f2b845fb7acfd4)

Reset executable bits on files where not needed.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2838)

Don't use deprecated EVP_CIPHER_CTX_cleanup() internally

Use EVP_CIPHER_CTX_reset() instead

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2812)
(cherry picked from commit 15d95dd7ea77e68bf9d8450e52230a6017735ec0)

Update the cipher(1) documentation to

explicitly state that the RSA cipher
string means the same a kRSA.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2821)
(cherry picked from commit f2bcff43bcd5b1e2632273ef8fea0900a15d7769)

Fix cherry-pick and put files in right place

Also SLS_set_bio.pod got copied, remove the clone.
[skip ci]

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2828)

Remove ref to err(7), update copyright.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2825)
(cherry picked from commit 73fb82b72c7544cf52d95ac29d4a45b253395715)

Configurations/10-main.conf: omit redundant -lresolv from Solaris configs.

GH#2816

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit 8cfc21f53af5187497a8567bb2801c36472d7016)

bio/b_addr.c: omit private hstrerror.

Private hstrerror was introduced to address linking problem on HP-UX,
but truth be told conemporary systems, HP-UX included, wouldn't come
to that call, they would use getaddrinfo and gai_strerror, while
gethostbyname and h_errno are there to serve legacy systems. Since
legacy systems are naturally disappearing breed, we can as well just
let user interpret number.

GH#2816

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit 3e49ee23eab5c3fa57d14dc5f82f50cbea718322)

Check for zero records and return immediately

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2822)
(cherry picked from commit a3004c820370b6bee82c919721fb1cbe95f72f3f)

Add NOTES.UNIX, with a description on how to deal with runpaths

[skip ci]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2818)
(cherry picked from commit 45632ee3bb7ab4ed405d5251d76dd5b94d782adb)

Check that async_jobs is not negative and not too high.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2693)
(cherry picked from commit f8aa15728ba960137faf77b298fa60c1e63dc50f)

VMS: compensate for gmtime_r() parameter pointer size

With VMS C, the second parameter takes a 32-bit pointer.  When
building with 64-bit pointer size default, we must compensate.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2811)
(cherry picked from commit 48ce800aa5a2ccee204ad3960a20c4ca14acb3a1)

Test framework: Add the possibility to have a test specific data dir

This data directory is formed automatically by taking the recipe name
and changing '.t' to '_data'.  Files in there can be reached with the
new function data_file()

(Merged from https://github.com/openssl/openssl/pull/2027)
(cherry picked from commit 6c6a2ae6fc964795304bbe7687e42b2b0cdf81b3)
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2802)

Don't free in cleanup routine

Cleanse instead, and free in the free routine.

Seems to have been introduced in commit
846ec07d904f9cc81d486db0db14fb84f61ff6e5 when EVP_CIPHER_CTX was made
opaque.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2798)
(cherry picked from commit 5c6c4c5c333c8ac469e53521cf747ff527b8813a)

Add AGL's "beer mug" PEM file as another test input

AGL has a history of pointing out the idiosynchronies/laxness of the
openssl PEM parser in amusing ways.  If we want this functionality to
stay present, we should test that it works.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2756)
(cherry picked from commit a00b9560f7ece1e51bd7a8dc6a7ffb7a3d20cf86)

Add test corpus for PEM reading

Generate a fresh certificate and DSA private key in their respective PEM
files.  Modify the resulting ASCII in various ways so as to produce input
files that might be generated by non-openssl programs (openssl always
generates "standard" PEM files, with base64 data in 64-character lines
except for a possible shorter last line).

Exercise various combinations of line lengths, leading/trailing
whitespace, non-base64 characters, comments, and padding, for both
unencrypted and encrypted files.  (We do not have any other test coverage
that uses encrypted files, as far as I can see, and the parser enforces
different rules for the body of encrypted files.)

Add a recipe to parse these test files and verify that they contain the
expected string or are rejected, according to the expected status.
Some of the current behavior is perhaps suboptimal and could be revisited.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2756)
(cherry picked from commit e8cee55718bb9cb957f449fbe7145a77f252bb73)

Code health: make update

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2793)
(cherry picked from commit 77baccd7fac7cf480e3a3981b7deae5ef3b812b9)

Code health: Remove VAX exceptions in util/mkdef.pl

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2793)
(cherry picked from commit 96bc5d03b813a318403d45600e07d6bdcb41d195)

Code health: Remove unused VAX transfer vector for engines

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2793)
(cherry picked from commit 38a322a5f29ae0b4a9bd42233310835487d875ac)

Exdata test was never enabled.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2787)
(cherry picked from commit 629192c1b9f17965e0a6b73229b7b1e004bfbd98)

Fix test_ssl_new when compiled with no-tls1_2 or no-dtls1_2

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2788)
(cherry picked from commit 4d118fe007692de2dd8c5dd084254f8d3b308167)

Update year, wording tweak

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2755)
(cherry picked from commit 6faa3456326afa56ea8c25a0b49239392074e192)

Code cleanup: remove the VMS specific reimplementation of gmtime

This reimplementation was necessary before VMS C V7.1.  Since that's
the minimum version we support in this OpenSSL version, the
reimplementation is no longer needed.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2762)
(cherry picked from commit 9d70ac97d9d8720e6ed280609c844da403b80440)

Adjust the default value of the private key size

...in the man page to reflect the actual default (2048 instead of 512)

CLA: trivial
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2754)

(cherry picked from commit 013bc448672cbc3c9cd154709400c676c2955229)

.travis.yml: limit mingw tests' resource consumption.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2735)

.travis.yml: make package pulls conditional.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2735)

Increase the size of the stack buffer to prevent an overflow.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2721)
(cherry picked from commit 8fce04ee3540ba3039bb66df34ea3f076a599ab9)

.travis.yml: remove osx from build matrix.

Travis OS X utilization and backlog statistics suggest that it became
bottleneck for our integration builds with requests piling up for days
during working days of the week. Suggestion is to remove osx till
capacity is lesser issue.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit e12e903e9ac675d08f9dd0db1f0c1a2049232c21)

Fix potential memory leak in ASN1_TIME_to_generalizedtime()

If ret is allocated, it may be leaked on error.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2666)
(cherry picked from commit 4483e23444fa18034344874ffbe67919207e9e47)

Add -Wundef to strict-warnings

Avoid a -Wundef warning in o_str.c
Avoid a -Wundef warning in testutil.h
Include internal/cryptlib.h before openssl/stack.h
to avoid use of undefined symbol OPENSSL_API_COMPAT.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2709)

Check for the presence of _WIN32 rather than its value.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2727)
(cherry picked from commit 46958a043d51633ed36bcfb13ff048a3381366a6)

In apps/rehash.c, decorate the inclusion of internal/o_dir.h for VMS

The library files are built with symbol names as is, while the
application is built with the default uppercase-all-symbols mode.
That's fine for public APIs, because we have __DECC_INCLUDE_PROLOGUE.H
and __DECC_INCLUDE_EPILOGUE.H automatically telling the compiler how
to treat the public header files.  However, we don't have the same
setup for internal library APIs, since they are usually only used by
the libraries.

Because apps/rehash.c uses a library internal header file, we have to
surround that inclusion with the same kind of pragmas found in
__DECC_INCLUDE_PROLOGUE.H and __DECC_INCLUDE_EPILOGUE.H, or we get
unresolved symbols when building no-shared.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2725)
(cherry picked from commit 2ac915f16218982f48dbc799b8308a07441d2e35)

On VMS, massage the fetch file names to remove the generation number

The generation number is ';nnn' at the end of the file name fetched
with readdir().  Because rehash checks for specific extensions and
doesn't expect an additional generation number, the easiest is to
massage the received file name early by simply removing the generation
number.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2717)
(cherry picked from commit 39aceac320a1561d50c7d71ac2560aec7ab8eddb)

Let the output from 'openssl enc -ciphers' go to stdout

Also, don't exit with an error code

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2716)
(cherry picked from commit 341de5f1997d21b60cee69be656f1ae709bccdac)

Fix typo, should be && rather than &

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2689)
(cherry picked from commit 50799f3558981eac0482d3ea77b21c58b56d4871)

Fix typo, missing ||

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2707)
(cherry picked from commit 6eb837583758506607f538fe2a3dd87925e4e69d)

Iterate over EC_GROUP's poly array in a safe way

Prevent that memory beyond the last element is accessed if every element
of group->poly[] is non-zero

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2689)

(cherry picked from commit 57f48f939ed5d3119e3c691ea0a8a3ac2f4a1a9e)

Make "openssl rehash" work on VMS 8.3 and up

A spelling error prevented it from building correctly.
Furthermore, we need to be more careful when to add a / at the end
of the dirname and when not.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2706)
(cherry picked from commit 5c80e2af3a7d8aa5129a1668c286c1464983e1ac)

Have the directory reader use the Unix API on VMS

opendir(), readdir() and closedir() have been available on VMS since
version 7.0.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2707)
(cherry picked from commit d8eaaf15356e1559f0f669b430b0d22b3514f8f0)

Add some more consistency checks in tls_decrypt_ticket.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2704)
(cherry picked from commit 79020b27beff060d02830870fdfd821fe8cbd439)

Fix i2d_SSL_SESSION pp output parameter should point to end of asn1 data.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2607)
(cherry picked from commit a0179d0afb621a0875ddcfd939719a9628ac4444)

Fix memory leak in pkcs12 -export

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2676)
(cherry picked from commit 1b8f19379a521ec11ce37e12316dd3edc0acfb82)

Fix some more memory leaks with TXT_DB_insert.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2684)
(cherry picked from commit 0fbaef9e64fa10446aff805791befaa2b967e322)

Fix a few memleaks in TXT_DB.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2684)
(cherry picked from commit 9ad52c562a93c9a57ae3024e54c575430753244c)

Prevent OOB in SRP base64 code.

Change size comparison from > (GT) to >= (GTE) to ensure an additional
byte of output buffer, to prevent OOB reads/writes later in the function
Reject input strings larger than 2GB
Detect invalid output buffer size and return early

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2672)

(cherry picked from commit ecca16632a73bb80ee27cdec8a97f6def0a4714d)

Removed ugly size_t less than zero check.

CLA: trivial.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2674)

(cherry picked from commit 5e1f879ab5a2bfdf2d58222f965f93fe1b511ce7)

Ensure minsize >= sizeof(SH_LIST)

The sh_add_to_list function will overwrite subsequent slots in the free list
for small allocations.  This causes a segmentation fault if the writes goes
off the end of the secure memory.  I've not investigated if this problem
can overwrite memory without the segmentation fault, but it seems likely.

This fix limits the minsize to the sizeof of the SH_LIST structure (which
also has a side effect of properly aligning the pointers).

The alternative would be to return an error if minsize is too small.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2657)

(cherry picked from commit 70e14ffbaf6a67dab56c24cae01f1248cf3f1e77)

Don't call memcpy if len is zero.

Prevent undefined behavior in CRYPTO_cbc128_encrypt: calling this function
with the 'len' parameter being 0 would result in a memcpy where the source
and destination parameters are the same, which is undefined behavior.
Do same for AES_ige_encrypt.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2671)

(cherry picked from commit b1498c98f3fb5b8a340acc9ce20b0fd5346294e5)

VMS fix of test/recipes/80-test_ssl_new.t

On VMS, file names with more than one period get all but the last get
escaped with a ^, so 21-key-update.conf.in becomes 21-key-update^.conf.in
That means that %conf_dependent_tests and %skip become useless unless
we massage the file names that are used as indexes.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2678)
(cherry picked from commit d89f66412ba5168e7d6fd9dd88619d927d716f55)

If all versions of a proto are disabled, disabled the proto as well

For example, 'no-dtls1 no-dtls1_2' will imply 'no-dtls'

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2670)
(cherry picked from commit 343a7467c270c54a8e1c85e88e807a1c2e0b6127)

Fix a slightly confusing if condition in a2i_ASN1_INTEGER.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2640)
(cherry picked from commit aa402e2ba408254c052b5750b14e7f01e48bced1)

Fix test_x509_store

Don't run this test unless 'openssl rehash' works properly.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2664)
(cherry picked from commit 73540f4729bb856ab066c6e7a57513a97e3ca36f)

Add a test of the X509_STORE / X509_LOOKUP API

Fortunately, "openssl verify" makes good use of that API

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2652)
(cherry picked from commit bb0f7eca75b8da1538c08c1f5be1bb7ea8f40638)

test/README: clarify test number groups

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2652)
(cherry picked from commit 532e7b36d9622ac06a96fb3557b5bc16016e5ca8)

Fix a mem leak in ssl_test_ctx.c

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2650)
(cherry picked from commit d605fc3a0ce4103ca6660904795bf1209cdb55b7)

Don't run MSBLOB conversion tests when RSA or DSA are disabled

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2174)
(cherry picked from commit d8594555ffaf98ada08b26ce3d1138f16bc029c5)

Change req_check_len error message

it also accepts 20 bytes, but states 'less than' in the error message

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2648)
(cherry picked from commit 0cb8c9d85e9d5690670d6f1f02e8ccc756520210)

Use _WIN32 over WIN32 for preprocessor conditional

The intent seems to be that the WIN32 symbol is for things that are a direct
byproduct of being a windows-variant configuration and should be used for
feature en/disablement on windows systems.  Use of the _WIN32 symbol is more
widespread, being used to implement platform portability of more generic code.

We do define WIN32 in some situations in e_os.h, but that is not included
universally.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2642)
(cherry picked from commit ac879ed62a19f3c878f7be3020a1b93cc77f4b38)

Prepare for 1.1.0f-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.0e release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update CHANGES and NEWS for new release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove an OPENSSL_assert() and replace with a soft assert and check

Following on from CVE-2017-3733, this removes the OPENSSL_assert() check
that failed and replaces it with a soft assert, and an explicit check of
value with an error return if it fails.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't change the state of the ETM flags until CCS processing

Changing the ciphersuite during a renegotiation can result in a crash
leading to a DoS attack. ETM has not been implemented in 1.1.0 for DTLS
so this is TLS only.

The problem is caused by changing the flag indicating whether to use ETM
or not immediately on negotiation of ETM, rather than at CCS. Therefore,
during a renegotiation, if the ETM state is changing (usually due to a
change of ciphersuite), then an error/crash will occur.

Due to the fact that there are separate CCS messages for read and write
we actually now need two flags to determine whether to use ETM or not.

CVE-2017-3733

Reviewed-by: Richard Levitte <levitte@openssl.org>

Provide a test for the Encrypt-Then-Mac renegotiation crash

Changing the ciphersuite during a renegotiation can result in a crash
leading to a DoS attack. ETM has not been implemented in 1.1.0 for DTLS
so this is TLS only.

This commit provides a test for the issue.

CVE-2017-3733

Reviewed-by: Richard Levitte <levitte@openssl.org>

Properly zero cipher_data for ChaCha20-Poly1305 on cleanup

Fix a typo. Probably this has not been found because EVP_CIPHER_CTX is
smaller than EVP_CHACHA_AEAD_CTX and heap overflow does not occur.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2294)
(cherry picked from commit a8f957686675194d786b41f6e1f7c48bb85723ec)

crypto/armcap.c: short-circuit processor capability probe in iOS builds.

Capability probing by catching SIGILL appears to be problematic
on iOS. But since Apple universe is "monocultural", it's actually
possible to simply set pre-defined processor capability mask.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2617)

(cherry picked from commit 8653e78f4319b23d60239f9557d8c1e1d23be1a5)

ARMv4 assembly pack: harmonize Thumb-ification of iOS build.

Three modules were left behind in a285992763f3961f69a8d86bf7dfff020a08cef9.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2617)

(cherry picked from commit c93f06c12f10c07cea935abd78a07a037e27f155)

Rework error handling of custom_ext_meth_add towards strong exception safety.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2636)
(cherry picked from commit ed874fac6399d5064d6eb8fe2022b918aeaf75af)

Fix a few typos

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2571)
(cherry picked from commit 7e12cdb52e3f4beff050caeecf3634870bb9a7c4)

Remove obsolete comment

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1613)
(cherry picked from commit 7c120357e5ef434c8a7d1d1c3ba4f2a33266374e)

Use TLSEXT_KEYNAME_LENGTH in tls_decrypt_ticket.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2618)
(cherry picked from commit 57b0d651f052ed86528da916397acbcce035fb21)

Prevent allocations of size 0 in sh_init.

which are not possible with the default OPENSSL_zalloc, but are possible if
the user has installed their own allocator using CRYPTO_set_mem_functions. If
the 0-allocations succeeds, the secure heap code will later access
(at least) the first byte of that space, which is technically an OOB
access. This could lead to problems with some custom allocators that only
return a valid pointer for subsequent free()-ing, and do not expect that
the pointer is actually dereferenced.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2605)
(cherry picked from commit 7f07149d25f8d7e00e9350ff2f064a4d25c1a13d)

Make -xcert work again.

When a certificate is prepended update the list pointer.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2628)
(cherry picked from commit 52f4840cb237cc37cad5eac8328828cf3d3e1049)

Add no-ec build

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2626)
(cherry picked from commit b4568b04c7cd425103ac8f1603682e8da2044238)

mem leak on error path and error propagation fix

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2559)
(cherry picked from commit e0670973d5c0b837eb5a9f1670e47107f466fbc7)

Further improvements to ASYNC_WAIT_CTX_clear_fd

Remove call to cleanup function
Use only one loop to find previous element

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2581)
(cherry picked from commit 219aa86cb04e1bfc9c156fab18da2f767502afb2)

Remove fd from the list when the engine clears the wait context before pause

This fixes the num of fds added/removed returned by ASYNC_WAIT_CTX_get_changed_fds

Previously, the numbers were not consistent with the fds actually written in
the buffers since the fds that have been both added and removed are explicitly
ignored in the loop.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2581)
(cherry picked from commit f89dd6738a0ec2b6cfb05a3cc5fa38843dc27d2f)

Add test to show wrong behavior of ASYNC_WAIT_CTX

This happens when a fd is added and then immediately removed from the
ASYNC_WAIT_CTX before pausing the job.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2581)
(cherry picked from commit f44e63644d29e5908be52b7896d5031a5cf460eb)

DES keys are not 7 days long.

CLA: trivial
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2604)

(cherry picked from commit 4fd7b54dc224930a0ce6dd67b35c598c5072857c)

test_rehash does nothing, have it do something

test/recipes/40-test_rehash.t uses test files from certs/demo, which
doesn't exist any longer.  Have it use PEM files from test/ instead.

Because rehash wants only one certificate or CRL per file, we must
also filter those PEM files to produce test files with a single object
each.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2594)
(cherry picked from commit 4bbd8a5daaa810c487f684971c0339a1d7c15da9)

Restore EVP_CIPH_FLAG_LENGTH_BITS working properly

EVP_CIPH_FLAG_LENGTH_BITS flag for CFB1 has been broken with the
introduction of the is_partially_overlapping() check that did not take
it into the account (treating number of bits passed as bytes). This
remedies that and allows this flag to work as intended.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1942)
(cherry picked from commit 64846096b18340b9a39ddd29a7a0e23c56f22959)

Don't read uninitialised data for short session IDs.

While it's always safe to read |SSL_MAX_SSL_SESSION_ID_LENGTH| bytes
from an |SSL_SESSION|'s |session_id| array, the hash function would do
so with without considering if all those bytes had been written to.

This change checks |session_id_length| before possibly reading
uninitialised memory. Since the result of the hash function was already
attacker controlled, and since a lookup of a short session ID will
always fail, it doesn't appear that this is anything more than a clean
up.

In particular, |ssl_get_prev_session| uses a stack-allocated placeholder
|SSL_SESSION| as a lookup key, so the |session_id| array may be
uninitialised.

This was originally found with libFuzzer and MSan in
https://boringssl.googlesource.com/boringssl/+/e976e4349d693b4bbb97e1694f45be5a1b22c8c7,
then by Robert Swiecki with honggfuzz and MSan here. Thanks to both.

Reviewed-by: Geoff Thorpe <geoff@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2583)
(cherry picked from commit bd5d27c1c6d3f83464ddf5124f18a2cac2cbb37f)

Fix a typo in the X509_get0_subject_key_id() documentation

Fixes a copy&paste error

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2568)
(cherry picked from commit fbc9eeaaa32ba1416d6cb2794201f440bbaeb629)

Centralize documentation about config file location

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2569)
(cherry picked from commit e9681f8314c64c6802b11997c471bd763de38c8c)

Remove unused variable

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2566)
(cherry picked from commit a19a6c8179faa3da0dedaaf2effae385cf7dd65d)

Fix a crash in EVP_CIPHER_CTX_cleanup due to cipher_data may be NULL
or EVP_CTRL_INIT/EVP_CTRL_COPY was not called or failed.
If that happens in EVP_CipherInit_ex/EVP_CIPHER_CTX_copy set cipher = NULL,
aes_gcm_cleanup should check that gctx != NULL before calling OPENSSL_cleanse.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2562)
(cherry picked from commit 273a0218e65f1737cdbb0ef65a5ddebd601e6bef)

Fix parsing of serial# in req

Reported by Jakub Wilk.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2556)
(cherry picked from commit be4c82aa767998ce2a5717fc895482052373f1b1)

Doc fix

Reported by Alexander Köppe

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2557)
(cherry picked from commit bb6c5e7f6956c5cd1049136d79e631ca8338fc7b)

Fix a crash with malformed user notice policy numbers

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2352)
(cherry picked from commit fe4075f25962dbd302d856c11ac47adb84edc9ca)

Combined patch against OpenSSL_1_1_0-stable branch for the following issues:

Fixed a memory leak in ASN1_digest and ASN1_item_digest.
Reworked error handling in asn1_item_embed_new.
Fixed error handling in int_ctx_new and EVP_PKEY_CTX_dup.
Fixed a memory leak in CRYPTO_free_ex_data.
Reworked error handing in x509_name_ex_d2i, x509_name_encode and x509_name_canon.
Check for null pointer in tls_process_cert_verify.

Fixes #2103 #2104 #2105 #2109 #2111 #2115

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2163)

Majority rules, use session_ctx vs initial_ctx

session_ctx and initial_ctx are aliases of each other, and with the
opaque data structures, there's no need to keep both around. Since
there were more references of session_ctx, replace all instances of
initial_ctx with session_ctx.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2340)

Grouped data declarations [skip ci]
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit bc1dba209533f2033a4de0d93380fc0f485e6f7e)

Removed tab spaces.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit 5d5eed4456ebc035893eedbcc4e32a9d065cecb3)

Corrections according the review comments.

Updated indentations according project rules, renamed file-local define to the shorter version - USE_RWLOCK, fixed declaration after the if statement in CRYPTO_THREAD_lock_new().
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit ec93a2921f6128ac931466ae171fba92a0eab45d)

Compile fix on platforms with missing pthread_rwlock_t.

Fix compilation on platforms with missing pthread_rwlock_t implementation by replacing it with pthread_mutex_t. An example of such platform can be Android OS 2.0 - 2.1, API level 5 (Eclair), Android NDK platform - android-5 where pthread_rwlock_t is not implemented and is missing in pthread.h.

In case of missing pthread_rwlock_t implementation CRYPTO_RWLOCK will work as exclusive lock in write-only mode of pthread_rwlock_t lock.

The implementation based on pthread_mutex_t must be using PTHREAD_MUTEX_RECURSIVE mode to be compatible with recursive behavior of pthread_rwlock_rdlock.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1981)
(cherry picked from commit 2accf3f7e013c3d02312afc27cc2edbd1f149db3)