In UI_OpenSSL's open(), generate an error on unknown errno

TTY_get() sometimes surprises us with new errno values to determine if
we have a controling terminal or not.  This generated error is a
helpful tool to figure out that this was what happened and what the
unknown value is.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2043)
(cherry picked from commit 4984448648f69ed4425df68900b1fd6f17c6c271)

Make sure that password_callback exercises UI

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2040)
(cherry picked from commit 57c0f378b8fdbdc55dba783e9b744b8ed2132819)

Add a test for the UI API

The best way to test the UI interface is currently by using an openssl
command that uses password_callback.  The only one that does this is
'genrsa'.
Since password_callback uses a UI method derived from UI_OpenSSL(), it
ensures that one gets tested well enough as well.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2040)
(cherry picked from commit 17ac8eaf611b588cca251ba63b187e7d9c7edb83)

UI_process() didn't generate errors

Since there are many parts of UI_process() that can go wrong, it isn't
very helpful to only return -1 with no further explanation.  With this
change, the error message will at least show which part went wrong.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2037)
(cherry picked from commit 0a687ab0a92d2d68289364a6e232028c229f44bb)

Restore last-resort expired untrusted intermediate issuers

Reviewed-by: Matt Caswell <matt@openssl.org>

Ensure we are in accept state in DTLSv1_listen

Calling SSL_set_accept_state() after DTLSv1_listen() clears the state, so
SSL_accept() no longer works. In 1.0.2 calling DTLSv1_listen() would set
the accept state automatically. We should still do that.

Fixes #1989

Reviewed-by: Andy Polyakov <appro@openssl.org>
(cherry picked from commit 5bdcd362d24cbbcf18c5eb9df655fe9f7bcf5850)

Fix ctrl operation for SHA1/MD5SHA1.

This makes S/MIME and CMS signing in MIME format for SHA1 work again.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit a5abd438f85737ffa56320b67c5ef5525fc495c3)

add CMS SHA1 signing test

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit c6d67f09f34d8203c5bad7171ed45ec8771c9764)

INSTALL: clarify 386 and no-sse2 options.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit 5ae5dc96610f0a598dac9d2f267b5c0ddd77b2e4)

modes/ctr128.c: fix false carry in counter increment procedure.

GH issue #1916 affects only big-endian platforms. TLS is not affected,
because TLS fragment is never big enough.

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit 76f572ed0469a277d92378848250b7a9705d3071)

test/evptests.txt: add regression test for false carry in ctr128.c.

GH issue #1916 affects only big-endian platforms. TLS is not affected,
because TLS fragment is never big enough.

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit b47f116b1e02d20b1f8a7488be5a04f7cf5bc712)

Fix a missing function prototype in AFALG engine

Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit a1fd1fb241069cc987d0d2cf13880bd16cada3c9)

Fix missing NULL checks in CKE processing

Reviewed-by: Rich Salz <rsalz@openssl.org>

Clarify what X509_NAME_online does with the given buffer and size

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1980)
(cherry picked from commit 19cb71ef6e414759d737918bab10be2cc1d8bd99)

Add missing -zdelete for some linux arches

b6d5ba1a9f004d637acac18ae3519fe063b6b5e1 forgot to update some linux arches.

Reviewed-by: Richard Levitte <levitte@openssl.org>
GH: #1977
(cherry picked from commit 55ab86e4c202e202a2b9200291d038878a727815)

Make SSL_read and SSL_write return the old behaviour and document it.

Backport of beacb0f0c1ae7b0542fe053b95307f515b578eb7, revert of
122580ef71e4e5f355a1a104c9bfb36feee43759

Fixes: #1903

Reviewed-by: Matt Caswell <matt@openssl.org>
GH: #1966

Make async_read and async_write return -1 on failure.

Reviewed-by: Matt Caswell <matt@openssl.org>
GH: #1966

Skipping tests in evp_test leaks memory

When configured with "no-mdc2 enable-crypto-mdebug" the evp_test
will leak memory due to skipped tests, and error out.

Also fix a skip condition

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1946)

Use consistent variable names

In the X509_NAME_get_index_by_NID.pod example, the initialized variable is called
"loc", but the one used in the for loop is called "lastpos". Make the names match.

CLA: trivial
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1949)

Support MSBLOB format if RC4 is disabled

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit b6c6898234a12b9c6cdaa8f16fb9156097649ad7)

Fix MSBLOB format with RSA.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit 159f6e7ecfde9e98194d6111c85587b85b6a8fc5)

Make MSBLOB format work with dsa utility.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit b3795987477f1d478fd8bd20efb812e71b190e8b)

Add conversion test for MSBLOB format.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit d922634d0c63cee01c89869d79306cd2df628855)

Raise an error on memory alloc failure.

Both strdup or malloc failure should raise an err.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1905)
(cherry picked from commit bad6b116a2d3c005330e618c726f172fd0fefc2a)

Missing free item on push failure

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1905)
(cherry picked from commit 2d13250fd695eba777fe7e2af4beb1b7d356bd8f)

Move SCT_LIST_free definition into a more logical place

This reflects its position in include/openssl/ct.h.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1548)
(cherry picked from commit e1940e9f7a73bf3a560fbe3550a9b69a612118ec)

Make sure things get deleted when test setup fails in ct_test.c

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1548)
(cherry picked from commit 765731a88899771989a53c72259cacd1c658bb3f)

Use valid signature in test_decode_tls_sct()

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1548)
(cherry picked from commit e2635c49f35c615820b1c6d92d180e31e28adeb2)

Pass a temporary pointer to o2i_SCT_signature from SCT_new_from_base64

Otherwise, |dec| gets moved past the end of the signature by
o2i_SCT_signature and then can't be correctly freed afterwards.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1548)
(cherry picked from commit 73ccf3ca01085d143aecb7fcfb0aac18caa678d2)

Subtract padding from outlen in ct_base64_decode

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1548)
(cherry picked from commit 70a06fc1a8b098e9934f837896159bfc6caf0228)

Construct SCT from base64 in ct_test

This gives better code coverage and is more representative of how a
user would likely construct an SCT (using the base64 returned by a CT log).

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1548)
(cherry picked from commit f7a39a5a3f7f91e0d1ba0030323eef26bc8ccddf)

On x86 machines where the compiler supports -m32, use 'linux-x86'

The rationale is that the linux-x86 is the most likely config target
to evolve and should therefore be chosen when possible, while
linux-elf is mostly reserved for older Linux machines.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1924)
(cherry picked from commit 27a451e3739d8331b9c180b0373b88ab6c382409)

Add a modern linux-x86 config target

'linux-x86' is similar to 'linux-x86_64' but uses -m32 rather than -m64.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1924)
(cherry picked from commit 7fbc0bfdd7a3c46bc7e36b191d11ab3853555a25)

Remove a hack from ssl_test_old

ssl_test_old was reaching inside the SSL structure and changing the internal
BIO values. This is completely unneccessary, and was causing an abort in the
test when enabling TLSv1.3.

I also removed the need for ssl_test_old to include ssl_locl.h. This
required the addition of some missing accessors for SSL_COMP name and id
fields.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit e304d3e20f45243f9e643607edfe4db49c329596)

Check return value of some BN functions.

Factorise multiple bn_get_top(group->field) calls
Add missing checks on some conditional BN_copy return value
Add missing checks on some BN_copy return value
Add missing checks on a few bn_wexpand return value

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1626)

(cherry picked from commit 78e09b53a40729f5e99829ccc733b592bd22fea1)

Cherry-pick doc updates from PR 1554

Also fix version in libcrypto.num, from backporting new
functions.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)
(cherry picked from commit ebcb536858a271e8812fb9bbafbc0b825e5ece24)

Add test for CT_POLICY_EVAL_CTX default time

Checks that the epoch_time_in_ms field of CT_POLICY_EVAL_CTX is initialized
to approximately the current time (as returned by time()) by default. This
prevents the addition of this field, and its verification during SCT
validation, from breaking existing code that calls SCT_validate directly.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)
(cherry picked from commit ebcb536858a271e8812fb9bbafbc0b825e5ece24)

Convert C++ comments to C-style comments

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)
(cherry picked from commit 08e588b7d5cefbfd107c88416900165a28a5b59e)

Cast time_t to uint64_t before converting to milliseconds in ct_policy.c

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)
(cherry picked from commit 5e08606619c0b0e065f1ffa12ce6411f321ed174)

By default, allow SCT timestamps to be up to 5 minutes in the future

As requested in
https://github.com/openssl/openssl/pull/1554#issuecomment-246371575.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)
(cherry picked from commit c22aa33e29ce162c672c9b2f0df591db977d4e9b)

Don't check for time() failing in CT_POLICY_EVAL_CTX_new

See https://github.com/openssl/openssl/pull/1554#issuecomment-246354677.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)
(cherry picked from commit f0f535e92b096db4a308ecc49ba7f0fd3f0f7945)

Default CT_POLICY_EVAL_CTX.epoch_time_in_ms to time()

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)
(cherry picked from commit e25233d99c30885bdf97bfb6df657e13ca2bf1da)

Reword documentation for {SCT_CTX/CT_POLICY_EVAL_CTX}_set_time

Do not call the time "current", as a different time can be provided.
For example, a time slightly in the future, to provide tolerance for
CT logs with a clock that is running fast.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)
(cherry picked from commit 1871a5aa8a538c2b8ac3d302c1e9e72867f5ee0f)

Remove obsolete error constant CT_F_CTLOG_NEW_NULL

ctlog_new_null() no longer exists.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)
(cherry picked from commit 333c2e43729a92cf37d4bd12d6a3531b4bd7e1da)

Check that SCT timestamps are not in the future

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)
(cherry picked from commit 1fa9ffd934429f140edcfbaf76d2f32cc21e449b)

Only build the body of e_padlock when there are lower level routines

engines/e_padlock.c assumes that for all x86 and x86_64 platforms, the
lower level routines will be present.  However, that's not always
true, for example for solaris-x86-cc, and that leads to build errors.

The better solution is to have configure detect if the lower level
padlock routines are being built, and define the macro PADLOCK_ASM if
they are, and use that macro in our C code.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1510)
(cherry picked from commit 7b176a549ea374fc9b64c3fa7f0812239528b696)

Add a warning stipulating how things should be coded in ossl_init_base

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1922)
(cherry picked from commit 8aa9cf7e655ae1e41f283fbf16dcc810970058a0)

Stop init loops

Under certain circumstances, the libcrypto init code would loop,
causing a deadlock.  This would typically happen if something in
ossl_init_base() caused an OpenSSL error, and the error stack routines
would recurse into the init code before the flag that ossl_init_base()
had been run was checked.

This change makes sure ossl_init_base isn't run once more of the base
is initiated.

Thanks to Dmitry Kostjuchenko for the idea.

Fixes Github issue #1899

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1922)
(cherry picked from commit b7a7f39afeb4748b4c25dbccb8951711b8b70eaf)

Configurations/10-main.conf: document GCC for Solaris config constraint.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit fe9e5b9ccce175d296c904486a29218c879adb73)

dsa/dsa_gen: add error message for seed_len < 0

prio openssl 1.1.0 seed_len < q was accepted and the seed argument was
then ignored. Now DSA_generate_parameters_ex() returns an error in such
a case but no error string.

Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1657)
(cherry picked from commit af5474126546b558b0e6f8be4bec4b70977e24b7)

Revert "Fixed deadlock in CRYPTO_THREAD_run_once for Windows"

This reverts commit edc18749bd5dfb7e12513d3978f78f9b56104fd6.

The proposed fix is incorrect. It marks the "run_once" code as having
finished before it has. The intended semantics of run_once is that no
threads should proceed until the code has run exactly once. With this
change the "second" thread will think the run_once code has already been
run and will continue, even though it is still in progress. This could
result in a crash or other incorrect behaviour.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Fixed deadlock in CRYPTO_THREAD_run_once for Windows

Fixed deadlock in CRYPTO_THREAD_run_once() if call to init() is causing
a recursive call to CRYPTO_THREAD_run_once() again that is causing a hot
deadloop inside do { } while (result == ONCE_ININIT); section.

CLA: trivial
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1913)

(cherry picked from commit 349d1cfddcfa33d352240582a3803f2eba39d9a0)

Solution proposal for issue #1647.

Avoid a memory alignment issue.

Signed-off-by: Matthias Kraft <Matthias.Kraft@softwareag.com>
CLA: trivial
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1650)

(cherry picked from commit af5883fec95eb8c79c379b09885440a0d88b2d38)

Update s_client and s_server documentation about some missing arguments

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1837)
(cherry picked from commit a22f9c84b468eed83c651cb5f2c68c7ad4103ffd)

Replace the 'SSL' broken link with SSL_CTX_set_security_level which seems not being referenced from elsewhere

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1898)
(cherry picked from commit e330f55d008ab99ee6c99b383061337fc4e7359d)

EVP docs: chacha20, chacha20-poly1305

CLA: trivial

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1909)
(cherry picked from commit 625b9d6b2a400e6b09f1e0278031f8417c363355)

Cast to an unsigned type before negating

llvm's ubsan reported:
runtime error: negation of -9223372036854775808 cannot be represented in
type 'int64_t' (aka 'long'); cast to an unsigned type to negate this
value to itself

Found using libfuzzer

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1908
(cherry picked from commit e80f3b6af295133107ac709329eee16ccf9af61c)

chacha/asm/chacha-x86.pl: improve [backward] portability.

In order to minimize dependency on assembler version a number of
post-SSE2 instructions are encoded manually. But in order to simplify
the procedure only register operands are considered. Non-register
operands are passed down to assembler. Module in question uses pshufb
with memory operands, and old [GNU] assembler can't handle it.
Fortunately in this case it's possible skip just the problematic
segment without skipping SSSE3 support altogether.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit d89773d659129368a341df746476da445d47ad31)

PPC assembler pack: add some PPC970/G5 performance data.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit cebb186989067b39fca6ebc378e4957408f6e701)

Fix the effect of no-dso in crypto/init.c

When configured no-dso, there are no DSO_{whatever} macros defined.
Therefore, before checking those, you have to check if OPENSSL_NO_DSO
is defined.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1902)
(cherry picked from commit 6e290a25c2cbdc26119c0866c20d9292f9e64dd8)

Small fixup of util/process_docs.pl

Apparently, pod2html doesn't add ".html" at the end of links, making
them useless, so we need to fix that

With thanks for the report to Michel <michel.sales@free.fr>

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1897)

Prepare for 1.1.0d-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.0c release

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update CHANGES and NEWS

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix the no-tls option

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix no-cms (CVE-2016-7053)

Reviewed-by: Matt Caswell <matt@openssl.org>

test/evptests.txt: add negative tests for AEAD ciphers.

This is done by taking one vector, "corrupting" last bit of the
tag value and verifying that decrypt fails.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

test: add TLS application data corruption test.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

add test for CVE-2016-7053

Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't set choice selector on parse failure.

Don't set choice selector on parse failure: this can pass unexpected
values to the choice callback. Instead free up partial structure
directly.

CVE-2016-7053

Thanks to Tyler Nighswander of ForAllSecure for reporting this issue.

Reviewed-by: Richard Levitte <levitte@openssl.org>

chacha20/poly1305: make sure to clear the buffer at correct position

The offset to the memory to clear was incorrect, causing a heap buffer
overflow.

CVE-2016-7054

Thanks to Robert Święcki for reporting this

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit b8e4011fb26364e44230946b87ab38cc1c719aae)

aes/asm/aesp8-ppc.pl: improve [backward] portability.

Some of stone-age assembler can't cope with r0 in address. It's actually
sensible thing to do, because r0 is shunted to 0 in address arithmetic
and by refusing r0 assembler effectively makes you understand that.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit a54aba531327285f64cf13a909bc129e9f9d5970)

bn/asm/x86_64-mont.pl: fix for CVE-2016-7055 (Low severity).

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit 2fac86d9abeaa643677d1ffd0a139239fdf9406a)

test/bntest.c: regression test for CVE-2016-7055.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit dca2e0ee1745ed2d9cba8c29f334f881a58f85dc)

Fix the evp_test Ctrl keyword processing

Skip the test if the value after ":" is a disabled algorithm, rather
than failing it

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit dfbdf4abb7c62156f36925db95728142c4223225)

Fix no-dso (shlibloadtest)

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit 586b79d8884b171eb3fae1ef230572921715ce1a)

When no SRP identity is found, no error was reported server side

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1859)

(cherry picked from commit 7bb37cb5938a0cf76c12c8421950e72634d5f61c)

Unix Makefile: Make sure to use $(PERL) when running ./Configure

For consistency, it's better to use the perl that was specified to
Configure last time it was called.

Use case:

perl v5.8.8 was first along $PATH, perl v5.22.2 was available and
specified as: PERL=/opt/local/bin/perl ./config.  When make wanted to
reconfigure and called './Configure reconf', configuration broke down,
complaining about a perl that's too old.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1884)
(cherry picked from commit 12ccb021be9e1c4c947e020ea2079e985b329a8a)

Missing BN_RECP_CTX field init.

BN_RECP_CTX_new direclty use bn_init to avoid twice memset calls

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1879)
(cherry picked from commit 318447bceb3aa2c50ac0081bdb4e917f8704e7da)

Zero stack variable with DSA nonce

Thanks to Falko Strenzke for bringing this to our attention.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1882)
(cherry picked from commit e5e71f2857275189577ab7b227608ab4ec985471)

INSTALL: small typo

libssl, not libddl.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1871)
(cherry picked from commit b77b6127e8de38726f37697bbbc736ced7b49771)

Allow null in  X509_CRL_METHOD_free

and fix documentation.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1634)
(cherry picked from commit 7cb1ecec59d7c8d6628fb9bfd435306f7e06fd33)

Improve PRF documentation

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1834
(cherry picked from commit 27ed73a98f88c98be996a6ffe7bda1b84bfc8be7)

Improve RSA test coverage.

MD5/SHA1 and MDC-2 have special-case logic beyond the generic DigestInfo
wrapping. Test that each of these works, including hash and length
mismatches (both input and signature). Also add VerifyRecover tests. It
appears 5824cc298174d462c827cd090675e30fc03f0caf added support for
VerifyRecover, but forgot to add the test data.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1474
(cherry picked from commit f320555735af7aa52172a2b8c56181445e8490dd)

Make RSA_sign.pod less confusing.

PKCS #1 v2.0 is the name of a document which specifies an algorithm
RSASSA-PKCS1-v1_5, often referred to as "PKCS #1 v1.5" after an earlier
document which specified it. This gets further confusing because the
document PKCS #1 v2.1 specifies two signature algorithms,
RSASSA-PKCS1-v1_5 and RSASSA-PSS. RSA_sign implements RSASSA-PKCS1-v1_5.

Refer to the document using the RFC number which is easier to find
anyway, and refer to the algorithm by its name.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1474
(cherry picked from commit aa90ca11c930114d5c0d68a2c1f446bf97853287)

Implement RSASSA-PKCS1-v1_5 as specified.

RFC 3447, section 8.2.2, steps 3 and 4 states that verifiers must encode
the DigestInfo struct and then compare the result against the public key
operation result. This implies that one and only one encoding is legal.

OpenSSL instead parses with crypto/asn1, then checks that the encoding
round-trips, and allows some variations for the parameter. Sufficient
laxness in this area can allow signature forgeries, as described in
https://www.imperialviolet.org/2014/09/26/pkcs1.html

Although there aren't known attacks against OpenSSL's current scheme,
this change makes OpenSSL implement the algorithm as specified. This
avoids the uncertainty and, more importantly, helps grow a healthy
ecosystem. Laxness beyond the spec, particularly in implementations
which enjoy wide use, risks harm to the ecosystem for all. A signature
producer which only tests against OpenSSL may not notice bugs and
accidentally become widely deployed. Thus implementations have a
responsibility to honor the specification as tightly as is practical.

In some cases, the damage is permanent and the spec deviation and
security risk becomes a tax all implementors must forever pay, but not
here. Both BoringSSL and Go successfully implemented and deployed
RSASSA-PKCS1-v1_5 as specified since their respective beginnings, so
this change should be compatible enough to pin down in future OpenSSL
releases.

See also https://tools.ietf.org/html/draft-thomson-postel-was-wrong-00

As a bonus, by not having to deal with sign/verify differences, this
version is also somewhat clearer. It also more consistently enforces
digest lengths in the verify_recover codepath. The NID_md5_sha1 codepath
wasn't quite doing this right.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1474
(cherry picked from commit 608a026494c1e7a14f6d6cfcc5e4994fe2728836)

Partial revert of "Fix client verify mode to check SSL_VERIFY_PEER"

This partially reverts commit c636c1c47. It also tweaks the documentation
and comments in this area. On the client side the documented interface for
SSL_CTX_set_verify()/SSL_set_verify() is that setting the flag
SSL_VERIFY_PEER causes verfication of the server certificate to take place.
Previously what was implemented was that if *any* flag was set then
verification would take place. The above commit improved the semantics to
be as per the documented interface.

However, we have had a report of at least one application where an
application was incorrectly using the interface and used *only*
SSL_VERIFY_FAIL_IF_NO_PEER_CERT on the client side. In OpenSSL prior to
the above commit this still caused verification of the server certificate
to take place. After this commit the application silently failed to verify
the server certificate.

Ideally SSL_CTX_set_verify()/SSL_set_verify() could be modified to indicate
if invalid flags were being used. However these are void functions!

The simplest short term solution is to revert to the previous behaviour
which at least means we "fail closed" rather than "fail open".

Thanks to Cory Benfield for reporting this issue.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit c8e2f98c97ff3327784843946c2d62761572e5d5)

Always ensure that init_msg is initialised for a CCS

We read it later in grow_init_buf(). If CCS is the first thing received in
a flight, then it will use the init_msg from the last flight we received. If
the init_buf has been grown in the meantime then it will point to some
arbitrary other memory location. This is likely to result in grow_init_buf()
attempting to grow to some excessively large amount which is likely to
fail. In practice this should never happen because the only time we receive
a CCS as the first thing in a flight is in an abbreviated handshake. None
of the preceding messages from the server flight would be large enough to
trigger this.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit c437757466e7bef632b26eaaf429a9e693330999)

Windows: use default ZLIB1 unless --with-zlib-lib is set

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1772)
(cherry picked from commit 475592e2419c5cb3098dfea4c9229d0c09ea7010)

Fix the LIBZ macro on VC config targets

If zlib-dynamic was given but not --with-zlib-lib, LIBZ was defined to
the empty string.  Instead, give it the default "ZLIB1".

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1772)
(cherry picked from commit 111b234c8f80371e7e31d922946cbd546491d4e8)

VMS: pretend to use -znodelete

VMS only unloads shared libraries at process rundown, so tell the
OpenSSL code so by pretending we linked with -znodelete.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1862)
(cherry picked from commit 1186a2b3d40e33cbf42d4fd3c7cc679f9f6e14f2)

VMS build file template: assign 'arch' to local symbol table

Since the local symbol table is looked up before the global symbol
table, 'arch' assigned in the local symbol table of the DCL where MMS
is called would be seen before the 'arch' defined in descrip.mms.
Assigning it to the local symbol table in descrip.mms removes that
issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1853)
(cherry picked from commit 3ee24d4acaff1c247db89c5cfcac17749dc3d7bc)

Missed a mention of RT

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1849)
(cherry picked from commit 1e62cc12f35408508594be254f40bf9b65d2a3a9)

Travis: add a strict build

Clang on Linux seems to catch things that we might miss otherwise.
Also, throw in 'no-deprecated' to make sure we test that as well.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1839)
(cherry picked from commit 7b1954384114643e1a3c3a0ababa3fd7a112c5e3)

Don't assume to know the shared library extension

test/shlibloadtest.c assumes all Unix style platforms use .so as
shared library extension.  This is not the case for Mac OS X, which
uses .dylib.  Instead of this, have the test recipe find out the
extension from configuration data.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1844)
(cherry picked from commit 62dd3351a16089aedb0f1e62e3b6df371c93389c)

Update CRYPTO_set_mem_debug() doc

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1842)
(cherry picked from commit 00bb5504cc62693e05ff4e699f379011c55ecc84)

Enable memory debugging while testing

Pre 1.1.0, 'make test' would set the environment variable
OPENSSL_DEBUG_MEMORY to "on".  This got lost when translating the old
build files to the new templates.  This changes reintroduces that
variable.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1840)
(cherry picked from commit 6d4bc8a3d2c2f7701588bbfdea80a1f7a3973f26)

test/shlibloadtest: small fixes

- Make sure to initialise SHLIB variables
- Make sure to make local variables static

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1838)
(cherry picked from commit 3b0478fe03191bedef44644996a5a6da51f77eaf)

Clang doesn't like -znodelete, make it a linker flag instead

gcc is kinder, it silently passes quite a few flags to ld, while clang
is stricter and wants them prefixed with -Wl,

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1836)
(cherry picked from commit 075e9da055454ec5586962b43d9923e44bdcb313)

test/build.info: typo, $ missing

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1832)
(cherry picked from commit 9c89c8460a4dcf828a22e2dfc279b5ea8a80ec60)