Fix mac-then-encrypt test with enable-tls1_3

Commit b3618f44 added a test for mac-then-encrypt. However the test fails
when running with "enable-tls1_3". The problem is that the test creates a
connection, which ends up being TLSv1.3. However it also restricts the
ciphers to a single mac-then-encrypt ciphersuite that is not TLSv1.3
compatible so the connection aborts and the test fails. Mac-then-encrypt
is not relevant to TLSv1.3, so the test should disable that protocol
version.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Ensure we are in accept state in DTLSv1_listen

Calling SSL_set_accept_state() after DTLSv1_listen() clears the state, so
SSL_accept() no longer works. In 1.0.2 calling DTLSv1_listen() would set
the accept state automatically. We should still do that.

Fixes #1989

Reviewed-by: Andy Polyakov <appro@openssl.org>

Make bntest be (mostly) file-based.

Test suite used from boring, written by David Benjamin.
Test driver converted from C++ to C.
Added a Perl program to check the testsuite file.
Extensive review feedback incorporated (thanks folks).

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Test mac-then-encrypt

Verify that the encrypt-then-mac negotiation is handled
correctly. Additionally, when compiled with no-asm, this test ensures
coverage for the constant-time MAC copying code in
ssl3_cbc_copy_mac. The proxy-based CBC padding test covers that as
well but it's nevertheless better to have an explicit handshake test
for mac-then-encrypt.

Reviewed-by: Andy Polyakov <appro@openssl.org>

add CMS SHA1 signing test

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix ctrl operation for SHA1/MD5SHA1.

This makes S/MIME and CMS signing in MIME format for SHA1 work again.

Reviewed-by: Rich Salz <rsalz@openssl.org>

INSTALL: clarify 386 and no-sse2 options.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

modes/ctr128.c: fix false carry in counter increment procedure.

GH issue #1916 affects only big-endian platforms. TLS is not affected,
because TLS fragment is never big enough.

Reviewed-by: Matt Caswell <matt@openssl.org>

test/evptests.txt: add regression test for false carry in ctr128.c.

GH issue #1916 affects only big-endian platforms. TLS is not affected,
because TLS fragment is never big enough.

Reviewed-by: Matt Caswell <matt@openssl.org>

bn/asm/ppc-mont.pl: signal no-op in 32-bit bit build.

The bug was introduced in 80d27cdb84985c697f8fabb7649abf1f54714d13,
one too many instructions was removed. It went unnoticed, because
new subroutine introduced in previous commit is called in real-life
RSA/DSA/DH cases, while original code is called only in rare tests.
The bug was caught in test_fuzz.

Reviewed-by: Rich Salz <rsalz@openssl.org>

coveralls: Use gcov-5 since we build it using gcc-5

Reviewed-by: Emilia Käsper <emilia@openssl.org>
GH: #2003

Fix a bogus uninit var warning

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix a warning about an uninit var

Reviewed-by: Richard Levitte <levitte@openssl.org>

Run BoringSSL tests on Travis

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix a missing function prototype in AFALG engine

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix missing NULL checks in key_share processing

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix missing NULL checks in CKE processing

Reviewed-by: Rich Salz <rsalz@openssl.org>

Use ClientHello.legacy_version for the RSA pre-master no matter what

Don't use what is in supported_versions for the RSA pre-master

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Fix some defines in ossl_shim

ossl_shim had some TLS1.3 defines that are now in ssl.h so need to be
removed.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Fix a double ;; causing a travis failure

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix an uninit variable usage

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update tls13secretstest to use the new simpler test framework

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix some style issues with TLSv1.3 state machine PR

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix some style issues identified during review

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix SSL_IS_TLS13(s)

The SSL_IS_TLS13() macro wasn't quite right. It would come back with true
in the case where we haven't yet negotiated TLSv1.3, but it could be
negotiated.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Extend test_tls13messages

Add various different handshake types that are possible.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add an s_server capability to read an OCSP Response from a file

Current s_server can only get an OCSP Response from an OCSP responder. This
provides the capability to instead get the OCSP Response from a DER encoded
file.

This should make testing of OCSP easier.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add a test for the TLSv1.3 state machine

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix some TLSProxy warnings

After the client processes the server's initial flight in TLS1.3 it may
respond with either an encrypted, or an unencrypted alert. We needed to
teach TLSProxy about this so that it didn't issue spurious warnings.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Ensure unexpected messages are handled consistently

In one case we weren't always sending an unexpected message alert if we
don't get what we expect.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Ensure the end of first server flight processing is done

There is a set of miscellaneous processing for OCSP, CT etc at the end of
the ServerDone processing. In TLS1.3 we don't have a ServerDone, so this
needs to move elsewhere.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix the tests following the state machine changes for TLSv1.3

Reviewed-by: Rich Salz <rsalz@openssl.org>

Create the Finished message payload

The previous commit had a dummy payload for the Finished data. This commit
fills it in with a real value.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update state machine to be closer to TLS1.3

This is a major overhaul of the TLSv1.3 state machine. Currently it still
looks like TLSv1.2. This commit changes things around so that it starts
to look a bit less like TLSv1.2 and bit more like TLSv1.3.

After this commit we have:

ClientHello
+ key_share          ---->
                           ServerHello
                           +key_share
                           {CertificateRequest*}
                           {Certificate*}
                           {CertificateStatus*}
                     <---- {Finished}
{Certificate*}
{CertificateVerify*}
{Finished}           ---->
[ApplicationData]    <---> [Application Data]

Key differences between this intermediate position and the final TLSv1.3
position are:
- No EncryptedExtensions message yet
- No server side CertificateVerify message yet
- CertificateStatus still exists as a separate message
- A number of the messages are still in the TLSv1.2 format
- Still running on the TLSv1.2 record layer

Reviewed-by: Rich Salz <rsalz@openssl.org>

Implement tls13_change_cipher_state()

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove old style NewSessionTicket from TLSv1.3

TLSv1.3 has a NewSessionTicket message, but it is *completely* different to
the TLSv1.2 one and may as well have been called something else. This commit
removes the old style NewSessionTicket from TLSv1.3. We will have to add the
new style one back in later.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix EXTMS error introduced by commit 94ed2c6

Commit 94ed2c6 dropped a ! operator by mistake, which causes extended
master secret connections to fail. This puts in back.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Clarify what X509_NAME_online does with the given buffer and size

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1980)

Add missing -zdelete for some linux arches

b6d5ba1a9f004d637acac18ae3519fe063b6b5e1 forgot to update some linux arches.

Reviewed-by: Richard Levitte <levitte@openssl.org>
GH: #1977

Skipping tests in evp_test leaks memory

When configured with "no-mdc2 enable-crypto-mdebug" the evp_test
will leak memory due to skipped tests, and error out.

Also fix a skip condition

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1968)

Make SSL_read and SSL_write return the old behaviour and document it.

This reverts commit 4880672a9b41a09a0984b55e219f02a2de7ab75e.

Fixes: #1903

Reviewed-by: Matt Caswell <matt@openssl.org>
GH: #1931

Add test to check EVP_PKEY method ordering.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Avoid warnings like unused enum value (as suggested by Rich Salz)

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1945)

Shift inside of switch back one level (code style change, as requested by Rich Salz)

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1945)

Add LMTP support (RFC 2033) to s_client ("-starttls lmtp")

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1945)

Use consistent variable names in example

In the X509_NAME_get_index_by_NID.pod example, the initialized variable is called
"loc", but the one used in the for loop is called "lastpos". Make the names match.

CLA: trivial
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1949)

Update the location of the pod files

CLA: trivial
Since 99d63d4 ("Move manpages to man[1357] structure.", 2016-10-26), the location
of the pod files has changed.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1951)

Fix name of "locked" variable

It's called with 0 when it's already locked, with 1 when it's not.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1500

Add support for reference counting using C11 atomics

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1500

Support MSBLOB format if RC4 is disabled

Reviewed-by: Matt Caswell <matt@openssl.org>

sha/asm/sha512-armv8.pl: fix big-endian support in __KERNEL__ case.

In non-__KERNEL__ context 32-bit-style __ARMEB__/__ARMEL__ macros were
set in arm_arch.h, which is shared between 32- and 64-bit builds. Since
it's not included in __KERNEL__ case, we have to adhere to official
64-bit pre-defines, __AARCH64EB__/__AARCH64EL__.

[If we are to share more code, it would need similar adjustment.]

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix missing SSL_IS_TLS13(s) usage

We should use the macro for testing if we are using TLSv1.3 rather than
checking s->version directly.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Handle "int_ctx_new:unsupported algorithm" error

Calling EVP_PKEY_CTX_new_id(curve_NID, NULL) causes an error for most
curves that are implemented through the EC low-level API, and in the
last commit we call it for every curve to avoid treating X25519 as a
special case.

Last commit code already handles correctly this failure, but does not
remove these events from the thread error queue, thus some
false-positive warnings are printed at the end of execution.

This commit ensures that the error queue is clean, without flushing
other errors.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1658)

Do not handle R_EC_X25519 as a special case

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1658)

Remove leftover KDF pointer

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1658)

Use CRYPTO_memcmp for comparing derived secrets

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1658)

Reintroduce preliminary sanity check in ECDH speed and remove further checks in the benchmark loop.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1658)

Remove ECDH_EVP_derive_key wrapper function

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1658)

More coding style fixes

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1658)

Run util/openssl-format-source against apps/speed.c

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1658)

bugfix: calculate outlen for each curve

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1658)

Handle EVP_PKEY_derive errors and fix coding style issues

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1658)

Fix coding style and remove some stale code/comments

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1658)

Use EVP interface for ECDH in apps/speed.c

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1658)

Add conversion test for MSBLOB format.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Make MSBLOB format work with dsa utility.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix MSBLOB format with RSA.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Raise an error on memory alloc failure.

Both strdup or malloc failure should raise à err.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1905)

Missing free item on push failure

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1905)

Merge find-undoc-api into find-doc-nits

Use \b on NOEXIST and EXPORT_VAR_AS_FUNC patterns as suggested by Andy.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1912)

Move SCT_LIST_free definition into a more logical place

This reflects its position in include/openssl/ct.h.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1548)

Make sure things get deleted when test setup fails in ct_test.c

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1548)

Use valid signature in test_decode_tls_sct()

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1548)

Pass a temporary pointer to o2i_SCT_signature from SCT_new_from_base64

Otherwise, |dec| gets moved past the end of the signature by
o2i_SCT_signature and then can't be correctly freed afterwards.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1548)

Subtract padding from outlen in ct_base64_decode

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1548)

Construct SCT from base64 in ct_test

This gives better code coverage and is more representative of how a
user would likely construct an SCT (using the base64 returned by a CT log).

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1548)

On x86 machines where the compiler supports -m32, use 'linux-x86'

The rationale is that the linux-x86 is the most likely config target
to evolve and should therefore be chosen when possible, while
linux-elf is mostly reserved for older Linux machines.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1924)

Add a modern linux-x86 config target

'linux-x86' is similar to 'linux-x86_64' but uses -m32 rather than -m64.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1924)

Fix a "defined but not used" warning when enabling ssl-trace

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove a hack from ssl_test_old

ssl_test_old was reaching inside the SSL structure and changing the internal
BIO values. This is completely unneccessary, and was causing an abort in the
test when enabling TLSv1.3.

I also removed the need for ssl_test_old to include ssl_locl.h. This
required the addition of some missing accessors for SSL_COMP name and id
fields.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add SSL_peek() and SSL_peek_ex() to NAME section

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix a typo in a comment

Reviewed-by: Rich Salz <rsalz@openssl.org>

Move getting the curvelist for client and server out of the loop

No need to continually get the list of supported curves for the client
and server. Just do it once.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fixed various style issues in the key_share code

Numerous style issues as well as references to TLS1_3_VERSION instead of
SSL_IS_TLS13(s)

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add some tests for the key_share extension

Reviewed-by: Rich Salz <rsalz@openssl.org>

Check key_exchange data length is not 0

Reviewed-by: Rich Salz <rsalz@openssl.org>

Ensure the whole key_share extension is well formatted

Reviewed-by: Rich Salz <rsalz@openssl.org>

Validate that the provided key_share is in supported_groups

Reviewed-by: Rich Salz <rsalz@openssl.org>

Start using the key_share data to derive the PMS

The previous commits put in place the logic to exchange key_share data. We
now need to do something with that information. In <= TLSv1.2 the equivalent
of the key_share extension is the ServerKeyExchange and ClientKeyExchange
messages. With key_share those two messages are no longer necessary.

The commit removes the SKE and CKE messages from the TLSv1.3 state machine.
TLSv1.3 is completely different to TLSv1.2 in the messages that it sends
and the transitions that are allowed. Therefore, rather than extend the
existing <=TLS1.2 state transition functions, we create a whole new set for
TLSv1.3. Intially these are still based on the TLSv1.2 ones, but over time
they will be amended.

The new TLSv1.3 transitions remove SKE and CKE completely. There's also some
cleanup for some stuff which is not relevant to TLSv1.3 and is easy to
remove, e.g. the DTLS support (we're not doing DTLSv1.3 yet) and NPN.

I also disable EXTMS for TLSv1.3. Using it was causing some added
complexity, so rather than fix it I removed it, since eventually it will not
be needed anyway.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add a TLS version consistency check during session resumption

This is a temporary fix for while we are still using the old session
resumption logic in the TLSv1.3 code. Due to differences in EXTMS support
we can't resume a <=TLSv1.2 session in a TLSv1.3 connection (the EXTMS
consistency check causes the connection to abort). This causes test
failures.

Ultimately we will rewrite the session resumption logic for TLSv1.3 so this
problem will go away. But until then we need a quick fix to keep the tests
happy.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add processing of the key_share received in the ServerHello

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add key_share info to the ServerHello

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update the trace code to know about the key_share extension

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add the key_share processing to the server side

At the moment the server doesn't yet do anything with this information.
We still need to send the server's key_share info back to the client. That
will happen in subsequent commits.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Generate the key_shares extension on the client side

In this commit we just generate the extension on the client side, but don't
yet do anything with it. Subsequent commits, will add the server side
capability.

At the moment we hard code a single key_share. In the future we should make
this configurable.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Check return value of some BN functions.

Factorise multiple bn_get_top(group->field) calls
Add missing checks on some conditional BN_copy return value
Add missing checks on some BN_copy return value
Add missing checks on a few bn_wexpand return value

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1626)

Introduce PATH_MAX and NAME_MAX

 to define the certificate filename storage buffer.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1569)

Simplify and fix usage of three string array variable...

 using two separated local variables.

buf[1] was unused.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1569)

Add test for CT_POLICY_EVAL_CTX default time

Checks that the epoch_time_in_ms field of CT_POLICY_EVAL_CTX is initialized
to approximately the current time (as returned by time()) by default. This
prevents the addition of this field, and its verification during SCT
validation, from breaking existing code that calls SCT_validate directly.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)

Convert C++ comments to C-style comments

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)