Address feedback on SSLv2 ClientHello processing

Feedback on the previous SSLv2 ClientHello processing fix was that it
breaks layering by reading init_num in the record layer. It also does not
detect if there was a previous non-fatal warning.

This is an alternative approach that directly tracks in the record layer
whether this is the first record.

GitHub Issue #1298

Reviewed-by: Tim Hudson <tjh@openssl.org>

Replaces CT_POLICY_EVAL_CTX_set0 entries with new setters in libcrypto.num

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1408)

Make CT_POLICY_EVAL_CTX_set1_{cert,issuer} into boolean functions

They may fail if they cannot increment the reference count of the
certificate they are storing a pointer for. They should return 0 if this
occurs.

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1408)

Improves CTLOG_STORE setters

Changes them to have clearer ownership semantics, as suggested in
https://github.com/openssl/openssl/pull/1372#discussion_r73232196.

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1408)

Skip the SRP tests in 80-test_ssl_old.t if no TLS versions is enabled

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix no-ec

Fix no-ec builds by having separate functions to create keys based on
an existing EVP_PKEY and a curve id.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Never return -1 from BN_exp

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1455)

update CHANGES

Reviewed-by: Rich Salz <rsalz@openssl.org>

add documentation

Reviewed-by: Rich Salz <rsalz@openssl.org>

Print out names of other temp key algorithms.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove old EC based X25519 code.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Modify TLS support for new X25519 API.

When handling ECDH check to see if the curve is "custom" (X25519 is
currently the only curve of this type) and instead of setting a curve
NID just allocate a key of appropriate type.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add encoded points to other EC curves too.

Add encoded point ctrl support for other curves: this makes it possible
to handle X25519 and other EC curve point encoding in a similar way
for TLS.

Reviewed-by: Rich Salz <rsalz@openssl.org>

make update

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add point ctrls to X25519

Add ctrl operations to set or retrieve encoded point in
EVP_PKEY structures containing X25519 keys.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update X25519 key format in evptests.txt

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add X25519 methods to internal tables

Reviewed-by: Rich Salz <rsalz@openssl.org>

add to build.info

Reviewed-by: Rich Salz <rsalz@openssl.org>

make errors

Reviewed-by: Rich Salz <rsalz@openssl.org>

X25519 public key methods

Add X25519 methods to match current key format defined in
draft-ietf-curdle-pkix-02

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix type of ptr field.

Since "ptr" is used to handle arbitrary other types it should be
void *.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Use OIDs from draft-ietf-curdle-pkix-02

Reviewed-by: Rich Salz <rsalz@openssl.org>

GH1446: Add SSL_SESSION_get0_cipher

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1451)

Check for bad filename in evp_test

Thanks to Brian Carpter for reporting this.

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>

Update documentation for DSA_SIG and ECDSA_SIG.

RT#4590

Reviewed-by: Rich Salz <rsalz@openssl.org>

crypto/sparcv9cap.c: add missing declaration.

Reviewed-by: Rich Salz <rsalz@openssl.org>

crypto/ui/ui_openssl.c: let new-line through after query in Windows path.

Originally new-line was suppressed, because double new-line was
observed under wine. But it appears rather to be a wine bug,
because on real Windows new-line is much needed.

Reviewed-by: Richard Levitte <levitte@openssl.org>

crypto/sparcv9cap.c: fix overstep in getisax.

Problem was introduced in 299ccadcdb99001c618d188fb243c1caaaa86a1c
as future extension, i.e. at this point it wasn't an actual problem,
because uninitialized capability bit was not actually used.

Reviewed-by: Tim Hudson <tjh@openssl.org>

sha/asm/sha1-x86_64.pl: fix crash in SHAEXT code on Windows.

RT#4530

Reviewed-by: Tim Hudson <tjh@openssl.org>

Fix doc and help about ca -valid option

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Don't attempt to load the CT log list with no-ec

In practice, CT isn't really functional without EC anyway, as most logs
use EC keys. So, skip loading the log list with no-ec, and skip CT tests
completely in that conf.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fixed typo

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1386)

Documented BIO_set_accept_port()/BIO_get_accept_port()

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1386)

Adapt BIO_new_accept() to call BIO_set_accept_name()

Commit 417be66 broken BIO_new_accept() by changing the definition of the
macro BIO_set_accept_port() which stopped acpt_ctrl() from calling
BIO_parse_hostserv(). This commit completes the series of changes
initiated in 417be66.

Updated pods to reflect new definition introduced by 417be66.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1386)

Change callers to use the new constants.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1429)

Add #defines for magic numbers in API.

Binary- and backward-compatible.  Just better.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1429)

Fix spelling of error code

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1421)

Add some const casts

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1397)

GH1383: Add casts to ERR_PACK

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1385

Gracefully free a NULL HANDSHAKE_RESULT

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add TEST_check

Like OPENSSL_assert, but also prints the error stack before exiting.

Reviewed-by: Rich Salz <rsalz@openssl.org>

SSL tests: port CT tests, add a few more

This commit only ports existing tests, and adds some coverage for
resumption. We don't appear to have any handshake tests that cover SCT
validation success, and this commit doesn't change that.

Reviewed-by: Rich Salz <rsalz@openssl.org>

CT: fix documentation

Make method names match reality

Reviewed-by: Rich Salz <rsalz@openssl.org>

SSL test ctx: fix tests

Some failure tests were failing for the wrong reason after the CTX
refactoring. Update those tests.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Kill PACKET_starts() from bad_dtls_test

As discussed in PR#1409 it can be done differently.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix clienthellotest to use PACKET functions

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix test of first of 255 CBC padding bytes.

Thanks to Peter Gijsels for pointing out that if a CBC record has 255
bytes of padding, the first was not being checked.

(This is an import of change 80842bdb from BoringSSL.)

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1431)

speed.c: use size_t instead of int to match function signatures

Signed-off-by: Cristian Stoica <cristian.stoica@nxp.com>
Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1378)

NPN and ALPN: test resumption

In NPN and ALPN, the protocol is renegotiated upon resumption. Test that
resumption picks up changes to the extension.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix ALPN tests when NPN is off

OPENSSL_NO_NEXTPROTONEG only disables NPN, not ALPN

Reviewed-by: Richard Levitte <levitte@openssl.org>

Reorganize SSL test structures

Move custom server and client options from the test dictionary to an
"extra" section of each server/client. Rename test expectations to say
"Expected".

This is a big but straightforward change. Primarily, this allows us to
specify multiple server and client contexts without redefining the
custom options for each of them. For example, instead of
"ServerNPNProtocols", "Server2NPNProtocols", "ResumeServerNPNProtocols",
we now have, "NPNProtocols".

This simplifies writing resumption and SNI tests. The first application
will be resumption tests for NPN and ALPN.

Regrouping the options also makes it clearer which options apply to the
server, which apply to the client, which configure the test, and which
are test expectations.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix CIPHER_DEBUG

Commit 3eb2aff renamed a field of ssl_cipher_st from algorithm_ssl -> min_tls but neglected to update the fprintf reference which is included by -DCIPHER_DEBUG

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1417)

Add a note about a perl issue on VMS and how to work around it

I bug in perl's File::Spec->canonpath() was uncovered.  There's
nothing we can do about it (except re-implementing canonpath()),
except working around the problem (a directory rename) and reporting
the issue to the perl module developers.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix irregularities in GENERAL_NAME_print().

Add colon when printing Registered ID.
Remove extra space when printing DirName.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1401)

Mkae CT_log_new_from_base64 always return 0 on failure

In one failure case, it used to return -1. That failure case
(CTLOG_new() returning NULL) was not usefully distinct from all of the
other failure cases.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1407)

fixing too optimistic typo-fix

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1413)

spelling fixes, just comments and readme.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1413)

Removes CTLOG_new_null from the CT public API

This is an entirely useless function, given that CTLOG is publicly
immutable.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1406)

openssl-format-source: A few more (DECLARE|IMPLEMENT) variants to care for

Reviewed-by: Rich Salz <rsalz@openssl.org>

The capi engine uses stdio, so don't build it when configuring 'no-stdio'

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove OPENSSL_NO_STDIO guards around certain SSL cert/key functions

These functions are:

    SSL_use_certificate_file
    SSL_use_RSAPrivateKey_file
    SSL_use_PrivateKey_file
    SSL_CTX_use_certificate_file
    SSL_CTX_use_RSAPrivateKey_file
    SSL_CTX_use_PrivateKey_file
    SSL_use_certificate_chain_file

Internally, they use BIO_s_file(), which is defined and implemented at
all times, even when OpenSSL is configured no-stdio.

Reviewed-by: Rich Salz <rsalz@openssl.org>

make update

Reviewed-by: Rich Salz <rsalz@openssl.org>

util/mkdef.pl: mark certain PEM function declarations with STDIO

The macros that produce PEM_write_FOO() andd PEM_read_FOO() only do so
unless 'no-stdio' has been configured.  mkdef.pl should mimic that by
marking those functions with the "STDIO" algo.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Travis: add a build with no-stdio

Reviewed-by: Rich Salz <rsalz@openssl.org>

Move the building of test/buildtest_*. to be done unconditionally

These were guarded by $disabled{tests}.  However, 'tests' is disabled
if we configure 'no-stdio', which means that we don't detect the lack
of OPENSSL_NO_STDIO guards in our public header files.  So we move the
generation and build of test/buildtest_*.c to be unconditional.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Sanity check input length in OPENSSL_uni2asc().

Thanks to Hanno Böck for reporting this bug.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add --gcov-options '\-lp' to coverage

Should result in more accurate header file coverage, see
https://github.com/eddyxu/cpp-coveralls/issues/54

Reviewed-by: Kurt Roeckx <kurt@openssl.org>

Add a coverage target

Run tests with coverage and report to coveralls.io

For simplicity, this currently only adds a single target in a
configuration that attempts to maximize coverage. The true CI coverage
from all the various builds may be a little larger.

The coverage run has the following configuration:
- no-asm: since we can't track asm coverage anyway, might as well measure the
  non-asm code coverage.
- Enable various disabled-by-default options:
  - rc5
  - md2
  - ec_nistp_64_gcc_128
  - ssl3
  - ssl3-method
  - weak-ssl-ciphers

Finally, observe that no-pic implies no-shared, and therefore running
both builds in the matrix is redundant.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>

Free buffer in a2i_ASN1_INTEGER() on error path.

Thank to Shi Lei for reporting this bug.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Make update, etc.

Reviewed-by: Richard Levitte <levitte@openssl.org>

indent: add a couple of types we use in apps

Reviewed-by: Emilia Käsper <emilia@openssl.org>

openssl-format-source: no dash marker on *INDENT-(ON|OFF)* comments

We mark small comments with a dash immediately following the starting /*.
However, *INDENT-(ON|OFF)* comments shouldn't be treated that way, or
indent will ignore them if we do.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

VMS: Fix building of bad_dtls_test

Reviewed-by: Tim Hudson <tjh@openssl.org>

Added appropriate OPENSSL_NO_STDIO to PKCS12 header

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Add basic test for Cisco DTLS1_BAD_VER and record replay handling

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix ubsan 'left shift of negative value -1' error in satsub64be()

Baroque, almost uncommented code triggers behaviour which is undefined
by the C standard. You might quite reasonably not care that the code was
broken on ones-complement machines, but if we support a ubsan build then
we need to at least pretend to care.

It looks like the special-case code for 64-bit big-endian is going to
behave differently (and wrongly) on wrap-around, because it treats the
values as signed. That seems wrong, and allows replay and other attacks.
Surely you need to renegotiate and start a new epoch rather than
wrapping around to sequence number zero again?

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Make DTLS1_BAD_VER work with DTLS_client_method()

DTLSv1_client_method() is deprecated, but it was the only way to obtain
DTLS1_BAD_VER support. The SSL_OP_CISCO_ANYCONNECT hack doesn't work with
DTLS_client_method(), and it's relatively non-trivial to make it work without
expanding the hack into lots of places.

So deprecate SSL_OP_CISCO_ANYCONNECT with DTLSv1_client_method(), and make
it work with SSL_CTX_set_{min,max}_proto_version(DTLS1_BAD_VER) instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix cipher support for DTLS1_BAD_VER

Commit 3eb2aff40 ("Add support for minimum and maximum protocol version
supported by a cipher") disabled all ciphers for DTLS1_BAD_VER.

That wasn't helpful. Give them back.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix DTLS_VERSION_xx() comparison macros for DTLS1_BAD_VER

DTLS version numbers are strange and backwards, except DTLS1_BAD_VER so
we have to make a special case for it.

This does leave us with a set of macros which will evaluate their arguments
more than once, but it's not a public-facing API and it's not like this is
the kind of thing where people will be using DTLS_VERSION_LE(x++, y) anyway.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix ossl_statem_client_max_message_size() for DTLS1_BAD_VER

The Change Cipher Spec message in this ancient pre-standard version of DTLS
that Cisco are unfortunately still using in their products, is 3 bytes.

Allow it.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix SSL_export_keying_material() for DTLS1_BAD_VER

Commit d8e8590e ("Fix missing return value checks in SCTP") made the
DTLS handshake fail, even for non-SCTP connections, if
SSL_export_keying_material() fails. Which it does, for DTLS1_BAD_VER.

Apply the trivial fix to make it succeed, since there's no real reason
why it shouldn't even though we never need it.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

When tr gets bracketed arguments, they need to be quoted

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove some unused options from 10-main.conf

The options RC4_CHUNK_LL, DES_PTR, and BF_PTR were removed by Rich
in commit 3e9e810f2e047effb1056211794d2d12ec2b04e7 but were still
sticking around in a coupule configuration entries.

Since they're unused, remove them.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1390)

Fix output text to avoid gratuitious git diff

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove get_hash completely

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1389)

Remove "lockit" from internal error-hash function

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1389)

Limit status message sisze in ts_get_status_check

Thanks to Shi Lei for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Pack globals variables used to control apps/verify_callback()

  into a structure , to avoid any accident .

Plus some few cleanups

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Ignore windows generated manifests

- Commit a95ce7f builds *.manifest files on windows -- added them to
  .gitignore.

- ignore pod -> html temp file

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Constify some ASN1_OBJECT *obj input parameters

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Constify inputs of two X509_LOOKUP_METHOD methods

... get_by_fingerprint() and get_by_alias()

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Constify input buffer

 of X509_NAME_add_entry_by_OBJ, X509_NAME_add_entry_by_NID, X509_NAME_ENTRY_create_by_NID

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Constify two internal methods
- append_ia5
- old_entry_print

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Constify ASN1_INTEGER_get, ASN1_ENUMERATED_get

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

VMS: have the IVP verify that a well known engine loads properly

Reviewed-by: Rich Salz <rsalz@openssl.org>

Have 'openssl engine' exit with non-zero when some engine fails to load

Reviewed-by: Rich Salz <rsalz@openssl.org>

VSI submission: make the VMS version of RAND_poll() faster and more secure

Reviewed-by: Rich Salz <rsalz@openssl.org>

VSI submisson: make better use of item lists in o_time.c

Reviewed-by: Rich Salz <rsalz@openssl.org>

Travis: When testing installation, build in separate dir, otherwise in checkout

The rationale is that installation from a tarball is a common task
that everyone performs.  For all other builds, we do specialised
tests, and might as well build them directly in the checkout, which
also gives us fuzz corpora.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Prepare for 1.1.0-pre7-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>