Fix ossl_statem_client_max_message_size() for DTLS1_BAD_VER

The Change Cipher Spec message in this ancient pre-standard version of DTLS
that Cisco are unfortunately still using in their products, is 3 bytes.

Allow it.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix SSL_export_keying_material() for DTLS1_BAD_VER

Commit d8e8590e ("Fix missing return value checks in SCTP") made the
DTLS handshake fail, even for non-SCTP connections, if
SSL_export_keying_material() fails. Which it does, for DTLS1_BAD_VER.

Apply the trivial fix to make it succeed, since there's no real reason
why it shouldn't even though we never need it.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

When tr gets bracketed arguments, they need to be quoted

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove some unused options from 10-main.conf

The options RC4_CHUNK_LL, DES_PTR, and BF_PTR were removed by Rich
in commit 3e9e810f2e047effb1056211794d2d12ec2b04e7 but were still
sticking around in a coupule configuration entries.

Since they're unused, remove them.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1390)

Fix output text to avoid gratuitious git diff

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove get_hash completely

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1389)

Remove "lockit" from internal error-hash function

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1389)

Limit status message sisze in ts_get_status_check

Thanks to Shi Lei for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Pack globals variables used to control apps/verify_callback()

  into a structure , to avoid any accident .

Plus some few cleanups

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Ignore windows generated manifests

- Commit a95ce7f builds *.manifest files on windows -- added them to
  .gitignore.

- ignore pod -> html temp file

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Constify some ASN1_OBJECT *obj input parameters

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Constify inputs of two X509_LOOKUP_METHOD methods

... get_by_fingerprint() and get_by_alias()

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Constify input buffer

 of X509_NAME_add_entry_by_OBJ, X509_NAME_add_entry_by_NID, X509_NAME_ENTRY_create_by_NID

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Constify two internal methods
- append_ia5
- old_entry_print

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Constify ASN1_INTEGER_get, ASN1_ENUMERATED_get

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

VMS: have the IVP verify that a well known engine loads properly

Reviewed-by: Rich Salz <rsalz@openssl.org>

Have 'openssl engine' exit with non-zero when some engine fails to load

Reviewed-by: Rich Salz <rsalz@openssl.org>

VSI submission: make the VMS version of RAND_poll() faster and more secure

Reviewed-by: Rich Salz <rsalz@openssl.org>

VSI submisson: make better use of item lists in o_time.c

Reviewed-by: Rich Salz <rsalz@openssl.org>

Travis: When testing installation, build in separate dir, otherwise in checkout

The rationale is that installation from a tarball is a common task
that everyone performs.  For all other builds, we do specialised
tests, and might as well build them directly in the checkout, which
also gives us fuzz corpora.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Prepare for 1.1.0-pre7-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.1.0-pre6 release

Reviewed-by: Richard Levitte <levitte@openssl.org>

make update

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix date in CHANGES

The release scripts expect to see the date "xx XXX xxxx" in CHANGES. At
some point the year got changed from xxxx to 2016. This changes it back.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't check any revocation info on proxy certificates

Because proxy certificates typically come without any CRL information,
trying to check revocation on them will fail.  Better not to try
checking such information for them at all.

Reviewed-by: Rich Salz <rsalz@openssl.org>

INSTALL: Make the use of [, ], { and } consistent and explain it

The diverse notations used in INSTALL are not as self explanatory as
we might imagine, so let's attempt a consistent notation for mandatory
and optional pieces of a command line, and to explain the meaning of
each notation.

This does away with the bash notation used in one spot, as it isn't
universally understood and will only confuse the unknowing more.

Reviewed-by: Rich Salz <rsalz@openssl.org>

INSTALL: Add missing details on VMS installation results

Reviewed-by: Rich Salz <rsalz@openssl.org>

VMS: make sure to provide an absolute source directory for pod2html

Experience shows that pod2html changes directory during its process
without properly adjusting the given source directory.

Reviewed-by: Rich Salz <rsalz@openssl.org>

VMS: If configured no-shared, don't provide shareable image logical names

Reviewed-by: Rich Salz <rsalz@openssl.org>

Check for overlows and error return from ASN1_object_size()

Reviewed-by: Richard Levitte <levitte@openssl.org>

Check for overflows in ASN1_object_size().

Reviewed-by: Richard Levitte <levitte@openssl.org>

80-test_ssl_new.t: only skip on $no_tls if no other skip conditions defined

Reviewed-by: Matt Caswell <matt@openssl.org>

In 80-test_ssl_new, more "plan tests" to a more useful position

Reviewed-by: Matt Caswell <matt@openssl.org>

Fix tests for no-nextprotoneg

Fix the 80-test_ssl_test_ctx and 80-test_ssl_new tests when used with the
no-nextprotoneg option

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix some style issues...

 extra spacing and 80 cols

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1366)

Don't overwrite existing installed openssl.cnf

Instead, install the new one as openssl.cnf.dist (openssl.cnf-dist on
VMS), and only install it as openssl.cnf if that file doesn't already
exist.

Also, don't install with exec privileges on VMS.

Reviewed-by: Rich Salz <rsalz@openssl.org>

All of ssldirs installation should be done by the install_ssldirs target

The Unix build file template didn't do that quite right.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Some minor tweaks to the fuzzing docs

Reviewed-by: Rich Salz <rsalz@openssl.org>

Document certificate and CRL time functions.

RT#4639

Reviewed-by: Rich Salz <rsalz@openssl.org>

Constify some X509_CRL, X509_REQ functions.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Constify some X509_CRL functions.

Reviewed-by: Rich Salz <rsalz@openssl.org>

make update

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add DSA_bits() function.

RT#4637

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix typo of BN_zero()

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Forgotten make update

Reviewed-by: Rich Salz <rsalz@openssl.org>

Const the ex data stuff too to fix warnings

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Const EC_KEY_dup

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Additional CMS content types from RFC 4073, RFC 5083, and RFC 5084.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1355)

Useless allocation

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Fix some style issues

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Add missing help string

Fix an error message

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Fix some awkward tests

Add some explanatory comments
Discard some useless parenthesis.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Merge some conditional blocks of code.

Fix an #if check about rsa_count...

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Simplify default inits, add const qualifiers
Simplify some loops to reuse k variable

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Cast to right type, simplify array args

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Fix sharing of two static variables

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

ECDH test is only one operation to run

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Fix three missing global declarations.

Fix compilation without SIGALRM.
It don't link because of remaning alarm(0); call.

Convert engine_id variable to local one

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Fix return values of do_passwd() in apps/passwd.c

do_passwd() was returning 0 on success and 1 on failure.  However,
those values were interpreted the other way around.  The fix that
makes the most sense is to change what do_passwd() returns.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix bad result in i2b_PVK()

The function i2b_PVK() was returning a bad pointer causing subsequent
crashes.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix param order in RSA_get0_key

Some calls to RSA_get0_key had the parameters in the wrong order causing a
failure.

GitHub Issue #1368

Reviewed-by: Richard Levitte <levitte@openssl.org>

Include what we use.

Reviewed-by: Rich Salz <rsalz@openssl.org>

peer_tmp doesn't exist if no-ec no-dh.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix various no-*s.

Reviewed-by: Rich Salz <rsalz@openssl.org>

crypto/ui/ui_openssl.c: UTF-y Windows code path.

Windows never composes UTF-8 strings as result of user interaction
such as input query. The only way to compose one is programmatic
conversion from WCHAR string, which in turn can be picked up with
ReadConsoleW.

Reviewed-by: Richard Levitte <levitte@openssl.org>

apps/openssl.c: UTF-y Windows argv.

Windows never composes UTF-8 strings as result of user interaction
such as passing command-line argument. The only way to compose one
is programmatic conversion from WCHAR string, which in turn can be
picked up on command line.

[For reference, why not wmain, it's not an option on MinGW.]

Reviewed-by: Richard Levitte <levitte@openssl.org>

Configurations/00-base-templates.conf: harmonize BASE_Windows.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Configurations/unix-Makefile.tmpl: add LC_ALL=C to unify messages.

RT#4138

Reviewed-by: Rich Salz <rsalz@openssl.org>

evp/bio_enc.c: perform enc_read operation without using overlapping buffers.

Reviewed-by: Stephen Henson <steve@openssl.org>

test/smcont.txt: trigger assertion in bio_enc.c.

Reviewed-by: Stephen Henson <steve@openssl.org>

evp/evp_enc.c: make assert error message more readable
and add EVPerr(PARTIALLY_OVERLAPPED)

Reviewed-by: Stephen Henson <steve@openssl.org>

Fix the docs too

Reviewed-by: Stephen Henson <steve@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1367)

Ignore the serial number for now and just do the rest.

Reviewed-by: Stephen Henson <steve@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1367)

Make some more X509 functions const.

Reviewed-by: Stephen Henson <steve@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1367)

rsa.c: fix incorrect guard for pvk-* options

This update syncs the #if guard protecting the pvk-* options
with the rest of the source handling those options. Also fix
some nearby whitespace.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1365)

Fix crash as a result of MULTIBLOCK

The MULTIBLOCK code uses a "jumbo" sized write buffer which it allocates
and then frees later. Pipelining however introduced multiple pipelines. It
keeps track of how many pipelines are initialised using numwpipes.
Unfortunately the MULTIBLOCK code was not updating this when in deallocated
its buffers, leading to a buffer being marked as initialised but set to
NULL.

RT#4618

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix bogus warnings

Fix some bogus "may be used uninitialized" warnings on some compilers.

Reviewed-by: Rich Salz <rsalz@openssl.org>

print out MAC algorithm

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix CRL time comparison.

Thanks to David Benjamin <davidben@google.com> for reporting this bug.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update the SSL_set_bio()/SSL_set0_rbio()/SSL_set0_wbio() docs

Update the documentation for the newly renamed and modified SSL_set0_rbio()
and SSL_set0_wbio() functions. State that they should be preferred over
SSL_set_bio(). Attempt to document the ownership rules for SSL_set_bio().

Reviewed-by: Rich Salz <rsalz@openssl.org>

Simplify and rename SSL_set_rbio() and SSL_set_wbio()

SSL_set_rbio() and SSL_set_wbio() are new functions in 1.1.0 and really
should be called SSL_set0_rbio() and SSL_set0_wbio(). The old
implementation was not consistent with what "set0" means though as there
were special cases around what happens if the rbio and wbio are the same.
We were only ever taking one reference on the BIO, and checking everywhere
whether the rbio and wbio are the same so as not to double free.

A better approach is to rename the functions to SSL_set0_rbio() and
SSL_set0_wbio(). If an existing BIO is present it is *always* freed
regardless of whether the rbio and wbio are the same or not. It is
therefore the callers responsibility to ensure that a reference is taken
for *each* usage, i.e. one for the rbio and one for the wbio.

The legacy function SSL_set_bio() takes both the rbio and wbio in one go
and sets them both. We can wrap up the old behaviour in the implementation
of that function, i.e. previously if the rbio and wbio are the same in the
call to this function then the caller only needed to ensure one reference
was passed. This behaviour is retained by internally upping the ref count.

This commit was inspired by BoringSSL commit f715c423224.

RT#4572

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add some SSL BIO tests

This adds some simple SSL BIO tests that check for pushing and popping of
BIOs into the chain. These tests would have caught the bugs fixed in the
previous three commits, if combined with a crypto-mdebug build.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix BIO_pop for SSL BIOs

The BIO_pop implementation assumes that the rbio still equals the next BIO
in the chain. While this would normally be the case, it is possible that it
could have been changed directly by the application. It also does not
properly cater for the scenario where the buffering BIO is still in place
for the write BIO.

Most of the existing BIO_pop code for SSL BIOs can be replaced by a single
call to SSL_set_bio(). This is equivalent to the existing code but
additionally handles the scenario where the rbio has been changed or the
buffering BIO is still in place.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix BIO_push ref counting for SSL BIO

When pushing a BIO onto an SSL BIO we set the rbio and wbio for the SSL
object to be the BIO that has been pushed. Therefore we need to up the ref
count for that BIO. The existing code was uping the ref count on the wrong
BIO.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Don't double free the write bio

When setting the read bio we free up any old existing one. However this can
lead to a double free if the existing one is the same as the write bio.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add a test for SSL_set_bio()

The SSL_set_bio() function has some complicated ownership rules. This adds a
test to make sure it all works as expected.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Make the checks for an SSLv2 style record stricter

SSLv2 is no longer supported in 1.1.0, however we *do* still accept an SSLv2
style ClientHello, as long as we then subsequently negotiate a protocol
version >= SSLv3. The record format for SSLv2 style ClientHellos is quite
different to SSLv3+. We only accept this format in the first record of an
initial ClientHello. Previously we checked this by confirming
s->first_packet is set and s->server is true. However, this really only
tells us that we are dealing with an initial ClientHello, not that it is
the first record (s->first_packet is badly named...it really means this is
the first message). To check this is the first record of the initial
ClientHello we should also check that we've not received any data yet
(s->init_num == 0), and that we've not had any empty records.

GitHub Issue #1298

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Various doc fixes.

Cannot nest B<> tags
Document "openssl speed" command.
Fix doc nits: missing NAME/SYNOPSIS stuff

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add memory sanitizer config, and run on travis.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Note cipher BIO write errors too.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Use sk_CONF_VALUE_pop_free in do_ext_nconf error path.

8605abf13523579ecab8b1f2a4bcb8354d94af79 fixed the nval leak, but it
used free instead of pop_free. nval owns its contents, so it should be
freed with pop_free. See the pop_free call a few lines down.

This is a no-op as, in this codepath, we must have nval == NULL or
sk_CONF_VALUE_num(nval) == 0. In those cases, free and pop_free are
identical. However, variables should be freed consistently.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1351)

Skip non-existing files.

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1324

Add Hurd shared extension

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1345

RT4639: Typo when -DSSL_DEBUG

Reviewed-by: Richard Levitte <levitte@openssl.org>

Set error if EVP_CipherUpdate fails.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Deprecate X509_LU_FAIL, X509_LU_RETRY

Instead of X509_LU_FAIL, X509_LU_RETRY use 0/1 for return values.

RT#4577

Reviewed-by: Rich Salz <rsalz@openssl.org>

Use X509_LOOKUP_TYPE for lookup type consistently.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Move X509_LU_RETRY, X509_LU_FAIL

X509_LU_RETRY and X509_LU_FAIL are not X509_OBJECT types so don't include
them in the enum.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove current_method from X509_STORE_CTX

Remove current_method: it was intended as a means of retrying
lookups bit it was never used. Now that X509_verify_cert() is
a "one shot" operation it can never work as intended.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update the example in proxy_certificates.txt

Reviewed-by: Rich Salz <rsalz@openssl.org>