Fix some awkward tests

Add some explanatory comments
Discard some useless parenthesis.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Merge some conditional blocks of code.

Fix an #if check about rsa_count...

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Simplify default inits, add const qualifiers
Simplify some loops to reuse k variable

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Cast to right type, simplify array args

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Fix sharing of two static variables

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

ECDH test is only one operation to run

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Fix three missing global declarations.

Fix compilation without SIGALRM.
It don't link because of remaning alarm(0); call.

Convert engine_id variable to local one

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1342)

Fix return values of do_passwd() in apps/passwd.c

do_passwd() was returning 0 on success and 1 on failure.  However,
those values were interpreted the other way around.  The fix that
makes the most sense is to change what do_passwd() returns.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix bad result in i2b_PVK()

The function i2b_PVK() was returning a bad pointer causing subsequent
crashes.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix param order in RSA_get0_key

Some calls to RSA_get0_key had the parameters in the wrong order causing a
failure.

GitHub Issue #1368

Reviewed-by: Richard Levitte <levitte@openssl.org>

Include what we use.

Reviewed-by: Rich Salz <rsalz@openssl.org>

peer_tmp doesn't exist if no-ec no-dh.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix various no-*s.

Reviewed-by: Rich Salz <rsalz@openssl.org>

crypto/ui/ui_openssl.c: UTF-y Windows code path.

Windows never composes UTF-8 strings as result of user interaction
such as input query. The only way to compose one is programmatic
conversion from WCHAR string, which in turn can be picked up with
ReadConsoleW.

Reviewed-by: Richard Levitte <levitte@openssl.org>

apps/openssl.c: UTF-y Windows argv.

Windows never composes UTF-8 strings as result of user interaction
such as passing command-line argument. The only way to compose one
is programmatic conversion from WCHAR string, which in turn can be
picked up on command line.

[For reference, why not wmain, it's not an option on MinGW.]

Reviewed-by: Richard Levitte <levitte@openssl.org>

Configurations/00-base-templates.conf: harmonize BASE_Windows.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Configurations/unix-Makefile.tmpl: add LC_ALL=C to unify messages.

RT#4138

Reviewed-by: Rich Salz <rsalz@openssl.org>

evp/bio_enc.c: perform enc_read operation without using overlapping buffers.

Reviewed-by: Stephen Henson <steve@openssl.org>

test/smcont.txt: trigger assertion in bio_enc.c.

Reviewed-by: Stephen Henson <steve@openssl.org>

evp/evp_enc.c: make assert error message more readable
and add EVPerr(PARTIALLY_OVERLAPPED)

Reviewed-by: Stephen Henson <steve@openssl.org>

Fix the docs too

Reviewed-by: Stephen Henson <steve@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1367)

Ignore the serial number for now and just do the rest.

Reviewed-by: Stephen Henson <steve@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1367)

Make some more X509 functions const.

Reviewed-by: Stephen Henson <steve@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1367)

rsa.c: fix incorrect guard for pvk-* options

This update syncs the #if guard protecting the pvk-* options
with the rest of the source handling those options. Also fix
some nearby whitespace.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1365)

Fix crash as a result of MULTIBLOCK

The MULTIBLOCK code uses a "jumbo" sized write buffer which it allocates
and then frees later. Pipelining however introduced multiple pipelines. It
keeps track of how many pipelines are initialised using numwpipes.
Unfortunately the MULTIBLOCK code was not updating this when in deallocated
its buffers, leading to a buffer being marked as initialised but set to
NULL.

RT#4618

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix bogus warnings

Fix some bogus "may be used uninitialized" warnings on some compilers.

Reviewed-by: Rich Salz <rsalz@openssl.org>

print out MAC algorithm

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix CRL time comparison.

Thanks to David Benjamin <davidben@google.com> for reporting this bug.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update the SSL_set_bio()/SSL_set0_rbio()/SSL_set0_wbio() docs

Update the documentation for the newly renamed and modified SSL_set0_rbio()
and SSL_set0_wbio() functions. State that they should be preferred over
SSL_set_bio(). Attempt to document the ownership rules for SSL_set_bio().

Reviewed-by: Rich Salz <rsalz@openssl.org>

Simplify and rename SSL_set_rbio() and SSL_set_wbio()

SSL_set_rbio() and SSL_set_wbio() are new functions in 1.1.0 and really
should be called SSL_set0_rbio() and SSL_set0_wbio(). The old
implementation was not consistent with what "set0" means though as there
were special cases around what happens if the rbio and wbio are the same.
We were only ever taking one reference on the BIO, and checking everywhere
whether the rbio and wbio are the same so as not to double free.

A better approach is to rename the functions to SSL_set0_rbio() and
SSL_set0_wbio(). If an existing BIO is present it is *always* freed
regardless of whether the rbio and wbio are the same or not. It is
therefore the callers responsibility to ensure that a reference is taken
for *each* usage, i.e. one for the rbio and one for the wbio.

The legacy function SSL_set_bio() takes both the rbio and wbio in one go
and sets them both. We can wrap up the old behaviour in the implementation
of that function, i.e. previously if the rbio and wbio are the same in the
call to this function then the caller only needed to ensure one reference
was passed. This behaviour is retained by internally upping the ref count.

This commit was inspired by BoringSSL commit f715c423224.

RT#4572

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add some SSL BIO tests

This adds some simple SSL BIO tests that check for pushing and popping of
BIOs into the chain. These tests would have caught the bugs fixed in the
previous three commits, if combined with a crypto-mdebug build.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix BIO_pop for SSL BIOs

The BIO_pop implementation assumes that the rbio still equals the next BIO
in the chain. While this would normally be the case, it is possible that it
could have been changed directly by the application. It also does not
properly cater for the scenario where the buffering BIO is still in place
for the write BIO.

Most of the existing BIO_pop code for SSL BIOs can be replaced by a single
call to SSL_set_bio(). This is equivalent to the existing code but
additionally handles the scenario where the rbio has been changed or the
buffering BIO is still in place.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix BIO_push ref counting for SSL BIO

When pushing a BIO onto an SSL BIO we set the rbio and wbio for the SSL
object to be the BIO that has been pushed. Therefore we need to up the ref
count for that BIO. The existing code was uping the ref count on the wrong
BIO.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Don't double free the write bio

When setting the read bio we free up any old existing one. However this can
lead to a double free if the existing one is the same as the write bio.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add a test for SSL_set_bio()

The SSL_set_bio() function has some complicated ownership rules. This adds a
test to make sure it all works as expected.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Make the checks for an SSLv2 style record stricter

SSLv2 is no longer supported in 1.1.0, however we *do* still accept an SSLv2
style ClientHello, as long as we then subsequently negotiate a protocol
version >= SSLv3. The record format for SSLv2 style ClientHellos is quite
different to SSLv3+. We only accept this format in the first record of an
initial ClientHello. Previously we checked this by confirming
s->first_packet is set and s->server is true. However, this really only
tells us that we are dealing with an initial ClientHello, not that it is
the first record (s->first_packet is badly named...it really means this is
the first message). To check this is the first record of the initial
ClientHello we should also check that we've not received any data yet
(s->init_num == 0), and that we've not had any empty records.

GitHub Issue #1298

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Various doc fixes.

Cannot nest B<> tags
Document "openssl speed" command.
Fix doc nits: missing NAME/SYNOPSIS stuff

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add memory sanitizer config, and run on travis.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Note cipher BIO write errors too.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Use sk_CONF_VALUE_pop_free in do_ext_nconf error path.

8605abf13523579ecab8b1f2a4bcb8354d94af79 fixed the nval leak, but it
used free instead of pop_free. nval owns its contents, so it should be
freed with pop_free. See the pop_free call a few lines down.

This is a no-op as, in this codepath, we must have nval == NULL or
sk_CONF_VALUE_num(nval) == 0. In those cases, free and pop_free are
identical. However, variables should be freed consistently.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1351)

Skip non-existing files.

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1324

Add Hurd shared extension

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1345

RT4639: Typo when -DSSL_DEBUG

Reviewed-by: Richard Levitte <levitte@openssl.org>

Set error if EVP_CipherUpdate fails.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Deprecate X509_LU_FAIL, X509_LU_RETRY

Instead of X509_LU_FAIL, X509_LU_RETRY use 0/1 for return values.

RT#4577

Reviewed-by: Rich Salz <rsalz@openssl.org>

Use X509_LOOKUP_TYPE for lookup type consistently.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Move X509_LU_RETRY, X509_LU_FAIL

X509_LU_RETRY and X509_LU_FAIL are not X509_OBJECT types so don't include
them in the enum.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove current_method from X509_STORE_CTX

Remove current_method: it was intended as a means of retrying
lookups bit it was never used. Now that X509_verify_cert() is
a "one shot" operation it can never work as intended.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update the example in proxy_certificates.txt

Reviewed-by: Rich Salz <rsalz@openssl.org>

Discard BIO_set(BIO* bio) method

Simplify BIO init using OPENSSL_zalloc().

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1261)

zero pad DHE public key in ServerKeyExchange message for interop

Some versions of the Microsoft TLS stack have problems when the DHE public key
is encoded with fewer bytes than the DHE prime.

There's some public acknowledgement of the bug at these links:

https://connect.microsoft.com/IE/feedback/details/1253526/tls-serverkeyexchange-with-1024-dhe-may-encode-dh-y-as-127-bytes-breaking-internet-explorer-11
https://connect.microsoft.com/IE/feedback/details/1104905/wininet-calculation-of-mac-in-tls-handshake-intermittently-fails-for-dhe-rsa-key-exchange

This encoding issue also causes the same errors with 2048-bit DHE, if the
public key is encoded in fewer than 256 bytes and includes the TLS stack on
Windows Phone 8.x.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1320)

Make it possible for external code to set the certiciate proxy path length

This adds the functions X509_set_proxy_pathlen(), which sets the
internal pc path length cache for a given X509 structure, along with
X509_get_proxy_pathlen(), which retrieves it.

Along with the previously added X509_set_proxy_flag(), this provides
the tools needed to manipulate all the information cached on proxy
certificates, allowing external code to do what's necessary to have
them verified correctly by the libcrypto code.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add X509_STORE lock and unlock functions

Since there are a number of function pointers in X509_STORE that might
lead to user code, it makes sense for them to be able to lock the
store while they do their work.

Reviewed-by: Rich Salz <rsalz@openssl.org>

make update

Reviewed-by: Rich Salz <rsalz@openssl.org>

Document the X509_STORE and X509_STORE_CTX setters and getters

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add setter and getter for X509_STORE's check_policy

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add getters / setters for the X509_STORE_CTX and X509_STORE functions

We only add setters for X509_STORE function pointers except for the
verify callback function.  The thought is that the function pointers
in X509_STORE_CTX are a cache for the X509_STORE functions.
Therefore, it's preferable if the user makes the changes in X509_STORE
before X509_STORE_CTX_init is called, and otherwise use the verify
callback to override any results from OpenSSL's internal
calculations.

Reviewed-by: Rich Salz <rsalz@openssl.org>

RT4627: Doc patch: fix constant names

Reviewed-by: Matt Caswell <matt@openssl.org>
Signed-off-by: Rich Salz <rsalz@openssl.org>

explicit init

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Typo and comment fix

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Unused variable, and cleanups

Break two long messages.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Unused variable cleanup

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Discard a dead option

Old inactive inherited code, a code relic for sure.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Discard some unused typedefs

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Constify X509|X509_CRL|X509_REVOKED_get_ext

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify ... X509|X509_CRL|X509_REVOKED|_get_ext*()

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify ...

X509_REVOKED_get0_extensions
X509_check_private_key

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify (X509|X509V3|X509_CRL|X509_REVOKED)_get_ext_d2i ...

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify i2s_ASN1_INTEGER, X509V3_get_d2i

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify input parameters of methods :

 - X509_NAME_entry_count, X509_ATTRIBUTE_count
 - X509_NAME_add_entry_by_OBJ, X509_NAME_ENTRY_create_by_OBJ, X509_NAME_ENTRY_set_object

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Enforce and explicit some const casting

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify i2t_ASN1_OBJECT, i2d_ASN1_OBJECT, i2a_ASN1_OBJECT.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify ASN1_buf_print

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify ASN1_TYPE_get, ASN1_STRING_type, ASN1_STRING_to_UTF8, ASN1_TYPE_get_octetstring & co...

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify SXNET_add_id_*

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify EC_KEY_*_oct2priv() input buffer

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify CMS_decrypt_set1_key input buffer

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify engine/eng_cnf.c internal method.

simplify and reindent some related code.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Fix no-tls1_2

Misc fixes impacting no-tls1_2. Also fixes no-dtls1_2.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix no-dtls*

Also fixes some other options like no-dgram and no-sock.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix no-ct

Ensure that we don't build/run the ct fuzzing code if no-ct is used.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add EVP_ENCODE_CTX_copy

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1344)

Add missing X509_set_proxy_flag num

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1343)

Properly initialise the internal proxy certificate path length cache

Reviewed-by: Rich Salz <rsalz@openssl.org>

Make it possible for external code to flag a certificate as a proxy one.

This adds the function X509_set_proxy_flag(), which sets the internal flag
EXFLAG_PROXY on a given X509 structure.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Correct misspelt OPENSSL_NO_SRP

RT#4619

Reviewed-by: Rich Salz <rsalz@openssl.org>

Use newest CRL.

If two CRLs are equivalent then use the one with a later lastUpdate field:
this will result in the newest CRL available being used.

RT#4615

Reviewed-by: Rich Salz <rsalz@openssl.org>

Send alert for bad DH CKE

RT#4511

Reviewed-by: Matt Caswell <matt@openssl.org>

Fix OOB read in TS_OBJ_print_bio().

TS_OBJ_print_bio() misuses OBJ_txt2obj: it should print the result
as a null terminated buffer. The length value returned is the total
length the complete text reprsentation would need not the amount of
data written.

CVE-2016-2180

Thanks to Shi Lei for reporting this bug.

Reviewed-by: Matt Caswell <matt@openssl.org>

SSL tests: compress generated output a little

Don't emit duplicate server/client sections when they are
identical. Instead, just point to the same section.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Make boolean SSL test conf values case-insensitive

Undo review mistake: I changed the wrong strcmp in a previous pull
request. Add test.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Clarify digest change in HMAC_Init_ex()

RT#4603

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add mask for newly created symlink.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Check suffixes properly.

Reviewed-by: Rich Salz <rsalz@openssl.org>

use correct name for duplicate

Reviewed-by: Rich Salz <rsalz@openssl.org>

Have load_buildtin_compression in ssl/ssl_ciph.c return RUN_ONCE result

Reviewed-by: Rich Salz <rsalz@openssl.org>

VMS: Rearrange installation targets for shared libraries

The way it was implemented before this change, the shared libraries
were installed twice.  On a file system that supports file
generations, that's a waste.  Slightly rearranging the install targets
solves the problem.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Test client-side resumption

Add tests for resuming with a different client version.

This happens in reality when clients persist sessions on disk through
upgrades.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add all publicly avaiable asn1 types to the asn1 fuzzer.

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1331

Cast to an unsigned type before negating

llvm's ubsan reported:
runtime error: negation of -9223372036854775808 cannot be represented in type
'long'; cast to an unsigned type to negate this value to itself

Found using afl

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1325