Deprecate X509_LU_FAIL, X509_LU_RETRY

Instead of X509_LU_FAIL, X509_LU_RETRY use 0/1 for return values.

RT#4577

Reviewed-by: Rich Salz <rsalz@openssl.org>

Use X509_LOOKUP_TYPE for lookup type consistently.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Move X509_LU_RETRY, X509_LU_FAIL

X509_LU_RETRY and X509_LU_FAIL are not X509_OBJECT types so don't include
them in the enum.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove current_method from X509_STORE_CTX

Remove current_method: it was intended as a means of retrying
lookups bit it was never used. Now that X509_verify_cert() is
a "one shot" operation it can never work as intended.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update the example in proxy_certificates.txt

Reviewed-by: Rich Salz <rsalz@openssl.org>

Discard BIO_set(BIO* bio) method

Simplify BIO init using OPENSSL_zalloc().

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1261)

zero pad DHE public key in ServerKeyExchange message for interop

Some versions of the Microsoft TLS stack have problems when the DHE public key
is encoded with fewer bytes than the DHE prime.

There's some public acknowledgement of the bug at these links:

https://connect.microsoft.com/IE/feedback/details/1253526/tls-serverkeyexchange-with-1024-dhe-may-encode-dh-y-as-127-bytes-breaking-internet-explorer-11
https://connect.microsoft.com/IE/feedback/details/1104905/wininet-calculation-of-mac-in-tls-handshake-intermittently-fails-for-dhe-rsa-key-exchange

This encoding issue also causes the same errors with 2048-bit DHE, if the
public key is encoded in fewer than 256 bytes and includes the TLS stack on
Windows Phone 8.x.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1320)

Make it possible for external code to set the certiciate proxy path length

This adds the functions X509_set_proxy_pathlen(), which sets the
internal pc path length cache for a given X509 structure, along with
X509_get_proxy_pathlen(), which retrieves it.

Along with the previously added X509_set_proxy_flag(), this provides
the tools needed to manipulate all the information cached on proxy
certificates, allowing external code to do what's necessary to have
them verified correctly by the libcrypto code.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add X509_STORE lock and unlock functions

Since there are a number of function pointers in X509_STORE that might
lead to user code, it makes sense for them to be able to lock the
store while they do their work.

Reviewed-by: Rich Salz <rsalz@openssl.org>

make update

Reviewed-by: Rich Salz <rsalz@openssl.org>

Document the X509_STORE and X509_STORE_CTX setters and getters

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add setter and getter for X509_STORE's check_policy

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add getters / setters for the X509_STORE_CTX and X509_STORE functions

We only add setters for X509_STORE function pointers except for the
verify callback function.  The thought is that the function pointers
in X509_STORE_CTX are a cache for the X509_STORE functions.
Therefore, it's preferable if the user makes the changes in X509_STORE
before X509_STORE_CTX_init is called, and otherwise use the verify
callback to override any results from OpenSSL's internal
calculations.

Reviewed-by: Rich Salz <rsalz@openssl.org>

RT4627: Doc patch: fix constant names

Reviewed-by: Matt Caswell <matt@openssl.org>
Signed-off-by: Rich Salz <rsalz@openssl.org>

explicit init

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Typo and comment fix

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Unused variable, and cleanups

Break two long messages.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Unused variable cleanup

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Discard a dead option

Old inactive inherited code, a code relic for sure.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Discard some unused typedefs

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Constify X509|X509_CRL|X509_REVOKED_get_ext

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify ... X509|X509_CRL|X509_REVOKED|_get_ext*()

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify ...

X509_REVOKED_get0_extensions
X509_check_private_key

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify (X509|X509V3|X509_CRL|X509_REVOKED)_get_ext_d2i ...

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify i2s_ASN1_INTEGER, X509V3_get_d2i

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify input parameters of methods :

 - X509_NAME_entry_count, X509_ATTRIBUTE_count
 - X509_NAME_add_entry_by_OBJ, X509_NAME_ENTRY_create_by_OBJ, X509_NAME_ENTRY_set_object

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Enforce and explicit some const casting

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify i2t_ASN1_OBJECT, i2d_ASN1_OBJECT, i2a_ASN1_OBJECT.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify ASN1_buf_print

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify ASN1_TYPE_get, ASN1_STRING_type, ASN1_STRING_to_UTF8, ASN1_TYPE_get_octetstring & co...

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify SXNET_add_id_*

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify EC_KEY_*_oct2priv() input buffer

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify CMS_decrypt_set1_key input buffer

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Constify engine/eng_cnf.c internal method.

simplify and reindent some related code.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Fix no-tls1_2

Misc fixes impacting no-tls1_2. Also fixes no-dtls1_2.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix no-dtls*

Also fixes some other options like no-dgram and no-sock.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix no-ct

Ensure that we don't build/run the ct fuzzing code if no-ct is used.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add EVP_ENCODE_CTX_copy

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1344)

Add missing X509_set_proxy_flag num

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1343)

Properly initialise the internal proxy certificate path length cache

Reviewed-by: Rich Salz <rsalz@openssl.org>

Make it possible for external code to flag a certificate as a proxy one.

This adds the function X509_set_proxy_flag(), which sets the internal flag
EXFLAG_PROXY on a given X509 structure.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Correct misspelt OPENSSL_NO_SRP

RT#4619

Reviewed-by: Rich Salz <rsalz@openssl.org>

Use newest CRL.

If two CRLs are equivalent then use the one with a later lastUpdate field:
this will result in the newest CRL available being used.

RT#4615

Reviewed-by: Rich Salz <rsalz@openssl.org>

Send alert for bad DH CKE

RT#4511

Reviewed-by: Matt Caswell <matt@openssl.org>

Fix OOB read in TS_OBJ_print_bio().

TS_OBJ_print_bio() misuses OBJ_txt2obj: it should print the result
as a null terminated buffer. The length value returned is the total
length the complete text reprsentation would need not the amount of
data written.

CVE-2016-2180

Thanks to Shi Lei for reporting this bug.

Reviewed-by: Matt Caswell <matt@openssl.org>

SSL tests: compress generated output a little

Don't emit duplicate server/client sections when they are
identical. Instead, just point to the same section.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Make boolean SSL test conf values case-insensitive

Undo review mistake: I changed the wrong strcmp in a previous pull
request. Add test.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Clarify digest change in HMAC_Init_ex()

RT#4603

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add mask for newly created symlink.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Check suffixes properly.

Reviewed-by: Rich Salz <rsalz@openssl.org>

use correct name for duplicate

Reviewed-by: Rich Salz <rsalz@openssl.org>

Have load_buildtin_compression in ssl/ssl_ciph.c return RUN_ONCE result

Reviewed-by: Rich Salz <rsalz@openssl.org>

VMS: Rearrange installation targets for shared libraries

The way it was implemented before this change, the shared libraries
were installed twice.  On a file system that supports file
generations, that's a waste.  Slightly rearranging the install targets
solves the problem.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Test client-side resumption

Add tests for resuming with a different client version.

This happens in reality when clients persist sessions on disk through
upgrades.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add all publicly avaiable asn1 types to the asn1 fuzzer.

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1331

Cast to an unsigned type before negating

llvm's ubsan reported:
runtime error: negation of -9223372036854775808 cannot be represented in type
'long'; cast to an unsigned type to negate this value to itself

Found using afl

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1325

Check for errors allocating the error strings.

Reviewed-by: Richard Levitte <levitte@openssl.org>
GH: #1330

Don't allocate r/s in DSA_SIG and ECDSA_SIG

To avoid having to immediately free up r/s when setting them
don't allocate them automatically in DSA_SIG_new() and ECDSA_SIG_new().

RT#4590

Reviewed-by: Richard Levitte <levitte@openssl.org>

Install shared libraries in runtime install

On non-Windows platforms, shared libraries are both development and
runtime files.  We only installed them as development files, this
makes sure they get installed as runtime files as well.

Reviewed-by: Rich Salz <rsalz@openssl.org>

VMS: fix typo, shared libraries have the extension .EXE, not .OLB

Reviewed-by: Rich Salz <rsalz@openssl.org>

Never expose ssl->bbio in the public API.

This is adapted from BoringSSL commit 2f87112b963.

This fixes a number of bugs where the existence of bbio was leaked in the
public API and broke things.

- SSL_get_wbio returned the bbio during the handshake. It must always return
  the BIO the consumer configured. In doing so, some internal accesses of
  SSL_get_wbio should be switched to ssl->wbio since those want to see bbio.

- The logic in SSL_set_rfd, etc. (which I doubt is quite right since
  SSL_set_bio's lifetime is unclear) would get confused once wbio got
  wrapped. Those want to compare to SSL_get_wbio.

- If SSL_set_bio was called mid-handshake, bbio would get disconnected and
  lose state. It forgets to reattach the bbio afterwards. Unfortunately,
  Conscrypt does this a lot. It just never ended up calling it at a point
  where the bbio would cause problems.

- Make more explicit the invariant that any bbio's which exist are always
  attached. Simplify a few things as part of that.

RT#4572

Reviewed-by: Richard Levitte <levitte@openssl.org>

SSL test framework: port resumption tests

Systematically test every server-side version downgrade or upgrade.

Client version upgrade or downgrade could be tested analogously but will
be done in a later change.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

fix crypto-mdebug build

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix if/for/while( in docs

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1292)

Fix a few if(, for(, while( inside code.

Fix some indentation at the same time

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1292)

Remove reduntant X509_STORE_CTX_set_verify_cb declaration

f0e0fd51fd8307f6eae64862ad9aaea113f1177a added X509_STORE_CTX_set_verify_cb
with a typedef'd argument, making the original one redundant.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Simplify buffer limit checking, and reuse BIO_snprintf returned value.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1284)

Code factorisation and simplification

Fix some code indentation

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1284)

Fix double calls to strlen

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1284)

Simplify code related to tmp_email_dn.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1284)

Use more X509_REQ_get0_pubkey & X509_get0_pubkey

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1284)

OCSP_request_add0_id() inconsistent error return

There are two failure cases for OCSP_request_add_id():
1. OCSP_ONEREQ_new() failure, where |cid| is not freed
2. sk_OCSP_ONEREQ_push() failure, where |cid| is freed

This changes makes the error behavior consistent, such that |cid| is
not freed when sk_OCSP_ONEREQ_push() fails. OpenSSL only takes
ownership of |cid| when the function succeeds.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1289)

Sanity check in ssl_get_algorithm2().

RT#4600

Reviewed-by: Rich Salz <rsalz@openssl.org>

Send alert on CKE error.

RT#4610

Reviewed-by: Rich Salz <rsalz@openssl.org>

Resolve over command syntax error which causes 'make install' to fail

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1312)

Document the slight change in CRYPTO_mem_ctrl()

Reviewed-by: Kurt Roeckx <kurt@openssl.org>

Document the slight change in ERR_get_next_error_library()

Reviewed-by: Kurt Roeckx <kurt@openssl.org>

make update

Reviewed-by: Kurt Roeckx <kurt@openssl.org>

Change all our uses of CRYPTO_THREAD_run_once to use RUN_ONCE instead

That way, we have a way to check if the init function was successful
or not.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>

Define a few internal macros for easy use of run_once functions

Because pthread_once() takes a function taking no argument and
returning nothing, and we want to be able to check if they're
successful, we define a few internal macros to get around the issue.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>

RT4593: Add space after comma (doc nits)

Update find-doc-nits to find errors in SYNOPSIS (the most common
place where they were missing).

Reviewed-by: Matt Caswell <matt@openssl.org>

Fix forgotten goto

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix building with no-cms

The new fuzzing code broke no-cms

Reviewed-by: Richard Levitte <levitte@openssl.org>

Convert the last uses of sockaddr in apps/* to use BIO_ADDR instead

Reviewed-by: Matt Caswell <matt@openssl.org>

Fix two bugs in clienthello processing

- Always process ALPN (previously there was an early return in the
  certificate status handling)
- Don't send a duplicate alert. Previously, both
  ssl_check_clienthello_tlsext_late and its caller would send an
  alert. Consolidate alert sending code in the caller.

Reviewed-by: Rich Salz <rsalz@openssl.org>

SSL test framework: port NPN and ALPN tests

Reviewed-by: Rich Salz <rsalz@openssl.org>

Cleanup after sk_push fail

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1281)

Update error codes following tls_process_key_exchange() refactor

Reviewed-by: Richard Levitte <levitte@openssl.org>

Tidy up tls_process_key_exchange()

After the refactor of tls_process_key_exchange(), this commit tidies up
some loose ends.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Split out ECDHE from tls_process_key_exchange()

Continuing from the previous commit. Refactor tls_process_key_exchange() to
split out into a separate function the ECDHE aspects.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Split out DHE from tls_process_key_exchange()

Continuing from the previous commit. Refactor tls_process_key_exchange() to
split out into a separate function the DHE aspects.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Split out SRP from tls_process_key_exchange()

Continuing from the previous commit. Refactor tls_process_key_exchange() to
split out into a separate function the SRP aspects.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Split out the PSK preamble from tls_process_key_exchange()

The tls_process_key_exchange() function is too long. This commit starts
the process of splitting it up by moving the PSK preamble code to a
separate function.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Move the PSK preamble for tls_process_key_exchange()

The function tls_process_key_exchange() is too long. This commit moves
the PSK preamble processing out to a separate function.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Narrow scope of locals vars in tls_process_key_exchange()

Narrow the scope of the local vars in preparation for split up this
function.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add more session tests

Add some more tests for sessions following on from the previous commit
to ensure the callbacks are called when appropriate.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove sessions from external cache, even if internal cache not used.

If the SSL_SESS_CACHE_NO_INTERNAL_STORE cache mode is used then we weren't
removing sessions from the external cache, e.g. if an alert occurs the
session is supposed to be automatically removed.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Have the Travis builds do a "make update"

Reviewed-by: Rich Salz <rsalz@openssl.org>

make update

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fixup a few SSLerr calls in ssl/statem/

Reviewed-by: Rich Salz <rsalz@openssl.org>