Convert x509 selftests to internal test

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1789)

Add a HEADER_MODES_H guard in include/openssl/modes.h

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1789)

Convert modes selftests (cts128 and gcm128) to internal test

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1789)

Convert asn1 selftests (a_strnid and ameth_lib) into internal test

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1789)

VMS: ignore multiply defined symbols when linking programs

The Unix and Windows linkers appear to simply ignore if any symbol is
defined multiple times in different object files and libraries.

The VMS linker, on the other hand, warns about it, loud and clear.  It
will still create the executable, but does so screaming.  So we
complicate things by saving the linker output, look through all the
errors and warnings, and if they are only made up of %LINK-W-MULDEF,
we let it pass, otherwise we output the linker output and raise the
same exit code we got from the linker.

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1789)

Explain the deal with internal test programs

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1789)

Allow indented comments in build.info

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1789)

Convert poly1305 selftest into internal test

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1789)

test/build.info: typo, $ missing

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1832)

conf fuzzer: also check for an empty file

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1828

Implement GET_MODULE_HANDLE_EX_FLAG_PIN for windows

Rather than leaking a reference, just call GetModuleHandleEx and pin the
module on Windows.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Link using -znodelete

Instead of deliberately leaking a reference to ourselves, use nodelete
which does this more neatly. Only for Linux at the moment.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add a test to dynamically load and unload the libraries

This should demonstrate that the atexit() handling is working properly (or
at least not crashing) on process exit.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Ensure that libcrypto and libssl do not unload until the process exits

Because we use atexit() to cleanup after ourselves, this will cause a
problem if we have been dynamically loaded and then unloaded again: the
atexit() handler may no longer be there.

Most modern atexit() implementations can handle this, however there are
still difficulties if libssl gets unloaded before libcrypto, because of
the atexit() callback that libcrypto makes to libssl.

The most robust solution seems to be to ensure that libcrypto and libssl
never unload. This is done by simply deliberately leaking a dlopen()
reference to them.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add a DSO_dsobyaddr() function

This works the same way as DSO_pathbyaddr() but instead returns a ptr to
the DSO that contains the provided symbol.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Partial revert of 3d8b2ec42 to add back DSO_pathbyaddr

Commit 3d8b2ec42 removed various unused functions. However now we need to
use one of them! This commit resurrects DSO_pathbyaddr(). We're not going to
resurrect the Windows version though because what we need to achieve can be
done a different way on Windows.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add a CHANGES entry for the unrecognised record type change

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add a test for unrecognised record types

We should fail if we receive an unrecognised record type

Reviewed-by: Tim Hudson <tjh@openssl.org>

Fail if an unrecognised record type is received

TLS1.0 and TLS1.1 say you SHOULD ignore unrecognised record types, but
TLS 1.2 says you MUST send an unexpected message alert. We swap to the
TLS 1.2 behaviour for all protocol versions to prevent issues where no
progress is being made and the peer continually sends unrecognised record
types, using up resources processing them.

Issue reported by 郭志攀

Reviewed-by: Tim Hudson <tjh@openssl.org>

Test recipes: remove duplicate OpenSSL::Test usage

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1815)

Fix link to LibFuzzer

CLA: trivial
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1801)

Add a read_ahead test

This test checks that read_ahead works correctly when dealing with large
records.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix read_ahead

The function ssl3_read_n() takes a parameter |clearold| which, if set,
causes any old data in the read buffer to be forgotten, and any unread data
to be moved to the start of the buffer. This is supposed to happen when we
first read the record header.

However, the data move was only taking place if there was not already
sufficient data in the buffer to satisfy the request. If read_ahead is set
then the record header could be in the buffer already from when we read the
preceding record. So with read_ahead we can get into a situation where even
though |clearold| is set, the data does not get moved to the start of the
read buffer when we read the record header. This means there is insufficient
room in the read buffer to consume the rest of the record body, resulting in
an internal error.

This commit moves the |clearold| processing to earlier in ssl3_read_n()
to ensure that it always takes place.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add documentation for DTLS_get_data_mtu()

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Add test cases for DTLS_get_data_mtu()

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Add unit test for ssl_cipher_get_overhead()

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Add DTLS_get_data_mtu() function

We add ssl_cipher_get_overhead() as an internal function, to avoid
having too much ciphersuite-specific knowledge in DTLS_get_data_mtu()
itself. It's going to need adjustment for TLSv1.3... but then again, so
is fairly much *all* of the SSL_CIPHER handling. This bit is in the noise.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix some style issues in ossltest

Based on feedback received

Reviewed-by: Rich Salz <rsalz@openssl.org>

Enable TLSProxy to talk TLS1.3

Now that ossltest knows about a TLS1.3 cipher we can now do TLS1.3 in
TLSProxy

Reviewed-by: Rich Salz <rsalz@openssl.org>

Make sure ossltest engine works with TLS1.3

This might need more changes once we do a "real" TLS1.3 ciphersuite. But it
should do for now.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert a big "if" into a "switch"

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update Configure to know about tls1_3

Also we disable TLS1.3 by default (use enable-tls1_3 to re-enable). This is
because this is a WIP and will not be interoperable with any other TLS1.3
implementation.

Finally, we fix some tests that started failing when TLS1.3 was disabled by
default.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Use ciphersuite id when matching if we've got one

When matching a ciphersuite if we are given an id, make sure we use it
otherwise we will match another ciphersuite which is identical except for
the TLS version.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add the SSL_METHOD for TLSv1.3 and all other base changes required

Includes addition of the various options to s_server/s_client. Also adds
one of the new TLS1.3 ciphersuites.

This isn't "real" TLS1.3!! It's identical to TLS1.2 apart from the protocol
and the ciphersuite...and the ciphersuite is just a renamed TLS1.2 one (not
a "real" TLS1.3 ciphersuite).

Reviewed-by: Rich Salz <rsalz@openssl.org>

TEST_check macro: don't end with semi

Reviewed-by: Rich Salz <rsalz@openssl.org>

Secure our notification email.

Forks will have to define their own

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1821)

Add SSL_CTX_set1_cert_store()

For convenience, combine getting a new ref for the new SSL_CTX
with assigning the store and freeing the old one.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1755)

[fuzzers] do not fail fuzzers with empty input

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1788

Fix grammar-o in CONTRIBUTING

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1625)

Revert "Disable MDC2 by default."

This reverts commit ca1574cec20589885000d039eed3a9375fb29a0d.
Not suitabled for a minor release as it breaks the ABI.

Reviewed-by: Richard Levitte <levitte@openssl.org>

HPUX: Add the forgotten $(DSTDIR) when linking DSOs

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1802)

Unix Makefile: Some sed implementation truncate long lines.  Use perl instead.

Fixes #1781

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1813)

sha/keccak1600.c: add couple of soft asserts.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Disable MDC2 by default.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Mark VC templates correctly.

VC-noCE-common and VC-WIN64-common were missing this line:

        template         => 1,

Fixes GH#1809

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1811)

Try to unify BIO read/write parameter names

After the recent reworking, not everything matched up, and some
comments didn't catch up to the outl-->dlen and inl-->dlen renames
that happened during the development of the recent patches.

Try to make parameter names consistent across header, implementation,
and manual pages.

Also remove some trailing whitespace that was inadvertently introduced.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1798)

Wordsmith INSTALL

Make it clear that the OPENSSL_LOCAL_CONFIG_DIR settings take
precedence over the in-tree configs.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1798)

Fix stdio build following BIO size_t work

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix more shadowed variable warnings

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix some feedback issues for BIO size_t-ify

Rename some parameters; add some error codes; fix a comment; etc

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add some sanity checks for BIO_read* and BIO_gets

Make sure the return value isn't bigger than the buffer len

Reviewed-by: Richard Levitte <levitte@openssl.org>

More parameter naming of BIO_read*/BIO_write* related functions

Based on feedback received.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Tweaks based on review feedback of BIO size_t work

Rename some parameters.
Also change handling of buffer sizes >INT_MAX in length.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Ensure that BIO_read_ex() and BIO_write_ex() only return 0 or 1

They should return 0 for a failure (retryable or not), and 1 for a success.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Read up to INT_MAX when calling legacy BIO_read() implementations

In converting a new style BIO_read() call into an old one, read
as much data as we can (INT_MAX), if the size of the buffer is
>INT_MAX.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix a shadowed variable declaration warning

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix some bogus uninit variable warnings

Reviewed-by: Richard Levitte <levitte@openssl.org>

Document the new BIO functions introduced as part of the size_t work

Reviewed-by: Richard Levitte <levitte@openssl.org>

Ensure all BIO functions call the new style callback

Reviewed-by: Richard Levitte <levitte@openssl.org>

Create BIO_write_ex() which handles size_t arguments

Also extend BIO_METHOD to be able to supply an implementation for the new
BIO_write_ex function.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Create BIO_read_ex() which handles size_t arguments

Also extend BIO_METHOD to be able to supply an implementation for the new
BIO_read function.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix a double free in ca command line

Providing a spkac file with no default section causes a double free.

Thanks to Brian Carpenter for reporting this issue.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>

Add a test for BIO_read() returning 0 in SSL_read() (and also for write)

A BIO_read() 0 return indicates that a failure occurred that may be
retryable. An SSL_read() 0 return indicates a non-retryable failure. Check
that if BIO_read() returns 0, SSL_read() returns <0. Same for SSL_write().

The asyncio test filter BIO already returns 0 on a retryable failure so we
build on that.

Reviewed-by: Richard Levitte <levitte@openssl.org>

A zero return from BIO_read()/BIO_write() could be retryable

A zero return from BIO_read()/BIO_write() could mean that an IO operation
is retryable. A zero return from SSL_read()/SSL_write() means that the
connection has been closed down (either cleanly or not). Therefore we
should not propagate a zero return value from BIO_read()/BIO_write() back
up the stack to SSL_read()/SSL_write(). This could result in a retryable
failure being treated as fatal.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Provide a cross-platform format specifier (OSSLzu) for printing size_t

Reviewed-by: Richard Levitte <levitte@openssl.org>

Move manpages to man[1357] structure.

Move manpages to manX directories
Add Windows/VMS install fix from Richard Levitte
Update README
Fix typo's
Remove some duplicates

Reviewed-by: Richard Levitte <levitte@openssl.org>

VMS: tell the C compiler to use the ISO C94 standard

The current version of the VMS compiler provides C99 features,
strictly language wise.  Unfortunately, even the most recent standard
library isn't fully updated for that standard, so we need to use an
earlier standard that the compiler supports.

Most importantly, this affects the __STDC_VERSION__ value, which the
compiler unfortunately currently defaults to 199901L.  With this
change we won't have to give VMS special treatment when looking for
features based on that macro.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1785)

Fix typo (reported by Matthias St. Pierre)

Reviewed-by: Tim Hudson <tjh@openssl.org>

Backdated note in CHANGES about shared library names

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1767)

Add some notes on shared library names on different platforms

This is overdue since the addition of the unified build system

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1767)

ppccap.c: engage new multipplication and squaring subroutines.

[And remove FPU mutiplication subroutine.]

Reviewed-by: Rich Salz <rsalz@openssl.org>

bn/asm/ppc-mont.pl: add optimized multiplication and squaring subroutines.

Reviewed-by: Rich Salz <rsalz@openssl.org>

bn/asm/ppc-mont.pl: prepare for extension.

Reviewed-by: Rich Salz <rsalz@openssl.org>

sha/asm/sha512-armv8.pl: adapt for kernel use.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add NPN to an SSL3 build

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1629)

x86_64 assembly pack: add Goldmont performance results.

Reviewed-by: Richard Levitte <levitte@openssl.org>

sha/keccak1600.c: add known answer and verify result with memcmp.

Reviewed-by: Richard Levitte <levitte@openssl.org>

crypto/sha: add Keccak1600 primitives to build SHA-3 upon.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add a github pull request template

Reviewed-by: Richard Levitte <levitte@openssl.org>
GH: #1770

Fix support for DragonFly BSD

The __DragonFly__ macros were introduced in issue #1546 along with a
function naming fix, but it was decided they should be handled
separately.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1765)

Efence is antiquated, remove all traces of using it

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1768)

Correct installation test in appveyor

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1759)

Fix grammar error in SSL_CTX_set_min_proto_version

CLA: trivial
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1762)

GH1546: Fix old names in cryptodev code.

Add DragonFly version of BSD.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1584)

apps: remove some #ifndef clutter

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1643)

apps: instead of varying implementation, make setup_engine a function always

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1643)

If an engine comes up explicitely, it must also come down explicitely

In apps/apps.c, one can set up an engine with setup_engine().
However, we freed the structural reference immediately, which means
that for engines that don't already have a structural reference
somewhere else (because it's a built in engine), we end up returning
an invalid reference.

Instead, the function release_engine() is added, and called at the end
of the routines that call setup_engine().

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1643)

OpenSSL::Test - small fixup

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1686)

OpenSSL::Test cleanup - no forward declarations needed

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1686)

appveyor: make tests verbose

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1686)

Add documentation of internal OpenSSL::Test functions

Also, fix __wrap_cmd so it doesn't return unnecessary empty strings

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1686)

Make OpenSSL::Test a bit more flexible

So far, apps and test programs, were a bit rigidely accessible as
executables or perl scripts.  But what about scripts in some other
language?  Or what about running entirely external programs?  The
answer is certainly not to add new functions to access scripts for
each language or wrapping all the external program calls in our magic!

Instead, this adds a new functions, cmd(), which is useful to access
executables and scripts in a more generalised manner.  app(), test(),
fuzz(), perlapp() and perltest() are rewritten in terms of cmd(), and
serve as examples how to do something similar for other scripting
languages, or constrain the programs to certain directories.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1686)

Fix broken link to ASYNC_get_wait_ctx and rewrap the paragraph

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1745)

Do not set load_crypto_strings_inited when OPENSSL_NO_ERR is defined

Only set the load_crypto_strings_inited to 1 when err_load_crypto_strings_int was called.

This solves the following issue:
- openssl is built with no-err
- load_crypto_strings_inited is set to 1 during the OPENSSL_init_crypto call
- During the cleanup: OPENSSL_cleanup, err_free_strings_int is called because load_crypto_strings_inited == 1
- err_free_strings_int calls do_err_strings_init because it has never been called
- Now do_err_strings_init calls OPENSSL_init_crypto
- But since we are in the cleanup (stopped == 1) this results in an error:
  CRYPTOerr(CRYPTO_F_OPENSSL_INIT_CRYPTO, ERR_R_INIT_FAIL);
- which then tries to initialize everything we are trying to clean up: ERR_get_state, ossl_init_thread_start, etc
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1654)

Add error checking, small nit on ouput

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1632)

Fix config option 'no-deprecated'

crypto/asn1/asn1_item_list.c needed including dh.h and rsa.h directly.
The reason is that they are not included by x509.h when configured
'no-deprecated'

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1741)

Add missing .pod extension to EVP_PKEY_CTX_set_tls1_prf_md

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix not-c code

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1742)

Fix strict-warnings build

crypto/s390xcap.c: internal/cryptlib.h needs to be included for
OPENSSL_cpuid_setup function prototype is located there to avoid
build error due to -Werror=missing-prototypes.

Signed-off-by: Patrick Steuer <psteuer@mail.de>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
CLA: trivial

Fix strict-warnings build

crypto/evp/e_aes.c: Types of inp and out parameters of
AES_xts_en/decrypt functions need to be changed from char to
unsigned char to avoid build error due to
'-Werror=incompatible-pointer-types'.

crypto/aes/asm/aes-s390x.pl: Comments need to reflect the above
change.

Signed-off-by: Patrick Steuer <psteuer@mail.de>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
CLA: trivial