Add a test for BIO_read() returning 0 in SSL_read() (and also for write)

A BIO_read() 0 return indicates that a failure occurred that may be
retryable. An SSL_read() 0 return indicates a non-retryable failure. Check
that if BIO_read() returns 0, SSL_read() returns <0. Same for SSL_write().

The asyncio test filter BIO already returns 0 on a retryable failure so we
build on that.

Reviewed-by: Richard Levitte <levitte@openssl.org>

A zero return from BIO_read()/BIO_write() could be retryable

A zero return from BIO_read()/BIO_write() could mean that an IO operation
is retryable. A zero return from SSL_read()/SSL_write() means that the
connection has been closed down (either cleanly or not). Therefore we
should not propagate a zero return value from BIO_read()/BIO_write() back
up the stack to SSL_read()/SSL_write(). This could result in a retryable
failure being treated as fatal.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Provide a cross-platform format specifier (OSSLzu) for printing size_t

Reviewed-by: Richard Levitte <levitte@openssl.org>

Move manpages to man[1357] structure.

Move manpages to manX directories
Add Windows/VMS install fix from Richard Levitte
Update README
Fix typo's
Remove some duplicates

Reviewed-by: Richard Levitte <levitte@openssl.org>

VMS: tell the C compiler to use the ISO C94 standard

The current version of the VMS compiler provides C99 features,
strictly language wise.  Unfortunately, even the most recent standard
library isn't fully updated for that standard, so we need to use an
earlier standard that the compiler supports.

Most importantly, this affects the __STDC_VERSION__ value, which the
compiler unfortunately currently defaults to 199901L.  With this
change we won't have to give VMS special treatment when looking for
features based on that macro.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1785)

Fix typo (reported by Matthias St. Pierre)

Reviewed-by: Tim Hudson <tjh@openssl.org>

Backdated note in CHANGES about shared library names

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1767)

Add some notes on shared library names on different platforms

This is overdue since the addition of the unified build system

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1767)

ppccap.c: engage new multipplication and squaring subroutines.

[And remove FPU mutiplication subroutine.]

Reviewed-by: Rich Salz <rsalz@openssl.org>

bn/asm/ppc-mont.pl: add optimized multiplication and squaring subroutines.

Reviewed-by: Rich Salz <rsalz@openssl.org>

bn/asm/ppc-mont.pl: prepare for extension.

Reviewed-by: Rich Salz <rsalz@openssl.org>

sha/asm/sha512-armv8.pl: adapt for kernel use.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add NPN to an SSL3 build

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1629)

x86_64 assembly pack: add Goldmont performance results.

Reviewed-by: Richard Levitte <levitte@openssl.org>

sha/keccak1600.c: add known answer and verify result with memcmp.

Reviewed-by: Richard Levitte <levitte@openssl.org>

crypto/sha: add Keccak1600 primitives to build SHA-3 upon.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add a github pull request template

Reviewed-by: Richard Levitte <levitte@openssl.org>
GH: #1770

Fix support for DragonFly BSD

The __DragonFly__ macros were introduced in issue #1546 along with a
function naming fix, but it was decided they should be handled
separately.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1765)

Efence is antiquated, remove all traces of using it

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1768)

Correct installation test in appveyor

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1759)

Fix grammar error in SSL_CTX_set_min_proto_version

CLA: trivial
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1762)

GH1546: Fix old names in cryptodev code.

Add DragonFly version of BSD.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1584)

apps: remove some #ifndef clutter

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1643)

apps: instead of varying implementation, make setup_engine a function always

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1643)

If an engine comes up explicitely, it must also come down explicitely

In apps/apps.c, one can set up an engine with setup_engine().
However, we freed the structural reference immediately, which means
that for engines that don't already have a structural reference
somewhere else (because it's a built in engine), we end up returning
an invalid reference.

Instead, the function release_engine() is added, and called at the end
of the routines that call setup_engine().

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1643)

OpenSSL::Test - small fixup

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1686)

OpenSSL::Test cleanup - no forward declarations needed

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1686)

appveyor: make tests verbose

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1686)

Add documentation of internal OpenSSL::Test functions

Also, fix __wrap_cmd so it doesn't return unnecessary empty strings

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1686)

Make OpenSSL::Test a bit more flexible

So far, apps and test programs, were a bit rigidely accessible as
executables or perl scripts.  But what about scripts in some other
language?  Or what about running entirely external programs?  The
answer is certainly not to add new functions to access scripts for
each language or wrapping all the external program calls in our magic!

Instead, this adds a new functions, cmd(), which is useful to access
executables and scripts in a more generalised manner.  app(), test(),
fuzz(), perlapp() and perltest() are rewritten in terms of cmd(), and
serve as examples how to do something similar for other scripting
languages, or constrain the programs to certain directories.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1686)

Fix broken link to ASYNC_get_wait_ctx and rewrap the paragraph

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1745)

Do not set load_crypto_strings_inited when OPENSSL_NO_ERR is defined

Only set the load_crypto_strings_inited to 1 when err_load_crypto_strings_int was called.

This solves the following issue:
- openssl is built with no-err
- load_crypto_strings_inited is set to 1 during the OPENSSL_init_crypto call
- During the cleanup: OPENSSL_cleanup, err_free_strings_int is called because load_crypto_strings_inited == 1
- err_free_strings_int calls do_err_strings_init because it has never been called
- Now do_err_strings_init calls OPENSSL_init_crypto
- But since we are in the cleanup (stopped == 1) this results in an error:
  CRYPTOerr(CRYPTO_F_OPENSSL_INIT_CRYPTO, ERR_R_INIT_FAIL);
- which then tries to initialize everything we are trying to clean up: ERR_get_state, ossl_init_thread_start, etc
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1654)

Add error checking, small nit on ouput

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1632)

Fix config option 'no-deprecated'

crypto/asn1/asn1_item_list.c needed including dh.h and rsa.h directly.
The reason is that they are not included by x509.h when configured
'no-deprecated'

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1741)

Add missing .pod extension to EVP_PKEY_CTX_set_tls1_prf_md

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix not-c code

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1742)

Fix strict-warnings build

crypto/s390xcap.c: internal/cryptlib.h needs to be included for
OPENSSL_cpuid_setup function prototype is located there to avoid
build error due to -Werror=missing-prototypes.

Signed-off-by: Patrick Steuer <psteuer@mail.de>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
CLA: trivial

Fix strict-warnings build

crypto/evp/e_aes.c: Types of inp and out parameters of
AES_xts_en/decrypt functions need to be changed from char to
unsigned char to avoid build error due to
'-Werror=incompatible-pointer-types'.

crypto/aes/asm/aes-s390x.pl: Comments need to reflect the above
change.

Signed-off-by: Patrick Steuer <psteuer@mail.de>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
CLA: trivial

Fix strict-warnings build

crypto/asn1/a_strex.c: Type of width variable in asn1_valid_host
function  needs to be changed from char to signed char to avoid
build error due to '-Werror=type-limits'.

Signed-off-by: Patrick Steuer <psteuer@mail.de>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
CLA: trivial

Add SSL_OP_NO_ENCRYPT_THEN_MAC

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix encrypt-then-mac implementation for DTLS

OpenSSL 1.1.0 will negotiate EtM on DTLS but will then not actually *do* it.

If we use DTLSv1.2 that will hopefully be harmless since we'll tend to use
an AEAD ciphersuite anyway. But if we're using DTLSv1, then we certainly
will end up using CBC, so EtM is relevant — and we fail to interoperate with
anything that implements EtM correctly.

Fixing it in HEAD and 1.1.0c will mean that 1.1.0[ab] are incompatible with
1.1.0c+... for the limited case of non-AEAD ciphers, where they're *already*
incompatible with other implementations due to this bug anyway. That seems
reasonable enough, so let's do it. The only alternative is just to turn it
off for ever... which *still* leaves 1.0.0[ab] failing to communicate with
non-OpenSSL implementations anyway.

Tested against itself as well as against GnuTLS both with and without EtM.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Add Postgres support to -starttls

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>

Fix embedded string handling.

Don't rely on embedded flag to free strings correctly: it wont be
set if there is a malloc failure during initialisation.

Thanks to Guido Vranken for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1725)

fix invalid use of incomplete type X509_STORE_CTX

CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Allow older versions in the *.num files

In 1.1.0 we only allowed a strictly increasing version number in the *.num
files, i.e. you could never introduce a symbol at the end of the *.num file
with a lower version number than the one preceding it. This made sense for
1.1.0. However in master we may be introducing symbols for backport to
1.1.0. Therefore it is ok in master to have a symbol for version 1.1.0c
coming after a symbol for version 1.1.1.

This commit fixes the check in mkdef.pl to be a bit looser to allow this.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix signatures of EVP_Digest{Sign,Verify}Update

These are implemented as macros delegating to `EVP_DigestUpdate`, which
takes a `size_t` as its third argument, not an `unsigned int`.

CLA: trivial

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Ensure we handle len == 0 in ERR_err_string_n

If len == 0 in a call to ERR_error_string_n() then we can read beyond the
end of the buffer. Really applications should not be calling this function
with len == 0, but we shouldn't be letting it through either!

Thanks to Agostino Sarubbo for reporting this issue. Agostino's blog on
this issue is available here:
https://blogs.gentoo.org/ago/2016/10/14/openssl-libcrypto-stack-based-buffer-overflow-in-err_error_string_n-err-c/

Reviewed-by: Richard Levitte <levitte@openssl.org>

Use clang++ for C++ for the linux-x86_64-clang target

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Engine afalg: properly set operation type also on big endian.

Copy the whole ALG_OP_TYPE to CMSG_DATA.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix style issue

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1694)

Constify command options

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1694)

Fix C++ support: set $target{cxx} correctly

Reviewed-by: Matt Caswell <matt@openssl.org>

Fix typo

I think the second "VC-WIN32" should be "VC-WIN64".
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
CLA: trivial

Configure: remove superfluous 0x

The number is taken from the OPENSSL_VERSION_NUMBER which is already
in the hex form.

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1706)

Add memory leak detection to d2i_test

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1707)

Fix copy-paste test labels

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove automatic RPATH - Add a CHANGES entry

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove automatic RPATH - adapt shlib_wrap.sh

Looking for something starting with '-Wl,-rpath,' isn't good enough,
as someone might give something like '-Wl,--enable-new-dtags,-rpath,/PATH'.
Looking for ',-rpath,' should be safe enough.

We could remove the preloading stuff entirely, but just in case the
user has chosen to given RPATH setting arguments at configuration,
we'd better make sure testing will still work.  Fair warning, there
are some configuration options that do not work with preloaded OpenSSL
libraries, such as the sanity checking ones.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove automatic RPATH - add user rpath support

Make Configure recognise -rpath and -R to support user added rpaths
for OSF1 and Solaris.  For convenience, add a variable LIBRPATH in the
Unix Makefile, which the users can use as follows:

    ./config [options] -Wl,-rpath,\$(LIBRPATH)

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove automatic RPATH

Before OpenSSL 1.1.0, binaries were installed in a non-standard
location by default, and runpath directories were therefore added in
those binaries, to make sure the executables would be able to find the
shared libraries they were linked with.

With OpenSSL 1.1.0 and on, binaries are installed in standard
directories by default, and the addition of runpath directories is
therefore not needed any more.

Reviewed-by: Rich Salz <rsalz@openssl.org>

RT is put out to pasture

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1702)

Add C++ settings in the Linux config targets

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add support for C++ in Configurations/unix-Makefile.tmpl

Note that it relies on a trick from Configure, where file names for
object files made from C++ source get '.cc' replaced with '_cc.o' to
recognise them.  This is needed so the correct compiler is used when
linking binaries.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add support for C++ in Configure

A note: this will form object file names by changing '.cc' to
'_cc.o'.  This will permit other configuration code to recognise these
object files were built for C++ rather than C.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix X509_NAME decode for malloc failures.

The original X509_NAME decode free code was buggy: this
could result in double free or leaks if a malloc failure
occurred.

Simplify and fix the logic.

Thanks to Guido Vranken for reporting this issue.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1691)

doc/crypto/OPENSSL_ia32cap.pod: update assembler requirements.

Reviewed-by: Richard Levitte <levitte@openssl.org>

.travis.yml: minor optimizations:

- make 'pip install --user cpp-coveralls' conditional;
- limit no-stdio to single build per operating environment;
- omit enable-asan duplicate;

Reviewed-by: Richard Levitte <levitte@openssl.org>

.travis.yml: install wine on demand.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix up bn_prime.pl formatting.

Align at 5 characters, not 4. There are 5-digit numbers in the output.
Also avoid emitting an extra blank line and trailing whitespace.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Remove trailing whitespace from some files.

The prevailing style seems to not have trailing whitespace, but a few
lines do. This is mostly in the perlasm files, but a few C files got
them after the reformat. This is the result of:

  find . -name '*.pl' | xargs sed -E -i '' -e 's/( |'$'\t'')*$//'
  find . -name '*.c' | xargs sed -E -i '' -e 's/( |'$'\t'')*$//'
  find . -name '*.h' | xargs sed -E -i '' -e 's/( |'$'\t'')*$//'

Then bn_prime.h was excluded since this is a generated file.

Note mkerr.pl has some changes in a heredoc for some help output, but
other lines there lack trailing whitespace too.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Add some missing types to indent.pro

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Update fuzz corpora

New minimal fuzz corpora for asn1, asn1parse, bndiv, crl and x509

Reviewed-by: Andy Polyakov <appro@openssl.org>
GH: #1678

Fix linebreaks in the tls_construct_client_certificate function

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add a typedef for the construction function

Reviewed-by: Rich Salz <rsalz@openssl.org>

Move setting of the handshake header up one more level

We now set the handshake header, and close the packet directly in the
write_state_machine. This is now possible because it is common for all
messages.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove the special case processing for finished construction

tls_construct_finished() used to have different arguments to all of the
other construction functions. It doesn't anymore, so there is no neeed to
treat it as a special case.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Harmonise setting the header and closing construction

Ensure all message types work the same way including CCS so that the state
machine doesn't need to know about special cases. Put all the special logic
into ssl_set_handshake_header() and ssl_close_construct_packet().

Reviewed-by: Rich Salz <rsalz@openssl.org>

Don't set the handshake header in every message

Move setting the handshake header up a level into the state machine code
in order to reduce boilerplate.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Move init of the WPACKET into write_state_machine()

Instead of initialising, finishing and cleaning up the WPACKET in every
message construction function, we should do it once in
write_state_machine().

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove untrue comment.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Make dependencies if Makefile is new.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Rename ssl_set_handshake_header2()

ssl_set_handshake_header2() was only ever a temporary name while we had
to have ssl_set_handshake_header() for code that hadn't been converted to
WPACKET yet. No code remains that needed that so we can rename it.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove ssl_set_handshake_header()

Remove the old ssl_set_handshake_header() implementations. Later we will
rename ssl_set_handshake_header2() to ssl_set_handshake_header().

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove the tls12_get_sigandhash_old() function

This is no longer needed now that all messages use WPACKET

Reviewed-by: Rich Salz <rsalz@openssl.org>

fix memory leak

Reviewed-by: Rich Salz <rsalz@openssl.org>

Don't use DES when disabled.

Reviewed-by: Rich Salz <rsalz@openssl.org>

fix memory leak

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add SRP test vectors from RFC5054

Reviewed-by: Rich Salz <rsalz@openssl.org>

SRP code tidy.

Tidy up srp_Calc_k and SRP_Calc_u by making them a special case of
srp_Calc_xy which performs SHA1(PAD(x) | PAD(y)).

This addresses an OCAP Audit issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert NewSessionTicket construction to WPACKET

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix an error in packet_locl.h

A convenience macro was using the wrong underlying function.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert CertStatus message construction to WPACKET

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix mis-named macro in packet_locl.h

A couple of the WPACKET_sub_memcpy* macros were mis-named.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert SeverDone construction to WPACKET

Reviewed-by: Rich Salz <rsalz@openssl.org>

make update

Reviewed-by: Rich Salz <rsalz@openssl.org>

Print <ABSENT> if a STACK is NULL.

If a STACK (corresponding to SEQUENCE OF or SET OF) is NULL then the
field is absent as opposed to empty (present but has zero elements).

Reviewed-by: Rich Salz <rsalz@openssl.org>

add item list support to d2i_test

Reviewed-by: Rich Salz <rsalz@openssl.org>

ASN1_ITEM should use type name not structure name.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add -item option to asn1parse

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add ASN1_ITEM lookup and enumerate functions.

Reviewed-by: Rich Salz <rsalz@openssl.org>