Remove the old threading API

All OpenSSL code has now been transferred to use the new threading API,
so the old one is no longer used and can be removed. We provide some compat
macros for removed functions which are all no-ops.

There is now no longer a need to set locking callbacks!!

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove use of the old CRYPTO_LOCK_X5O9_STORE

The locking here is a bit strange and unclear. Rather than refactor
anything and possibly break stuff I have just moved to using the new
thread API following as closely as possible what was there previously.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't call ENGINE_cleanup when configured "no-engine"

Reviewed-by: Matt Caswell <matt@openssl.org>

Don't add engines if configured "no-engine"

Similarly, don't add e_capi if configured "no-capieng"

Also, indent a little deeper, for clarity.

Reviewed-by: Matt Caswell <matt@openssl.org>

Don't add afalg engine if configured "no-engine"

Also, indent a little deeper, for clarity.

Reviewed-by: Matt Caswell <matt@openssl.org>

Don't check the conditions to build e_afalg if configured "no-engine"

Reviewed-by: Matt Caswell <matt@openssl.org>

Have Configure display the value of SHARED_CFLAG

Reviewed-by: Andy Polyakov <appro@openssl.org>

Make sure the effect of "pic" / "no-pic" is used with assembler compilations

Before the 'Introduce the "pic" / "no-pic" config option' commit, the
shared_cflag value for the chosen config would be part of the make
variable CFLAG, which got replicated into CFLAGS and ASFLAGS.

Since said commit, the shared_cflag value has become a make variable
of its own, SHARED_CFLAG (which is left empty in a "no-pic" build).

However, ASFLAGS was forgotten.  That's what's corrected with this
change.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Touch the correct variables for the system; shlib_wrap.sh on Solaris

If there is cause to think LD_LIBRARY_PATH_32 and LD_PRELOAD_32 are
appropriate variables to touch, do so.  Otherwise, touch the usual
LD_LIBRARY_PATH and LD_PRELOAD.  This covers for older installations
that don't have a mix of 32-bit and 64-bit libs.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Recognise 32-bit Solaris in util/shlib_wrap.sh

Submitted by Erik Forsberg <erik@efca.com>

Reviewed-by: Andy Polyakov <appro@openssl.org>

Check gcc version to see if it supports -MM and friends

According to manuals found here: https://gcc.gnu.org/onlinedocs/, GNU
C version 3 and on support the dependency generation options.  We
therefore need to check the gcc version to see if we're going to use
it or makedepend for dependency generation.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Add extra include directory for includers of ppc_arch.h

crypto/evp/e_aes.c and crypto/modes/gcm128.c include ppc_arch.h, which
is located in crypto/, so add that as extra include directory for them.

Issue reported by Jeffrey Walton <noloader@gmail.com>

Reviewed-by: Andy Polyakov <appro@openssl.org>

Adapt INSTALL and related notes for Windows

Reviewed-by: Rich Salz <rsalz@openssl.org>

Adapt appveyor.yml for the new unified build

Reviewed-by: Rich Salz <rsalz@openssl.org>

Adapt the Windows makefile template to source generation

Reviewed-by: Rich Salz <rsalz@openssl.org>

Don't run the TLSProxy based tests in native Windows

There are issues binding listening ports.  This may be analyzed more
thoroughly later on.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Unified - name native Windows shared libraries like MingW builds do

Reviewed-by: Rich Salz <rsalz@openssl.org>

Unified - extract settings from util/pl/VC-32.pl and make the config settings

This introduces the settings loutflag and aroutflag, because different
Windows tools that do the same thing have different ways to specify
the output file.

The Borland C++ config is commented away for the monent, perhaps
permanently.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Unified - a native Windows makefile template

Reviewed-by: Rich Salz <rsalz@openssl.org>

ec/asm/ecp_nistz256-sparcv9.pl: get corner logic right.

RT#4284

Reviewed-by: Rich Salz <rsalz@openssl.org>

Adapt mk1mf.pl and companions to changed perlasm script semantics

Reviewed-by: Matt Caswell <matt@openssl.org>

Unified - adapt the generation of whirlpool assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/whrlpool/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of sha assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/sha/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of rc4 assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/rc4/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of rc5 assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/rc5/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of ripemd assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/ripemd/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of md5 assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/md5/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of modes assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/modes/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of poly1305 assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/poly1305/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of des assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/des/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of ec assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/ec/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of camellia assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/camellia/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of cast assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/cast/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of chacha assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/chacha/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Because crypto/build.info demands CFLAGS_Q, descrip.mms.tmpl must deliver

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of cpuid, uplink and buildinf to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of aes assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/aes/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unified - adapt the generation of blowfish assembler to use GENERATE

This gets rid of the BEGINRAW..ENDRAW sections in crypto/bf/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Convert the dynlocks in e_chil to the new Thread API locks

Reviewed-by: Matt Caswell <matt@openssl.org>

Remove another lock from e_chil

Reviewed-by: Richard Levitte <levitte@openssl.org>

Always call ENGINE_cleanup() in de-init

Even if we haven't loaded an engine, we might have set up the
global_engine_lock, so we should still clean up.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Move chil engine to the new thread api

Move the chil engine to use the new thread API. As I don't have access to
the hardware I can't test this :-(. I think its ok...

Reviewed-by: Richard Levitte <levitte@openssl.org>

Move engine library over to using the new thread API

Remove usage of CRYPTO_LOCK_ENGINE

Reviewed-by: Richard Levitte <levitte@openssl.org>

err_lcl.h is gone, don't pretend it's there

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Add missing semi

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Counter mixed signedness with a cast

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Fix names of the #define used for platform specific code

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Add empty line after local variables

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix error with wait set of fds for the select()

It also makes the call to select blocking to reduce CPU usage

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Use correct function ID in error path

This fixes "make update".

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Move variable declaration to the start of the function

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Ensure CRYPTO_mem_leaks is the last thing we do

CRYPTO_mem_leaks de-inits the library, so we must not do anything
interesting after we've used it!

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix memory leak in ssltest

The new Rand usage of Thread API exposed a bug in ssltest. ssltest "cheats"
and uses internal headers to directly call functions that normally you
wouldn't be able to do. This means that auto-init doesn't happen, and
therefore auto-deinit doesn't happen either, meaning that the new rand locks
don't get cleaned up properly.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert rand code to new threading API

Replace the CRYPTO_LOCK_RAND and CRYPTO_LOCK_RAND2 locks with new thread
API style locks.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Adapt unix Makefile template to 'no-makedepend'

This change is a bit more complex, as it involves several recipe
variants.

Also, remove the $(CROSS_COMPILE) prefix for the makedepend program.
When we use the program "makedepend", this doesn't serve anything,
and when we use the compiler, this value isn't even used.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Adapt descrip.mms.tmpl to 'no-makedepend'

VMS doesn't have "makedepend" anyway, so this is just a matter of using
the right qualifiers when 'makedepend' is enabled.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add the configure option 'no-makedepend'

If no makedepend program or equaly capable compiler is present,
'makedepend' gets disabled automatically.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix SSL_CIPHER_get_auth_nid return

Copy/paste error between SSL_CIPHER_get_kx_nid() and
SSL_CIPHER_get_auth_nid(), wrong table was referenced

Signed-off-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix a typo in dynamic_load()

Reviewed-by: Matt Caswell <matt@openssl.org>

Convert RSA blinding to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert CRYPTO_LOCK_SSL_* to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove the transfer of lock hooks from bind_engine

With the new threads API, this is no longer needed.

Reviewed-by: Matt Caswell <matt@openssl.org>

Engine API repair - memory management hooks

The Engine API lost the setting of memory management hooks in
bind_engine.  Here's putting that back.

EX_DATA and ERR functions need the same treatment.

Reviewed-by: Matt Caswell <matt@openssl.org>

Redo the Unix source code generator

For assembler, we want the final target to be foo.s (lowercase s).
However, the build.info may have lines like this (note upper case S):

    GENERATE[foo.S]=foo.pl

This indicates that foo.s (lowercase s) is still to be produced, but
that producing it will take an extra step via $(CC) -E.  Therefore,
the following variants (simplified for display) can be generated:

    GENERATE[foo.S]=foo.pl  =>  foo.s: foo.pl
                                 $(PERL) $foo.pl $@.S; \
                                 $(CC) $(CFLAGS) -E -P $@.S > $@ && \
                                 rm -f $@.S

    GENERATE[foo.s]=foo.pl  =>  foo.s: foo.pl
                                 $(PERL) $foo.pl $@

    GENERATE[foo.S]=foo.m4  =>  foo.s: foo.m4
                                 m4 -B 8192 $foo.m4 > $@.S; \
                                 $(CC) $(CFLAGS) -E -P $@.S > $@ && \
                                 rm -f $@.S

    GENERATE[foo.s]=foo.m4  =>  foo.s: foo.m4
                                 m4 -B 8192 $foo.m4 > $@

Reviewed-by: Andy Polyakov <appro@openssl.org>

Fix error in ssltest

Compiling ssltest with some compilers using --strict-warnings results in
complaints about an unused result.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>

Adapt e_capi to the DSA_SIG_get0() API

Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert mem_dbg and mem_sec to the new Thread API

Use new Thread API style locks, and thread local storage for mem_dbg

Reviewed-by: Rich Salz <rsalz@openssl.org>

OpenSSLDie --> OPENSSL_die

Also removed a bunch of unused define's from e_os.h

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove the -n tar flag from osx dist creation

With the unified build scheme, tar MUST recurse, or the tarball will
be empty.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Remove some old ms/* files

Reviewed-by: Richard Levitte <levitte@openssl.org>

Include e_os.h from ec_lcl.h

The EC code recently started using REF_PRINT_COUNT and REF_ASSERT_ISNT.
Those are defined in e_os.h.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix cert leaks in s_server

Signed-off-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Reuse strndup(), simplify code

Signed-off-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

make update

Reviewed-by: Rich Salz <rsalz@openssl.org>

Make DSA_SIG opaque.

This adds a new accessor function DSA_SIG_get0.
The customisation of DSA_SIG structure initialisation has been removed this
means that the 'r' and 's' components are automatically allocated when
DSA_SIG_new() is called. Update documentation.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix travis builds

Travis doesn't seem to know about 'expr'

Reviewed-by: Emilia Käsper <emilia@openssl.org>

make update

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert CRYPTO_LOCK_X509_* to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert CRYPTO_LOCK_UI to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert CRYPTO_LOCK_EC_* to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert CRYPTO_LOCK_EVP_PKEY to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert CRYPTO_LOCK_BIO to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Only enable ccache if it's available

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Use ccache for the unified builds

This may speed up our builds considerably

Reviewed-by: Emilia Käsper <emilia@openssl.org>

For unified builds, make a separate build directory and build there

Reviewed-by: Emilia Käsper <emilia@openssl.org>

SPARCv9 assembly pack: unify build rules and argument handling.

Make all scripts produce .S, make interpretation of $(CFLAGS)
pre-processor's responsibility, start accepting $(PERLASM_SCHEME).
[$(PERLASM_SCHEME) is redundant in this case, because there are
no deviataions between Solaris and Linux assemblers. This is
purely to unify .pl->.S handling across all targets.]

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix build; ssltest

Reviewed-by: Richard Levitte <levitte@openssl.org>

Tweak some more information in INSTALL

The summary on how to add configs and how Makefile et al wasn't quite
correct any more.

Reviewed-by: Rich Salz <rsalz@openssl.org>

GH787: Fix ALPN

* Perform ALPN after the SNI callback; the SSL_CTX may change due to
  that processing
* Add flags to indicate that we actually sent ALPN, to properly error
  out if unexpectedly received.
* clean up ssl3_free() no need to explicitly clear when doing memset
* document ALPN functions

Signed-off-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Add an SSL_has_pending() function

This is similar to SSL_pending() but just returns a 1 if there is data
pending in the internal OpenSSL buffers or 0 otherwise (as opposed to
SSL_pending() which returns the number of bytes available). Unlike
SSL_pending() this will work even if "read_ahead" is set (which is the
case if you are using read pipelining, or if you are doing DTLS). A 1
return value means that we have unprocessed data. It does *not* necessarily
indicate that there will be application data returned from a call to
SSL_read(). The unprocessed data may not be application data or there
could be errors when we attempt to parse the records.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Add an ability to set the SSL read buffer size

This capability is required for read pipelining. We will only read in as
many records as will fit in the read buffer (and the network can provide
in one go). The bigger the buffer the more records we can process in
parallel.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Add cipher query functions

Add functions to determine authentication, key-exchange, FIPS and AEAD.

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert ERR_STRING_DATA to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert ERR_STATE to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert CRYPTO_LOCK_EX_DATA to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert CRYPTO_LOCK_DSO to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert CRYPTO_LOCK_{DH,DSA,RSA} to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Convert CRYPTO_LOCK_GET*BYNAME to new multi-threading API

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

GH815: The ChaCha20/Poly1305 codepoints are official

CCA8, CCA9, CCAA, CCAB, CCAC, CCAD, and CCAE are now present in
https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml
so remove the "as per draft-ietf-tls-chacha20-poly1305-03" note
accordingly.

Signed-off-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

GH787: Fix ALPN

* Perform ALPN after the SNI callback; the SSL_CTX may change due to
  that processing
* Add flags to indicate that we actually sent ALPN, to properly error
  out if unexpectedly received.
* clean up ssl3_free() no need to explicitly clear when doing memset
* document ALPN functions

Signed-off-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Emilia Käsper <emilia@openssl.org>