Make a2i_ipadd an internal function

Reviewed-by: Richard Levitte <levitte@openssl.org>

More API docs; small changes.

Also fix typo noted on GitHub.
Suppport typedef and #define to find-doc-nits

Reviewed-by: Richard Levitte <levitte@openssl.org>

Clean up "generic" intro pod files.

Files like dh.pod, etc., mostly duplicated the API-specific pod files.
Removed the duplicated content; that often mean the whole file could
be removed.  Some of the content about internals got moved into README
files in the source tree.  Some content (e.g., err.pod) got moved
into other pod pages.

Annotate generic pages, remove dup NAME

Reviewed-by: Richard Levitte <levitte@openssl.org>

Write X509_dup, PEM_read, etc.

Partially document the ASN1 template stuff, and its use for i2d/d2i
and PEM I/O.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Docs: install generic manpages to man section 7

Reviewed-by: Rich Salz <rsalz@openssl.org>

hmac/hmac.c: fix sizeof typo in hmac_ctx_cleanup.

Reviewed-by: Matt Caswell <matt@openssl.org>

Fix session ticket and SNI

When session tickets are used, it's possible that SNI might swtich the
SSL_CTX on an SSL. Normally, this is not a problem, because the
initial_ctx/session_ctx are used for all session ticket/id processes.

However, when the SNI callback occurs, it's possible that the callback
may update the options in the SSL from the SSL_CTX, and this could
cause SSL_OP_NO_TICKET to be set. If this occurs, then two bad things
can happen:

1. The session ticket TLSEXT may not be written when the ticket expected
flag is set. The state machine transistions to writing the ticket, and
the client responds with an error as its not expecting a ticket.
2. When creating the session ticket, if the ticket key cb returns 0
the crypto/hmac contexts are not initialized, and the code crashes when
trying to encrypt the session ticket.

To fix 1, if the ticket TLSEXT is not written out, clear the expected
ticket flag.
To fix 2, consider a return of 0 from the ticket key cb a recoverable
error, and write a 0 length ticket and continue. The client-side code
can explicitly handle this case.

Fix these two cases, and add unit test code to validate ticket behavior.

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1098)

Fixes to get -ansi working

Various fixes to get the following to compile:

./config no-asm -ansi -D_DEFAULT_SOURCE

RT4479
RT4480

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Add some accessor API's

GH1098: Add X509_get_pathlen() (and a test)
GH1097:  Add SSL_is_dtls() function.

Documented.

Reviewed-by: Matt Caswell <matt@openssl.org>

crypto/x509/x509_vpm.c: Simplify int_x509_param_set1()

This change also avoids calling strlen twice when srclen is 0

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Always use session_ctx when removing a session

Sessions are stored on the session_ctx, which doesn't change after
SSL_set_SSL_CTX().

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix some typos in pod files

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1189)

Ignore tsget.pl and c_rehash.pl, as well as other compiler generated files

Reviewed-by: Matt Caswell <matt@openssl.org>

Add empty record tests

The previous commit changed how we handle out-of-context empty records.
This commit adds some tests for the various scenarios. There are three
tests:
1: Check that if we inject an out-of-context empty record then we fail
2: Check that if we inject an in-context empty record then we succeed
3: Check that if we inject too many in-context empty records then we fail.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Reject out of context empty records

Previously if we received an empty record we just threw it away and
ignored it. Really though if we get an empty record of a different content
type to what we are expecting then that should be an error, i.e. we should
reject out of context empty records. This commit makes the necessary changes
to achieve that.

RT#4395

Reviewed-by: Andy Polyakov <appro@openssl.org>

Fix pipelining bug

The number of read pipelines should be reset in the event of reuse of an
SSL object.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Add a BN_mod_word test()

The previous commit fixed a bug with BN_mod_word() which would have been
caught if we had a test for it. This commit adds one.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Fix BN_mod_word bug

On systems where we do not have BN_ULLONG (e.g. typically 64 bit systems)
then BN_mod_word() can return incorrect results if the supplied modulus is
too big.

RT#4501

Reviewed-by: Andy Polyakov <appro@openssl.org>

Unify d2i/i2d documentation.

Make d2i_X509 a generic d2i/i2d manpage.
Pull common stuff out of other d2i/i2d docs.
Update find-doc-nits to know about "generic" manpages.
Cleanup some overlap.
Fix up a bunch of other references.

Reviewed-by: Matt Caswell <matt@openssl.org>

Nit about pod filenames

The asdf.pod filename must have asdf in its NAME section.
also check for names existing as a different filename (via Levitte)

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Fix potential access of null pointer (pp)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1186)

Add an SSL get/set test

We just do the getters/setter for tlsext_status_type. This could be extended
for others in the future.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add documentation for the newly added SSL_get_tlsext_status_type()

And also for SSL_CTX_get_tlsext_status_type()

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add SSL_CTX_get_tlsext_status_type()

Reviewed-by: Rich Salz <rsalz@openssl.org>

Return the value of tlsext_status_type in the return not arg

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add SSL_get_tlsext_status_type() method

The tlsext_status_type field in SSL is used by e.g. OpenResty to determine
if the client requested the certificate status, but SSL is now opaque.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix some GitHub issues.

GH1180: Local variable sometimes unused
GH1181: Missing close paren.

Thanks to <wipedout@yandex.ru> for reporting these.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Set flag BN_FLG_CONSTTIME earlier

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Remove kq and set BN_FLG_CONSTTIME in k for BN_mod_inverse

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Add script to find undocumented API

Also tweaks to find-doc-nits, including name/synopsis checking.
Ironically, it also reports on duplicated doc names :)

Reviewed-by: Richard Levitte <levitte@openssl.org>

Document the issue with threads and dlopen()

If using threads and OpenSSL is loaded via dlopen(), and subsequently
closed again via dlclose() *before* the threads are destroyed, then
OpenSSL will not free up the per thread resources. We need to document
this restriction, and provide some guidance on what to do about it.

I did some testing and discovered/verified a few of things (at least
this is the behaviour on Linux):

- Using OpenSSL via dlopen in a mutli-threaded app does leak memory if
threads are destroyed after dlcose() is called.
- In a single threaded environment, or if threads are destroyed prior to
dlclose() being called, then no memory is leaked
- Using the RTLD_NODELETE flag to dlopen solves the above problem
- Interestingly the OpenSSL atexit() handler gets called when dlclose()
is called rather than at application exit (I was worred that it might crash
if there was an atexit() handler for a function that has been unloaded)
- RTLD_NODELETE is a non-standard flag - but it does seem to be fairly
widely supported. As far as I could determine (via google), at least Linux,
Solaris, OpenBSD, FreeBSD, HP-UX all seem to support it.

I also tested on Windows (using LoadLibrary instead of dlopen and
FreeLibrary instead of dlclose) and experienced similar behaviour, except
that (AFAIK) there is no equivalent of RTLD_NODELETE on Windows.

GitHub Issue #653

Reviewed-by: Richard Levitte <levitte@openssl.org>

More doc nits

Update script to look for period or POD markup in NAME section, and
fix them.

Reviewed-by: Andy Polyakov <appro@openssl.org>

NOTES.WIN: use secure urls
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1175)

tests: fix the shutting up of the shell

If we're going to redirect STDERR and STDOUT, it's better to do it by
the book.  This fix is a straight copy of examples in the perlfunc
manual.

Reviewed-by: Matt Caswell <matt@openssl.org>

Fix the directory target generation

The directories for the final products were never registered, it was
plain luck that intermediary files were in the same place and
registered the directory anyway.

Also, scripts are generated directly from source (binaries go through
intermadiary object files), so we need to explicitely make sure to
avoid registering the source directory unless it's an in source
build.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Deprecate the flags that switch off constant time

The flags RSA_FLAG_NO_CONSTTIME, DSA_FLAG_NO_EXP_CONSTTIME and
DH_FLAG_NO_EXP_CONSTTIME which previously provided the ability to switch
off the constant time implementation for RSA, DSA and DH have been made
no-ops and deprecated.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Simplify dsa_ossl.c

The dsa_ossl.c file defined a couple of multi-line macros, but then only
used each one once. The macros just serve to complicate the code and make
it more difficult to understand what is really going on. Hence they are
removed.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix DSA, preserve BN_FLG_CONSTTIME

Operations in the DSA signing algorithm should run in constant time in
order to avoid side channel attacks. A flaw in the OpenSSL DSA
implementation means that a non-constant time codepath is followed for
certain operations. This has been demonstrated through a cache-timing
attack to be sufficient for an attacker to recover the private DSA key.

CVE-2016-2178

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

s390x assembly pack: improve portability.

Some of the instructions used in latest additions are extension
ones. There is no real reason to limit ourselves to specific
processors, so [re-]adhere to base instruction set.

RT#4548

Reviewed-by: Rich Salz <rsalz@openssl.org>

tests: Shut the shell up unless verbose

In rare cases, the shell we run test programs in may have complaints.
Shut those up unless testing verbosely.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Remove extra include's in synopsis.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Configure: complete the changed fuzz option checks

Reviewed-by: Andy Polyakov <appro@openssl.org>

Silence misleading test_abort stderr output

Reviewed-by: Rich Salz <rsalz@openssl.org>

Constify X509V3_EXT_*_conf*

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1074)

Constify ASN1_generate_nconf

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1074)

Constify ASN1_generate_v3

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1074)

Constify s2i_ASN1_IA5STRING

Return directly NULL after ASN1_STRING_set, as it already has set an error code.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1074)

Constify s2i_ASN1_INTEGER

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1074)

Constify X509_OBJECT_get_type & X509_OBJECT_get0_X509

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1074)

Clean away the last unixmake vestiges

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix re-used function code

Reviewed-by: Richard Levitte <levitte@openssl.org>

Ignore buildtest artifacts.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add developer targets for each subdirectory we have something to build in

Previous build scheme allowed building just the stuff in one
subdirectory, like this:

    make -C crypto/aes

Because the unified only has a top-level Makefile, this is not
possible with it.  This change adds a replacement where each directory
we have something to build in becomes a target in its own right,
allowing building something like this:

    make crypto/aes

The exception is the directory test, because we already have such a
target.

Reviewed-by: Stephen Henson <steve@openssl.org>

Specifiy size of arrays

Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1139

Check for overflows in EOC.

RT#4474 (partial)

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add support for fuzzing with AFL

Reviewed-by: Ben Laurie <ben@links.org>
MR: #2740

RT3895: Remove fprintf's from SSL library.

Reviewed-by: Richard Levitte <levitte@openssl.org>

More utils cleanup.

Remove some unused files.
Rename doc-nit-check to be consistent.
Add check for multiple #include in synopsis.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Make 25-test_gen.t and 25-test_req.t into one

Since one generates files that the other depends on, there's no
real reason to keep them separate.  Since they were both different
aspects of 'openssl req', the merge ends up in 25-test_req.t.

This also makes cleanup easier.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Have some more test recipes clean up after themselves

Reviewed-by: Rich Salz <rsalz@openssl.org>

Windows build: Remove .manifest files in test/ as well

Reviewed-by: Rich Salz <rsalz@openssl.org>

Change inclusion of sys/types.h to stdlib.h in include/openssl/ebcdic.h

Needed to get size_t on Windows

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add inclusion of stdlib.h in include/openssl/mdc2.h

Needed to get size_t

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add inclusion of openssl/x509.h in include/openssl/tls1.h

Needed to get the needed declarations for STACK_OF(X509)

Reviewed-by: Rich Salz <rsalz@openssl.org>

Generate simple build test files

Generate small test programs to check that external programs can be
built with our stuff at a very basic level.

For now, they check that each of our header files can be included
individually without compile failure.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix documentation error in x509 app certopt flag

According to the x509 man page in the section discussing -certopt it says
that the ca_default option is the same as that used by the ca utility and
(amongst other things) has the effect of suppressing printing of the
signature - but in fact it doesn't. This error seems to have been present
since the documentation was written back in 2001. It never had this effect.

The default config file sets the certopt value to ca_default. The ca utility
takes that and THEN adds additional options to suppress printing of the
signature. So the ca utility DOES suppress printing of the signature - but
it is not as a result of using the ca_default option.

GitHub Issue #247

Reviewed-by: Rich Salz <rsalz@openssl.org>

BIO_printf() can fail to print the last character

If the string to print is exactly 2048 character long (excluding the NULL
terminator) then BIO_printf will chop off the last byte. This is because
it has filled its static buffer but hasn't yet allocated a dynamic buffer.
In cases where we don't have a dynamic buffer we need to truncate but that
is not the case for BIO_printf(). We need to check whether we are able to
have a dynamic buffer buffer deciding to truncate.

Reviewed-by: Rich Salz <rsalz@openssl.org>

cryptodev_asym, zapparams: use OPENSSL_* allocation routines, handle errors

zapparams modification based on tip from Matt Caswell

RT#3198

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Handle a memory allocation failure in ssl3_init_finished_mac()

The ssl3_init_finished_mac() function can fail, in which case we need to
propagate the error up through the stack.

RT#3198

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update INSTALL instructions

Fill out the INSTALL instructions with more information on Configure
arguments, environment variables and Makefile targets.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Added define for STATUS_SUCCESS

Use STATUS_SUCCESS instead of 0.
Renamed USE_BCRYPT to RAND_WINDOWS_USE_BCRYPT to avoid possible collisions with other defines.
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1142)

Adds casts for 64-bit

Adds missing casts for 64-bit.
Removed zero initialization of hProvider. hProvider is an "out" parameter of CryptAcquireContextW.
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1142)

Define USE_BCRYPT

Define USE_BCRYPT
Removed _WIN32_WINNT define
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1142)

Use BCryptGenRandom on Windows 7 or higher

When openssl is compiled with MSVC and _WIN32_WINNT>=0x0601 (Windows 7), BCryptGenRandom is used instead of the legacy CryptoAPI.

This change brings the following benefits:
- Removes dependency on CryptoAPI (legacy API) respectively advapi32.dll
- CryptoAPI Cryptographic Service Providers (rsa full) are not dynamically loaded.
- Allows Universal Windows Platform (UWP) apps to use openssl (CryptGenRandom is not available for Windows store apps)
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1142)

Add a paragraph on documentation to CONTRIBUTING

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Further update CONTRIBUTING

Tweak to the wording on merge commits.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Tweaks to NOTES.PERL

Fix some typos and other minor amendments to NOTES.PERL.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Update NOTES.WIN

Make the recommendation for MSYS perl in an MSYS environment more forceful.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Update CONTRIBUTING

Fix typos and clarify a few things in the CONTRIBUTING file.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Bring the README file up to date

The README file was a little out of date so needed a refresh

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix threading issue that at best will leak memory

The problem is the checking in policy_cache_set, there is a race
condition between the null check and obtaining the lock.  The fix is in
policy_cache_new to detect if the creation has happened already.

Reviewed-by: Richard Levitte <levitte@openssl.org>

tests: clean up temporary SSL session files.

RT#4557

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add ct fuzzer.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Linkify libfuzzer.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fuzz everything with every input.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Sort.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update client authentication tests

Port client auth tests to the new framework, add coverage. The old tests
were only testing success, and only for some protocol versions; the new
tests add all protocol versions and various failure modes.

Reviewed-by: Rich Salz <rsalz@openssl.org>

chacha/chacha_enc.c: harmonize counter width with subroutine name.

_ctr32 in function name refers to 32-bit counter, but it was implementing
64-bit one. This didn't pose problem to EVP, but 64-bit counter was just
misleading.

RT#4512

Reviewed-by: Richard Levitte <levitte@openssl.org>

GH1123: sort dir before rehash

This is needed to generate stable output names/symlinks.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Build the 'openssl rehash' command on VMS version 8.3 and up

Include a note in INSTALL that tests must be run from an unprivileged
process.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix test failures when using enable-ubsan

Numerous test failures were occuring when Configured with enable-ubsan
although they could all be traced back to one issue.

Reviewed-by: Andy Polyakov <appro@openssl.org>

Tidy up OCSP print handling.

Also fixes -Wstrict-oveflow warning on this file.

RT#4474 (partial)

Reviewed-by: Richard Levitte <levitte@openssl.org>

perl: refactor .pod name section extractor into its own module

Adapt util/process_docs.pl

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add a warning about using enable-crypto-mdebug-backtrace

Reviewed-by: Matt Caswell <matt@openssl.org>

Testing symbol presence: also take note of small objects

The S symbol class wasn't checked.

Notified by Sebastian Andrzej Siewior

Reviewed-by: Tim Hudson <tjh@openssl.org>

Remove NOEXIST entries

checkpoint before release.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't leak memory on set_reasons() error path

The set_reasons() function in v3_crld.c leaks a STACK_OF(CONF_VALUE)
object on an error path.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't leak memory on int X509_PURPOSE_add() error path

The int X509_PURPOSE_add() function was leaking an X509_PURPOSE object
on error.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't leak memory on X509_TRUST_add() error path

The X509_TRUST_add() function was leaking an X509_TRUST object on error.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't leak memory on ASN1_item_pack() error path

The ASN1_item_pack() function was leaking an ASN1_STRING object on error
paths.

Reviewed-by: Richard Levitte <levitte@openssl.org>