Don't fail on an out-of-order CCS in DTLS

Fixes #4929

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6196)

Fix a mem leak in CMS

The function CMS_RecipientInfo_set0_pkey() is a "set0" and therefore
memory management passes to OpenSSL. If the same function is called again
then we should ensure that any previous value that was set is freed first
before we set it again.

Fixes #5052

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6142)

(cherry picked from commit 3d551b20df1acd01f80d3ae00d37177e0fdf344a)

X509 time: tighten validation per RFC 5280

- Reject fractional seconds
- Reject offsets
- Check that the date/time digits are in valid range.
- Add documentation for X509_cmp_time

GH issue 2620

Backported from 80770da39e

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6181)

Configure: fix Mac OS X builds that still require makedepend

Earlier Apple Xcode compilers, e.g. one targeting Mac OS X 10.7, don't
support dependency generation and one still has to use makedepend.  It's
unclear when it was fixed, but all clang-based Apple compilers seem to
support -M options.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6167)

BIO_s_mem() write: Skip early when input length is zero

When the input length is zero, just return zero early.  Otherwise,
there's a small chance that memory allocation is engaged, fails and
returns -1, which is a bit confusing when nothing should be written.

Fixes #4782 #4827

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/6175)

(cherry picked from commit 0d94212a046e87fafea6e83e8ea2b2a58db49979)

docs: Fix typo EVP_PKEY_new_id -> EVP_PKEY_CTX_new_id

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6178)

bn/asm/*-mont.pl: harmonize with BN_from_montgomery_word.

Montgomery multiplication post-conditions in some of code paths were
formally non-constant time. Cache access pattern was result-neutral,
but a little bit asymmetric, which might have produced a signal [if
processor reordered load and stores at run-time].

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6163)

(cherry picked from commit 774ff8fed67e19d4f5f0df2f59050f2737abab2a)

Resolved conflicts in ppc-mont.pl and x86_64-mont.pl.

v3_purp.c: add locking to x509v3_cache_extensions()

Fixes #6121

Thanks to Mingtao Yang for reporting this bug.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6162)

Return an error from BN_mod_inverse if n is 1 (or -1)

Calculating BN_mod_inverse where n is 1 (or -1) doesn't make sense. We
should return an error in that case. Instead we were returning a valid
result with value 0.

Fixes #6004

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6119)

(cherry picked from commit b1860d6c71733314417d053a72af66ae72e8268e)

Make X509_VERIFY_PARAM_get_hostflags() take a const arg

Commit 14e55560 added this function which should have taken a const
argument.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6155)

Add some documentation for SSL_get_shared_ciphers()

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6114)

Fix comment in ssl_locl.h

The ciphers field in a session contains the stack of ciphers offered by
the client.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6114)

Fix SSL_get_shared_ciphers()

The function SSL_get_shared_ciphers() is supposed to return ciphers shared
by the client and the server. However it only ever returned the client
ciphers.

Fixes #5317

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6114)

a_strex.c: prevent out of bound read in do_buf()

which is used for ASN1_STRING_print_ex*() and X509_NAME_print_ex*().

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6117)

Fix regression with session cache use by clients

Commit d316cdcf6d8d6934663278145fe0a8191e14a8c5 introduced some extra
checks into the session-cache update procedure, intended to prevent
the caching of sessions whose resumption would lead to a handshake
failure, since if the server is authenticating the client, there needs to
be an application-set "session id context" to match up to the authentication
context.  While that change is effective for its stated purpose, there
was also some collatoral damage introduced along with the fix -- clients
that set SSL_VERIFY_PEER are not expected to set an sid_ctx, and so
their usage of session caching was erroneously denied.

Fix the scope of the original commit by limiting it to only acting
when the SSL is a server SSL.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5967)

(cherry picked from commit c4fa1f7fc016919a5b3d4ea2aa66c77e0cc40c9d)

Add getter for X509_VERIFY_PARAM_get_hostflags

Fixes #5061

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6140)

Clarify BN_mod_exp docs

Specifically this is not supported with an even modulus and
BN_FLG_CONSTTIME.

Fixes #5082

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6138)

Fix some errors and missing info in the CMS docs

Fixes #5063

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6135)

Update version docs

Make it clear that you should not attempt to get the version before the
first handshake is complete.

Fixes #2893

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6014)

apps/s_server.c: Avoid unused variable due to 'no-dtls'

Fixes #6098

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6104)

(cherry picked from commit ff25dd1a47525c88500dd61cdeb27faaea6ee599)

fix: BN_swap mishandles flags (1.1.0)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6101)

Fix memleaks in async api

Fixes: #5950

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6038)

(cherry picked from commit 74a8acbdfb2c7f398d1ae2fe914cd32b437f6df4)

Update the *use_certificate* docs

Note that calling the *use_certificate* functions will replace any existing
certificate of the same type. The same thing applies for private keys.

Fixes #2147

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6093)

Fix no-ui

In a recent commit, a OPENSSL_NO_UI guard was dropped by mistake in
PEM_def_callback(), so this puts it back.  At the same time, that
previous commit got rid of the necessity to have a OPENSSL_NO_STDIO
guard, so we drop that instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6096)

Use  get_last_sys_error() instead of get_last_rtl_error()

get_last_sys_error() already exists, so there's no need for yet
another macro that fulfills the same purpose.

Fixes #4120

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6088)

(cherry picked from commit b4c35656441929ce6f6be112648d637cba51dc91)

PEM_def_callback(): use same parameter names as for pem_password_cb

Add a bit more commentary to explain what's going on.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6080)

(cherry picked from commit d6d94d339756332bbabe2a1032ac511ae31b3fdc)

PEM_def_callback(): don't loop because of too short password given

That error is already caught by EVP_read_pw_string_min, and causes
this function to return -1, so the code detecting too short passwords
in this function is practically dead.

Fixes #5465

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6080)

(cherry picked from commit 4977b4e9281c981efcf6a8b31378b8bbd6a8a96f)

openssl rehash: exit 0 on warnings, same as c_rehash

Fixes #6083

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6084)

(cherry picked from commit e6a833cb97ed762408b57ea3efa83bd10c1d2a78)

ms/uplink-x86.pl: close the file handle that was opened

Fixes #5656

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6079)

(cherry picked from commit 3625f6c977beccd7bb0d80a1a670c19f4338a27a)

Fix documentation for the -showcerts s_client option

This option shows the certificates as sent by the server. It is not the
full verified chain.

Fixes #4933

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6068)

Fix the MAX_CURVELIST definition

The MAX_CURVELIST macro defines the total number of in-built SSL/TLS curves
that we support. However it has not been updated as new curves are added.

Fixes #5232

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/6065)

(cherry picked from commit ca50cd911ca3c9dc9ec8dd956f8eb45557585a98)

In a reneg use the same client_version we used last time

In 1.0.2 and below we always send the same client_version in a reneg
ClientHello that we sent the first time around, regardless of what
version eventually gets negotiated. According to a comment in
statem_clnt.c this is a workaround for some buggy servers that choked if
we changed the version used in the RSA encrypted premaster secret.

In 1.1.0+ this behaviour no longer occurs. This restores the original
behaviour.

Fixes #1651

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6060)

Document when a session gets removed from cache

Document the fact that if a session is not closed down cleanly then the
session gets removed from the cache and marked as non-resumable.

Fixes #4720

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6054)

Allow intermediate CAs to use RSA PSS in 1.1.0

In 1.1.0 and above we check the digest algorithm used to create signatures
in intermediate CA certs. If it is not sufficiently strong then we reject
the cert. To work out what digest was used we look at the OID for the
signature. This works for most signatures, but not for RSA PSS where the
digest is stored as parameter of the SignatureAlgorithmIdentifier. This
results in the digest look up routines failing and the cert being rejected.

PR #3301 added support for doing this properly in master. So in that
branch this all works as expected. It also works properly in 1.0.2 where we
don't have the digest checks at all. So the only branch where this fails is
1.1.0.

PR #3301 seems too significant to backport to 1.1.0. Instead we simply skip
the signature digest algorithm strength checks if we detect RSA PSS.

Fixes #3558.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/6052)

Improve backwards compat with 1.0.2 for ECDHParameters

In 1.0.2 you could configure automatic ecdh params by using the
ECDHParameters config directive and setting it to the value
"+Automatic" or just "Automatic". This is no longer required in 1.1.0+
but we still recognise the "+Automatic" keyword for backwards compatibility.
However we did not recognise just "Automatic" without the leading "+" which
is equally valid. This commit fixes that omission.

Fixes #4113

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6036)

Remove superfluous NULL checks. Add Andy's BN_FLG comment.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6009)

(cherry picked from commit 39df51522ba2e3773ae2f1d4df5a6031ef41c1ba)

Move up check for EC_R_INCOMPATIBLE_OBJECTS and for the point at infinity case

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6009)

(cherry picked from commit 736b31e5ea33166d89d5cff5774697d0c15d96bd)

Pass through

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6009)

(cherry picked from commit f4675379275c304dbfa593cc573b4e4c4eb54bd4)

ladder description: why it works

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6009)

(cherry picked from commit a067a8705a654c85d43b942e0d1616e282667969)

Address code style comments

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6009)

(cherry picked from commit 36bed230b580f92d2e10d13e4ba472236e622562)

Elliptic curve scalar multiplication with timing attack defenses

Co-authored-by: Nicola Tuveri <nic.tuv@gmail.com>
Co-authored-by: Cesar Pereida Garcia <cesar.pereidagarcia@tut.fi>
Co-authored-by: Sohaib ul Hassan <soh.19.hassan@gmail.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6009)

(cherry picked from commit 40e48e54582e46c1a01e184ecf5bd31f4f7f8294)

ARM assembly pack: make it work with older assembler.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/6043)

(cherry picked from commit 198a2ed791e8f4f00d0b92272987f564ca1d9783)

Fix openssl ca, to correctly make output file binary when using -spkac

On Unix, this doesn't matter, but on other platforms, it may.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6050)

(cherry picked from commit 25642ad29e6a2c15c10ceb5e4f029638f73a879e)

Fix late opening of output file

For 'openssl dhparams', the output file was opened after calculations
were made, which is a waste of cycles and time if the output file
turns out not to be writable.

Fixes #3404

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6051)

(cherry picked from commit 10b37541dc7f27b0ca74102970691ce4636cc0bd)

Ensure the thread keys are always allocated in the same order

Back-port of #5911
Fixes: #5899

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6037)

Fix a warning about missing prototype on arm

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6028)

Fix a gcc-8 warning -Wcast-function-type

Casting to the generic function type "void (*)(void)"
prevents the warning.

Backport of #5816

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6027)

Fix building linux-armv4 with --strict-warnings

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6026)

(cherry picked from commit 0e0f8116e247912f5c48f8b3786e543f37fc1f87)

Add a test for SSL_pending()

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6021)

Fix SSL_pending() for DTLS

DTLS was not correctly returning the number of pending bytes left in
a call to SSL_pending(). This makes the detection of truncated packets
almost impossible.

Fixes #5478

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6021)

Fix the alert sent if no shared sig algs

We were sending illegal parameter. This isn't correct. The parameters are
legal, we just don't have an overlap. A more appropriate alert is
handshake failure.

Fixes #2919

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6010)

Don't distribute team internal config targets

Configurations/90-team.conf isn't for public consumption, so we rename
it to 90-team.norelease.conf and make sure 'make dist' and 'make tar'
don't include it in the tarball.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6016)

Add a test for a NULL X509_STORE in X509_STORE_CTX_init

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6002)

Don't crash if there are no trusted certs

The X509_STORE_CTX_init() docs explicitly allow a NULL parameter for the
X509_STORE. Therefore we shouldn't crash if we subsequently call
X509_verify_cert() and no X509_STORE has been set.

Fixes #2462

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6002)

Fix ocsp app exit code

If we run the ocsp command line app and the responder returns a
non-successful status code then the app should exit with a failure code.

Based on an original patch by Tatsuhiro Tsujikawa.

Fixes #2387

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5999)

X509_cmp_time: only return 1, 0, -1.

The behaviour of X509_cmp_time used to be undocumented.

The new behaviour, documented in master, is to return only 0, 1, or -1.
Make the code in the other branches to adhere to this behaviour too,
to reduce confusion. There is nothing to be gained from returning
other values.

Fixes GH#4954

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4955)

Update EVP_DigestSignInit() docs

Explicitly state which digests can be used with which algorithms.

Fixes #5854

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5995)

poly1305/asm/poly1305-armv4.pl: remove unintentional relocation.

Branch to global symbol results in reference to PLT, and when compiling
for THUMB-2 - in a R_ARM_THM_JUMP19 relocation. Some linkers don't
support this relocation (ld.gold), while others can end up truncating
the relocation to fit (ld.bfd).

Convert this branch through PLT into a direct branch that the assembler
can resolve locally.

See https://github.com/android-ndk/ndk/issues/337 for background.

The current workaround is to disable poly1305 optimization assembly,
which is not optimal and can be reverted after this patch:
https://github.com/freedesktop/gstreamer-cerbero/commit/beab607d2b1ff23c41b7e01aa9c64be5e247d1e6

CLA: trivial

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5949)

(cherry picked from commit 5bb1cd2292b388263a0cc05392bb99141212aa53)

Check the return from EVP_PKEY_get0_DH()

Fixes #5934

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5993)

Fix assertion failure in SSL_set_bio()

If SSL_set_bio() is called with a NULL wbio after a failed connection then
this can trigger an assertion failure. This should be valid behaviour and
the assertion is in fact invalid and can simply be removed.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5966)

(cherry picked from commit bd7775e14a19c326d3720f2345c2ae324409e979)

Update fingerprints.txt

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5987)

(cherry picked from commit 2bdeffefddd8e8a65a51a7b020f8d51a4a3b1602)

Ignore the status_request extension in a resumption handshake

We cannot provide a certificate status on a resumption so we should
ignore this extension in that case.

Fixes #1662

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5897)

p5_scrypt.c: fix error check of RAND_bytes() call

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/5977)

Revert "Add OPENSSL_VERSION_AT_LEAST"

Fixes #5961

This reverts commit d8adfdcd2e5de23f3e1d1a1d10c2fda6f4a65c4a.

The macros OPENSSL_MAKE_VERSION() and OPENSSL_VERSION_AT_LEAST() contain
errors and don't work as designed. Apart from that, their introduction
should be held back until a decision has been mad about the future
versioning scheme.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5968)

RSA key generation: ensure BN_mod_inverse and BN_mod_exp_mont both get called with BN_FLG_CONSTTIME flag set.

CVE-2018-0737

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Custome built dladdr() for AIX.

Implemented a stripped down dladdr()-implementation using AIX' own
loadquery()-function. Following the SGI example in the same code, the
DL_info only has the dli_fname member. As the scope of
dlfcn_pathbyaddr() is the filename, this implementation does not
consider archive members, which can be dlopen()ed in AIX.
Added DATA segment checking to catch ptrgl virtual addresses.

Added test case for DSO_dsobyaddr(), but only for DSO_DLFCN.
Added PIC-flag to aix*-cc build targets.

Signed-off-by: Matthias Kraft <makr@gmx.eu>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5626)

test/recipes/test_genrsa.t : don't fail because of size limit changes

There is a test to check that 'genrsa' doesn't accept absurdly low
number of bits.  Apart from that, this test is designed to check the
working functionality of 'openssl genrsa', so instead of having a hard
coded lower limit on the size key, let's figure out what it is.

Partially fixes #5751

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/5754)

Docs for OpenSSL_init_crypto: there is no way to specify another file

The documentation erroneously stated that one can change the default
configuration file name.

Fixes #5939

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5941)

(cherry picked from commit 0320e8e2869fb6cde4579375e65f6d576bbec95e)

openssl rehash: use libcrypto variables for default dir

X509_get_default_cert_dir_env() returns the default environment
variable to check for certificate directories.
X509_get_default_cert_dir() returns the default configured certificate
directory.

Use these instead of hard coding our own values, and thereby be more
integrated with the rest of OpenSSL.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5937)

(cherry picked from commit 362ff3c39703633ac53ba9130c7acc02177bedc5)

openssl rehash: document -compat

Fixes #5902

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5937)

(cherry picked from commit adaf3cfad2319fe5c0d6fe179c726cc85eebf2d3)

Clarify default section in config.pod

This is a minor update which hopefully makes these particular lines
read a little easier.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5938)

(cherry picked from commit 0652e8a7fe6dd8cbdf4516b221642b10bbfc68fe)

Fix bugs in X509_NAME_ENTRY_set

The wrong "set" field was incremented in the wrong place and would
create a new RDN, not a multi-valued RDN.
RDN inserts would happen after not before.
Prepending an entry to an RDN incorrectly created a new RDN

Anything which built up an X509_NAME could get a messed-up structure,
which would then be "wrong" for anyone using that name.

Thanks to Ingo Schwarze for extensive debugging and the initial
fix (documented in GitHub issue #5870).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/5882)

(cherry picked from commit bbf27cd58337116c57a1c942153330ff83d5540a)

bio/b_addr.c: resolve HP-UX compiler warnings.

The warning reads "[cast] may cause misaligned access". Even though
this can be application-supplied pointer, misaligned access shouldn't
happen, because structure type is "encoded" into data itself, and
application would customarily pass correctly aligned pointer. But
there is no harm in resolving the warning...

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5894)

(cherry picked from commit 55bd917bc4213bc668f48b87d8c6feb9918fef8f)

Configurations/10-main.conf: further HP-UX cleanups/unifications.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Configurations/10-main.conf: add magic macros to hpux targets.

HP-UX provides sockets symbols with incompatible prototypes under same
names. Additional macros force unambitious symbols with unambitious
prototypes.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/5862)

Updated to CONTRIBUTING to reflect GitHub, etc.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5889)
(cherry picked from commit 2876872ffe5dd53ec1c446656e924ff463e5d4bf)

Set biom->type in BIO_METH_new

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5812)

Fix an error code to be consistent with master

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5892)

Add test/versions to gitignore

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5881)

Update the genpkey documentation

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5883)

Pick a q size consistent with the digest for DSA param generation

There are two undocumented DSA parameter generation options available in
the genpkey command line app:
dsa_paramgen_md and dsa_paramgen_q_bits.

These can also be accessed via the EVP API but only by using
EVP_PKEY_CTX_ctrl() or EVP_PKEY_CTX_ctrl_str() directly. There are no
helper macros for these options.

dsa_paramgen_q_bits sets the length of q in bits (default 160 bits).
dsa_paramgen_md sets the digest that is used during the parameter
generation (default SHA1). In particular the output length of the digest
used must be equal to or greater than the number of bits in q because of
this code:

            if (!EVP_Digest(seed, qsize, md, NULL, evpmd, NULL))
                goto err;
            if (!EVP_Digest(buf, qsize, buf2, NULL, evpmd, NULL))
                goto err;
            for (i = 0; i < qsize; i++)
                md[i] ^= buf2[i];

            /* step 3 */
            md[0] |= 0x80;
            md[qsize - 1] |= 0x01;
            if (!BN_bin2bn(md, qsize, q))
                goto err;

qsize here is the number of bits in q and evpmd is the digest set via
dsa_paramgen_md. md and buf2 are buffers of length SHA256_DIGEST_LENGTH.
buf2 has been filled with qsize bits of random seed data, and md is
uninitialised.

If the output size of evpmd is less than qsize then the line "md[i] ^=
buf2[i]" will be xoring an uninitialised value and the random seed data
together to form the least significant bits of q (and not using the
output of the digest at all for those bits) - which is probably not what
was intended. The same seed is then used as an input to generating p. If
the uninitialised data is actually all zeros (as seems quite likely)
then the least significant bits of q will exactly match the least
significant bits of the seed.

This problem only occurs if you use these undocumented and difficult to
find options and you set the size of q to be greater than the message
digest output size. This is for parameter generation only not key
generation. This scenario is considered highly unlikely and
therefore the security risk of this is considered negligible.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5883)

Don't crash if an unrecognised digest is used with dsa_paramgen_md

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5883)

Move the loading of the ssl_conf module to libcrypto

The GOST engine needs to be loaded before we initialise libssl. Otherwise
the GOST ciphersuites are not enabled. However the SSL conf module must
be loaded before we initialise libcrypto. Otherwise we will fail to read
the SSL config from a config file properly.

Another problem is that an application may make use of both libcrypto and
libssl. If it performs libcrypto stuff first and OPENSSL_init_crypto()
is called and loads a config file it will fail if that config file has
any libssl stuff in it.

This commit separates out the loading of the SSL conf module from the
interpretation of its contents. The loading piece doesn't know anything
about SSL so this can be moved to libcrypto. The interpretation of what it
means remains in libssl. This means we can load the SSL conf data before
libssl is there and interpret it when it later becomes available.

Fixes #5809

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5879)

fix build error in 32 bit debug build

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5861)

Don't use CPP in Configurations/unix-Makefile.tmpl

We started using $(CPP) instead of $(CC) -E, with the assumption that
CPP would be predefined.  This is, however, not always true, and
rather depends on the 'make' implementation.  Furthermore, on
platforms where CPP=cpp or something else other than '$(CC) -E',
there's a risk that it won't understand machine specific flags that we
pass to it.  So it turns out that trying to use $(CPP) was a mistake,
and we therefore revert that use back to using $(CC) -E directly.

Fixes #5867

Note: this affects config targets that use Alpha, ARM, IA64, MIPS,
s390x or SPARC assembler modules.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5871)

Fix some errors in the mem leaks docs

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5795)

Fix a text canonicalisation bug in CMS

Where a CMS detached signature is used with text content the text goes
through a canonicalisation process first prior to signing or verifying a
signature. This process strips trailing space at the end of lines, converts
line terminators to CRLF and removes additional trailing line terminators
at the end of a file. A bug in the canonicalisation process meant that
some characters, such as form-feed, were incorrectly treated as whitespace
and removed. This is contrary to the specification (RFC5485). This fix
could mean that detached text data signed with an earlier version of
OpenSSL 1.1.0 may fail to verify using the fixed version, or text data
signed with a fixed OpenSSL may fail to verify with an earlier version of
OpenSSL 1.1.0. A workaround is to only verify the canonicalised text data
and use the "-binary" flag (for the "cms" command line application) or set
the SMIME_BINARY/PKCS7_BINARY/CMS_BINARY flags (if using CMS_verify()).

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5791)

Change the "offset too large" message to more generic wording

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

(cherry picked from commit 1518c55a796b058eff01f3cbf177f4b726c01d7c)

Remove an unnecessary cast in the param to BUF_MEM_grow

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

(cherry picked from commit 21db0e1afdcf17a189ebe49af3506551b108a2f2)

Fix range checks with -offset and -length in asn1parse

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

(cherry picked from commit 16e1eea6a67c85c9d786f3c4448182b1aca101b8)

Use strtol instead of atoi in asn1parse

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

(cherry picked from commit 18ada952d1dcea8dfc5008df9f317b3eb47c13fa)

Improve diagnostics for invalid arguments in asn1parse -strparse

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

(cherry picked from commit b998745a4596c05f673ed8acdcaedcb4c5e208ad)

Fix a crash in the asn1parse command

Thanks to Sem Voigtländer for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5826)

(cherry picked from commit 752837e0664e990b5edf6f0b69e1b4612efadce0)

EVP,KDF: Add more error code along some return 0

in methods :
- EVP_PBE_scrypt
- EVP_PKEY_meth_add0
- EVP_PKEY_meth_new
- EVP_PKEY_CTX_dup

Backport of 3484236d8d7afedd3e5c7771bd49d3385340e3bf

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5803)

Remove import/use of File::Spec::Function

It looks like the usage of these functions were removed in
in commit 0a4edb931b883b9973721ae012e60c028387dd50 ("Unified - adapt
the generation of cpuid, uplink and buildinf to use GENERATE").

This commit removes the import/use of File::Spec::Functions module as it
is no longer needed by crypto/build.info.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5832)

(cherry picked from commit 0e34f37fb1b7474c86ac9a170dfda5226351ecc9)

VMS: Copy DECC inclusion epi- and prologues to internals

Because many of our test programs use internal headers, we need to make
sure they know how, exactly, to mangle the symbols.  So far, we've done
so by specifying it in the affected test programs, but as things change,
that will develop into a goose chase.  Better then to declare once and
for all how symbols belonging in our libraries are meant to be treated,
internally as well as publically.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3259)

(cherry picked from commit f46f69f4092768ec4b911ced45c8cc73cc008739)

Refuse to run test_cipherlist unless shared library matches build

test/cipherlist_test.c is an internal consistency check, and therefore
requires that the shared library it runs against matches what it was
built for.  test/recipes/test_cipherlist.t is made to refuse running
unless library version and build version match.

This adds a helper program test/versions.c, that simply displays the
library and the build version.

Partially fixes #5751

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5753)

.travis.yml: with fast fuzz testing, there is no point avoiding it

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5788)

Faster fuzz test: teach the fuzz test programs to handle directories

Instead of invoking the fuzz test programs once for every corpora
file, we invoke them once for each directory of corpora files.  This
dramatically reduces the number of program invokations, as well as the
time 90-test_fuzz.t takes to complete.

fuzz/test-corpus.c was enhanced to handle directories as well as
regular files.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5788)

Revert "util/dofile.pl: only quote stuff that actually needs quoting"

This wasn't a good solution, too many things depend on the quotes being
there consistently.

This reverts commit 49cd47eaababc8c57871b929080fc1357e2ad7b8.

Fixes #5772

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5773)

(cherry picked from commit 00701e5ea84861b74d9d624f21a6b3fcb12e8acd)