Apply self-imposed path length also to root CAs

Also, some readers of the code find starting the count at 1 for EE
cert confusing (since RFC5280 counts only non-self-issued intermediate
CAs, but we also counted the leaf).  Therefore, never count the EE
cert, and adjust the path length comparison accordinly.  This may
be more clear to the reader.

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit dc5831da59e9bfad61ba425d886a0b06ac160cd6)

Only CA certificates can be self-issued

At the bottom of https://tools.ietf.org/html/rfc5280#page-12 and
top of https://tools.ietf.org/html/rfc5280#page-13 (last paragraph
of above https://tools.ietf.org/html/rfc5280#section-3.3), we see:

   This specification covers two classes of certificates: CA
   certificates and end entity certificates.  CA certificates may be
   further divided into three classes: cross-certificates, self-issued
   certificates, and self-signed certificates.  Cross-certificates are
   CA certificates in which the issuer and subject are different
   entities.  Cross-certificates describe a trust relationship between
   the two CAs.  Self-issued certificates are CA certificates in which
   the issuer and subject are the same entity.  Self-issued certificates
   are generated to support changes in policy or operations.  Self-
   signed certificates are self-issued certificates where the digital
   signature may be verified by the public key bound into the
   certificate.  Self-signed certificates are used to convey a public
   key for use to begin certification paths.  End entity certificates
   are issued to subjects that are not authorized to issue certificates.

that the term "self-issued" is only applicable to CAs, not end-entity
certificates.  In https://tools.ietf.org/html/rfc5280#section-4.2.1.9
the description of path length constraints says:

   The pathLenConstraint field is meaningful only if the cA boolean is
   asserted and the key usage extension, if present, asserts the
   keyCertSign bit (Section 4.2.1.3).  In this case, it gives the
   maximum number of non-self-issued intermediate certificates that may
   follow this certificate in a valid certification path.  (Note: The
   last certificate in the certification path is not an intermediate
   certificate, and is not included in this limit.  Usually, the last
   certificate is an end entity certificate, but it can be a CA
   certificate.)

This makes it clear that exclusion of self-issued certificates from
the path length count applies only to some *intermediate* CA
certificates.  A leaf certificate whether it has identical issuer
and subject or whether it is a CA or not is never part of the
intermediate certificate count.  The handling of all leaf certificates
must be the same, in the case of our code to post-increment the
path count by 1, so that we ultimately reach a non-self-issued
intermediate it will be the first one (not zeroth) in the chain
of intermediates.

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit ed422a2d0196ada0f5c1b6e296f4a4e5ed69577f)

rsa/rsa_ossl.c: fix and extend commentary [skip ci].

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/7123)

(cherry picked from commit d1c008f66bad435b18aa45aa59f72bed7c682849)

util/domd: omit superfluous shift in -MD handling.

While reviewing last modification in GH#6261 Richard actually spotted
the inconsistency, but withdrew the remark, correct one in aftermath...

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7298)

Access `group->mont_data` conditionally in EC_GROUP_set_generator()

It appears that, in FIPS mode, `ec_precompute_mont_data()` always failed
but the error was ignored until commit e3ab8cc from #6810.

The actual problem lies in the fact that access to the `mont_data` field
of an `EC_GROUP` struct should always be guarded by an
`EC_GROUP_VERSION(group)` check to avoid OOB accesses, because `group`
might come from the FIPS module, which does not define the `mont_data`
field inside the EC_GROUP structure.

This commit adds the required check before any access to
`group->mont_data` in `EC_GROUP_set_generator()`.

Fixes #7127

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7135)

Merge 1.0.2 setuid calls to getenv(3) safety.

Manual merge of #7047 to 1.0.2-stable.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7300)

Use gethostbyname_r if available

Fixes #7228

The function BIO_get_host_ip uses gethostbyname, which is not thread safe
and hence we grab a lock. In multi-threaded applications, this lock sometimes
causes performance bottlenecks.
This patch uses the function gethostbyname_r (thread safe version), when
available.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7250)

Document OPENSSL_VERSION_TEXT macro

This commit documents the OPENSSL_VERSION_TEXT which is currently
missing in the man page.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7301)

(cherry picked from commit 7c69495712e3dc9aa8db38271f0c3faeb2037165)

crypto/bn/asm/x86_64-gcc.c: remove unnecessary redefinition of BN_ULONG

This module includes bn.h via other headers, so it picks up the
definition from there and doesn't need to define them locally (any
more?).  Worst case scenario, the redefinition may be different and
cause all sorts of compile errors.

Fixes #7227

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7287)

(cherry picked from commit dda5396aaec315bdbcb080e42fb5cd0191f2ad72)

drbg_get_entropy: force a reseed before calling ssleay_rand_bytes()

Fixes #7240

In FIPS mode, the default FIPS DRBG uses the drbg_get_entropy()
callback to reseed itself, which is provided by the wrapping
libcrypto library. This callback in turn uses ssleay_rand_bytes()
to generate random bytes.

Now ssleay_rand_bytes() calls RAND_poll() once on first call to
seed itself, but RAND_poll() is never called again (unless the
application calls RAND_poll() explicitely). This implies that
whenever the DRBG reseeds itself (which happens every 2^14
generate requests) this happens without obtaining fresh random
data from the operating system's entropy sources.

This patch forces a reseed from system entropy sources on every
call to drbg_get_entropy(). In contrary to the automatic reseeding
of the DRBG in master, this reseeding does not break applications
running in a chroot() environment (see c7504aeb640a), because the
SSLEAY PRNG does not maintain an error state. (It does not even
check the return value of RAND_poll() on its instantiation.)

In the worst case, if no random device is available for reseeding,
no fresh entropy will be added to the SSLEAY PRNG but it will happily
continue to generate random bytes as 'entropy' input for the DRBG's
reseeding, which is just as good (or bad) as before this patch.

To prevent ssleay_rand_bytes_from_system() (and hence RAND_poll())
from being called twice during instantiation, a separate
drbg_get_nonce() callback has been introduced, which is identical
with the previous implementation of drbg_get_entropy().

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/7259)

crypto/ui/ui_openssl.c: make sure to recognise ENXIO and EIO too

These both indicate that the file descriptor we're trying to use as a
terminal isn't, in fact, a terminal.

Fixes #7271

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7272)

(cherry picked from commit 276bf8620ce35a613c856f2b70348f65ffe94067)
(cherry picked from commit ad1730359220cef5903d16c7f58b602fc3713414)

Make the config script fail with an error code if Configure failed

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7166)

(cherry picked from commit e13dc23cc0fd64c304c25a67d5fa516a77f9e8f1)

CAPI engine: add support for RSA_NO_PADDING

Since the SSL code started using RSA_NO_PADDING, the CAPI engine became
unusable.  This change fixes that.

Fixes #7131

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7132)

Check the return value from ASN1_INTEGER_set

Found by Coverity

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7169)

(cherry picked from commit 512d811719fc955f574090af4c3586a9aba46fa7)

openssl req: don't try to report bits

With the introduction of -pkeyopt, the number of bits may change
without |newkey| being updated.  Unfortunately, there is no API to
retrieve the information from a EVP_PKEY_CTX either, so chances are
that we report incorrect information.  For the moment, it's better not
to try to report the number of bits at all.

Fixes #7086

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7096)

(cherry picked from commit 17147181bd3f97c53592e2a5c9319b854b954039)

VMS: add missing x509_time test to test scripts

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7112)

Clarify the EVP_DigestSignInit docs

They did not make it clear how the memory management works for the |pctx|
parameter.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7043)

Fix example in crl(1) man page

The default input format is PEM, so explicit "-inform DER" is needed to
read DER-encoded CRL.

CLA: trivial

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7094)

(cherry picked from commit 785e614a95a134831f213749332bcf40c4920f69)
(cherry picked from commit e25fc6b5b2b99ed02f8966192c94c820b6f69add)

The req documentation incorrectly states that we default to md5

Just remove that statement. It's not been true since 2005.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/6906)

Check the return from BN_sub() in BN_X931_generate_Xpq().

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7088)

(cherry picked from commit 6bcfcf16bf6aef4f9ec267d8b86ae1bffd8deab9)

Remove redundant ASN1_INTEGER_set call

This trivial patch removes a duplicated call to ASN1_INTEGER_set.

Fixes Issue #6977

Signed-off-by: Eric Brown <browne@vmware.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/6984)

(cherry picked from commit 59701e6363531cddef5b2114c0127b8453deb1f3)

bn/bn_lib.c: conceal even memmory access pattern in bn2binpad.

(cherry picked from commit 324b95605225410763fe63f7cff36eb46ca54ee9)

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/6942)

bn/bn_blind.c: use Montgomery multiplication when possible.

(cherry picked from commit e02c519cd32a55e6ad39a0cfbeeda775f9115f28)

Resolved conflicts:
crypto/bn/bn_blind.c

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/6942)

rsa/rsa_eay.c: implement variant of "Smooth CRT-RSA."

In [most common] case of p and q being of same width, it's possible to
replace CRT modulo operations with Montgomery reductions. And those are
even fixed-length Montgomery reductions...

(cherry picked from commit 41bfd5e7c8ac3a0874a94e4d15c006ad5eb48e59)

Resolved conflicts:
crypto/rsa/rsa_eay.c

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/6942)

crypto/bn: add more fixed-top routines.

Add bn_mul_fixed_top, bn_from_mont_fixed_top, bn_mod_sub_fixed_top.
Switch to bn_{mul|sqr}_fixed_top in bn_mul_mont_fixed_top and remove
memset in bn_from_montgomery_word.

(cherry picked from commit fcc4ee09473cac511eca90faa003661c7786e4f9)

Resolved conflicts:
crypto/bn/bn_mod.c
crypto/bn_int.h

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/6942)

document the -no_ecdhe option in s_server man page

the option is provided in the -help message of the s_server utility
but it is not documented in the man page, this fixes it

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/7046)

x509v3/v3_purp.c: resolve Thread Sanitizer nit.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6916)

(cherry picked from commit 0da7358b0757fa35f2c3a8f51fa036466ae50fd7)

Resolved conflicts:
crypto/x509v3/v3_purp.c

Prepare for 1.0.2q-dev

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prepare for 1.0.2p release

Reviewed-by: Richard Levitte <levitte@openssl.org>

make update

Reviewed-by: Richard Levitte <levitte@openssl.org>

i2d_ASN1_BOOLEAN(): correct error module

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6957)

Update copyright year

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6955)

Updates to CHANGES and NEWS for the new release

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6951)

i2d_ASN1_BOOLEAN(): allocate memory if the user didn't provide a buffer

Just as was done recently for i2d_ASN1_OBJECT, we also make
i2d_ASN1_BOOLEAN comply with the documentation.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6943)

i2d_ASN1_OBJECT(): allocate memory if the user didn't provide a buffer

Since 0.9.7, all i2d_ functions were documented to allocate an output
buffer if the user didn't provide one, under these conditions (from
the 1.0.2 documentation):

    For OpenSSL 0.9.7 and later if B<*out> is B<NULL> memory will be
    allocated for a buffer and the encoded data written to it. In this
    case B<*out> is not incremented and it points to the start of the
    data just written.

i2d_ASN1_OBJECT was found not to do this, and would crash if a NULL
output buffer was provided.

Fixes #6914

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6918)

(cherry picked from commit 6114041540d8d1fecaf23a861788c3c742d3b467)

rsa/*: switch to BN_bn2binpad.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6889)

(cherry picked from commit 582ad5d4d9b7703eb089016935133e3a18ea8205)

Resolved conflicts:
crypto/rsa/rsa_ossl.c
crypto/rsa/rsa_pk1.c

bn/bn_lib.c address Coverity nit in bn2binpad.

It was false positive, but one can as well view it as readability issue.
Switch even to unsigned indices because % BN_BYTES takes 4-6 instructions
with signed dividend vs. 1 (one) with unsigned.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6889)

(cherry picked from commit 83e034379fa3f6f0d308ec75fbcb137e26154aec)

bn/bn_lib.c: add computationally constant-time bn_bn2binpad.

"Computationally constant-time" means that it might still leak
information about input's length, but only in cases when input
is missing complete BN_ULONG limbs. But even then leak is possible
only if attacker can observe memory access pattern with limb
granularity.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6889)

(cherry picked from commit 89d8aade5f4011ddeea7827f08ec544c914f275a)

Resolved conflicts:
crypto/bn/bn_lib.c

Make EVP_PKEY_asn1_new() stricter with its input

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6881)

CHANGES: mention blinding reverting in ECDSA.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

ecdsa/ecs_ossl.c: switch to fixed-length Montgomery multiplication.

(back-ported from commit 37132c9702328940a99b1307f742ab094ef754a7)

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

Fix BN_gcd errors for some curves

Those even order that do not play nicely with Montgomery arithmetic

(back-ported from commit 3a6a4a93518fbb3d96632bfdcb538d340f29c56b)

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

bn/bn_mod.c: harmonize BN_mod_add_quick with original implementation.

New implementation failed to correctly reset r->neg flag. Spotted by
OSSFuzz.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

(cherry picked from commit 70a579ae2f37437a1e02331eeaa84e1b68ba021e)

ecdsa/ecs_ossl.c: revert blinding in ECDSA signature.

Originally suggested solution for "Return Of the Hidden Number Problem"
is arguably too expensive. While it has marginal impact on slower
curves, none to ~6%, optimized implementations suffer real penalties.
Most notably sign with P-256 went more than 2 times[!] slower. Instead,
just implement constant-time BN_mod_add_quick.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

(cherry picked from commit 3fc7a9b96cbed0c3da6f53c08e34d8d0c982745f)

Resolved onflicts:
crypto/ec/ecdsa_ossl.c
crypto/include/internal/bn_int.h

bn/bn_{mont|exp}.c: switch to zero-padded intermediate vectors.

Note that exported functions maintain original behaviour, so that
external callers won't observe difference. While internally we can
now perform Montogomery multiplication on fixed-length vectors, fixed
at modulus size. The new functions, bn_to_mont_fixed_top and
bn_mul_mont_fixed_top, are declared in bn_int.h, because one can use
them even outside bn, e.g. in RSA, DSA, ECDSA...

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

(cherry picked from commit 71883868ea5b33416ae8283bcc38dd2d97e5006b)

Resolved conflicts:
crypto/bn/bn_exp.c
crypto/bn/bn_lcl.h
crypto/bn/bn_mont.c
crypto/include/internal/bn_int.h

bn/bn_lib.c: add BN_FLG_FIXED_TOP flag.

The new flag marks vectors that were not treated with bn_correct_top,
in other words such vectors are permitted to be zero padded. For now
it's BN_DEBUG-only flag, as initial use case for zero-padded vectors
would be controlled Montgomery multiplication/exponentiation, not
general purpose. For general purpose use another type might be more
appropriate. Advantage of this suggestion is that it's possible to
back-port it...

bn/bn_div.c: fix memory sanitizer problem.
bn/bn_sqr.c: harmonize with BN_mul.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

(cherry picked from commit 305b68f1a2b6d4d0aa07a6ab47ac372f067a40bb)

Resolved conflicts:
crypto/bn/bn_lcl.h
crypto/bn/bn_lib.c

bn/bn_mont.c: improve readability of post-condition code.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

(cherry picked from commit 6c90182a5f87af1a1e462536e7123ad2afb84c43)

bn/bn_lib.c: remove bn_check_top from bn_expand2.

Trouble is that addition is postponing expansion till carry is
calculated, and if addition carries, top word can be zero, which
triggers assertion in bn_check_top.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6810)

(cherry picked from commit e42395e637c3507b80b25c7ed63236898822d2f1)

Resolved conflicts:
crypto/bn/bn_lib.c

Fix inconsistent use of bit vs bits

Reviewed-by: Tim Hudson <tjh@openssl.org>
GH: #6794
(cherry picked from commit b9e54e98066c1ff8adab5d68b6c114b14d2f74e5)

Make number of Miller-Rabin tests for a prime tests depend on the security level of the prime

The old numbers where all generated for an 80 bit security level. But
the number should depend on security level you want to reach. For bigger
primes we want a higher security level and so need to do more tests.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
GH: #6075
Fixes: #6012
(cherry picked from commit feac7a1c8be49fbcb76fcb721ec9f02fdd91030e)

Change the number of Miller-Rabin test for DSA generation to 64

This changes the security level from 100 to 128 bit.
We only have 1 define, this sets it to the highest level supported for
DSA, and needed for keys larger than 3072 bit.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
GH: #6075
(cherry picked from commit 74ee379651fb2bb12c6f7eb9fa10e70be89ac7c8)

Check for failures, to avoid memory leak

Thanks to Jiecheng Wu, Zuxing Gu for the report.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6791)

(cherry picked from commit 037241bf046be8cfc7e9216959393dd20b06fc21)
(Only the EC part)

ec/ecp_nistz256.c: fix ecp_nistz256_set_from_affine.

ecp_nistz256_set_from_affine is called when application attempts to use
custom generator, i.e. rarely. Even though it was wrong, it didn't
affect point operations, they were just not as fast as expected.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6761)

PKCS12: change safeContentsBag from a SET OF to a SEQUENCE OF

As per RFC 7292.

Fixes #6665

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/6708)

(cherry picked from commit b709babbca0498cd2b05f543b09f57f4a670298e)

Documentation typo fix in BN_bn2bin.pod

Change the description for BN_hex2bn() so that it uses the same BIGNUM argument name as its prototype.

CLA: trivial

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6712)

Don't create an invalid CertificateRequest

We should validate that the various fields we put into the
CertificateRequest are not too long. Otherwise we will construct an
invalid message.

Fixes #6609

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6629)

Zero-fill IV by default.

Fixes uninitialized memory read reported by Nick Mathewson

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6603)
(cherry picked from commit 10c3c1c1ec41ce16e51b92bb18fab92d1a42b49c)

Fix some more gcc-9 warnings [-Wstringop-truncation]

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6586)

Fix a new gcc-9 warning [-Wstringop-truncation]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6581)

(cherry picked from commit dc6c374bdb4872f6d5d727e73a2ed834e972842c)

Improve the example getpass() implementation to show an error return

Also, modernize the code, so that it isn't trying to store a size_t
into an int, and then check the int's sign. :/

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6271)

(cherry picked from commit c8c250333cd254ab3f4d709ebc5ed86a7c065721)
(cherry picked from commit 50d06d1c7d2682b0042e921a76beb509d7ea68e1)

doc/crypto/pem.pod: modernise the example code

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6552)

Update documentation for PEM callback: error is now -1.

In previous versions of OpenSSL, the documentation for PEM_read_*
said:

   The callback B<must> return the number of characters in the
   passphrase or 0 if an error occurred.

But since c82c3462267afdbbaa5, 0 is now treated as a non-error
return value.  Applications that want to indicate an error need to
return -1 instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6271)

(cherry picked from commit bbbf752a3c8b5a966bcb48fc71a3dc03832e7b27)
(cherry picked from commit e4b47f7f19392e3be604e44f6999de2bc9e7ecf3)

Add blinding to a DSA signature

This extends the recently added ECDSA signature blinding to blind DSA too.

This is based on side channel attacks demonstrated by Keegan Ryan (NCC
Group) for ECDSA which are likely to be able to be applied to DSA.

Normally, as in ECDSA, during signing the signer calculates:

s:= k^-1 * (m + r * priv_key) mod order

In ECDSA, the addition operation above provides a sufficient signal for a
flush+reload attack to derive the private key given sufficient signature
operations.

As a mitigation (based on a suggestion from Keegan) we add blinding to
the operation so that:

s := k^-1 * blind^-1 (blind * m + blind * r * priv_key) mod order

Since this attack is a localhost side channel only no CVE is assigned.

This commit also tweaks the previous ECDSA blinding so that blinding is
only removed at the last possible step.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6524)

Fix no-ssl3-method in 1.0.2

Fixes #5322

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6510)

sha/asm/sha{1|256}-586.pl: harmonize clang version detection.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6499)

(cherry picked from commit b55e21b357902959ae8ec0255952402f5ccaa515)

bn/asm/rsaz-avx2.pl: harmonize clang version detection.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6499)

(cherry picked from commit 9e97f61dec312084abe03226e5c962d818c9fc2b)

Warn against nonce reuse in DSA_sign_setup() doc

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6465)

Add blinding to an ECDSA signature

Keegan Ryan (NCC Group) has demonstrated a side channel attack on an
ECDSA signature operation. During signing the signer calculates:

s:= k^-1 * (m + r * priv_key) mod order

The addition operation above provides a sufficient signal for a
flush+reload attack to derive the private key given sufficient signature
operations.

As a mitigation (based on a suggestion from Keegan) we add blinding to
the operation so that:

s := k^-1 * blind^-1 (blind * m + blind * r * priv_key) mod order

Since this attack is a localhost side channel only no CVE is assigned.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reject excessively large primes in DH key generation.

CVE-2018-0732

Signed-off-by: Guido Vranken <guidovranken@gmail.com>
(cherry picked from commit 91f7361f47b082ae61ffe1a7b17bb2adf213c7fe)

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6457)

bn/asm/sparcv9-mont.pl: iron another glitch in squaring code path.

This module is used only with odd input lengths, i.e. not used in normal
PKI cases, on contemporary processors. The problem was "illuminated" by
fuzzing tests.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6440)

(cherry picked from commit f55ef97b5c0f8559f393b72ebd4b2de32ad6d231)

Document failure return of ECDSA_SIG_new

ECDSA_SIG_new() returns NULL on error.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6398)
(cherry picked from commit 6da34cfbddede5e46f9c9183b724c99999dcfb41)

ENGINE_pkey_asn1_find_str(): don't assume an engine implements ASN1 method

Just because an engine implements algorithm methods, that doesn't mean
it also implements the ASN1 method.  Therefore, be careful when looking
for an ASN1 method among all engines, don't try to use one that doesn't
exist.

Fixes #6381

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6383)

apps: when the 'compat' nameopt has been set, leave it be

XN_FLAG_COMPAT has a unique property, its zero for value.  This means
it needs special treatment; if it has been set (which can only be
determined indirectly) and set alone (*), no other flags should be
set.

(*) if any other nameopt flag has been set by the user, compatibility
mode is blown away.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6382)

(cherry picked from commit 3190d1dca43ecfd748c06aa06752de06af3768b9)

The result of a ^ 0 mod -1 is 0 not 1

Thanks to Guido Vranken and OSSFuzz for finding this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6355)

(cherry picked from commit 4aa5b725d549b3ebc3a4f2f1c44e44a11f68752b)

Fix a bogus warning about an uninitialised var

The variable in question can never be used uninitialised, but we silence
the compiler anyway.

Fixes #6301

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6329)

Fix undefined behaviour in X509_NAME_cmp()

If the lengths of both names is 0 then don't attempt to do a memcmp.

Issue reported by Simon Friedberger, Robert Merget and Juraj Somorovsky.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6291)

(cherry picked from commit 511190b691183a1fb160e7e05e2974dc73cab0c6)

util/domd: harmonize with compiler detection in Configure.

Configure detects if compiler can generate dependency lists for any
command name, while this script didn't quite tolerate arbitrary one.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6261)

Make BN_GF2m_mod_arr more constant time

Experiments have shown that the lookup table used by BN_GF2m_mod_arr
introduces sufficient timing signal to recover the private key for an
attacker with access to cache timing information on the victim's host.
This only affects binary curves (which are less frequently used).

No CVE is considered necessary for this issue.

The fix is to replace the lookup table with an on-the-fly calculation of
the value from the table instead, which can be performed in constant time.

Thanks to Youngjoo Shin for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6270)

(cherry picked from commit b336ce57f2d5cca803a920d2a9e622b588cead3c)

When configuring 'no-comp', zlib support should be disabled too

Fixes #6241

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6248)

UI console: Restore tty settings, do not force ECHO after prompt

The Console UI method always set echo on after prompting without
echo.  However, echo might not have been on originally, so just
restore the original TTY settings.

Fixes #2373

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6158)

Fix memory leaks in CA related functions.

(cherry picked from commit aebd0e5ca12d1ba0b229a4121a54afa5ea2d8aa1)

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6238)

Don't memcpy the contents of an empty fragment

In DTLS if we have buffered a fragment for a zero length message (e.g.
ServerHelloDone) then, when we unbuffered the fragment, we were attempting
to memcpy the contents of the fragment which is zero length and a NULL
pointer. This is undefined behaviour. We should check first whether we
have a zero length fragment.

Fixes a travis issue.

[extended tests]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6225)

In cases where we ask PEM_def_callback for minimum 0 length, accept 0 length

Fixes #4716

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6173)

(cherry picked from commit c82c3462267afdbbaa53e11da0508ce4e03c02b3)

Fix typo 'is an error occurred' in documentation

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6231)

X509 time: tighten validation per RFC 5280

- Reject fractional seconds
- Reject offsets
- Check that the date/time digits are in valid range.
- Add documentation for X509_cmp_time

GH issue 2620

Backported from 80770da39e

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6182)

BIO_s_mem() write: Skip early when input length is zero

When the input length is zero, just return zero early.  Otherwise,
there's a small chance that memory allocation is engaged, fails and
returns -1, which is a bit confusing when nothing should be written.

Fixes #4782 #4827

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/6175)

(cherry picked from commit 0d94212a046e87fafea6e83e8ea2b2a58db49979)

bn/asm/*-mont.pl: harmonize with BN_from_montgomery_word.

Montgomery multiplication post-conditions in some of code paths were
formally non-constant time. Cache access pattern was result-neutral,
but a little bit asymmetric, which might have produced a signal [if
processor reordered load and stores at run-time].

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6163)

(cherry picked from commit 774ff8fed67e19d4f5f0df2f59050f2737abab2a)

Resolved conflicts in ppc-mont.pl and x86_64-mont.pl.

(cherry picked from commit d067ba3bcc4944140578bdbf857affa6faf9e8c1)

v3_purp.c: add locking to x509v3_cache_extensions()

Fixes #6121

Thanks to Mingtao Yang for reporting this bug.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6164)

Configure: fix Mac OS X builds that still require makedepend

Earlier Apple Xcode compilers, e.g. one targeting Mac OS X 10.7, don't
support dependency generation and one still has to use makedepend.  It's
unclear when it was fixed, but all clang-based Apple compilers seem to
support -M options.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6073)

Add some documentation for SSL_get_shared_ciphers()

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6115)

Fix comment in ssl.h

The ciphers field in a session contains the stack of ciphers offered by
the client.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6115)

Fix SSL_get_shared_ciphers()

The function SSL_get_shared_ciphers() is supposed to return ciphers shared
by the client and the server. However it only ever returned the client
ciphers.

Fixes #5317

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6115)

a_strex.c: prevent out of bound read in do_buf()

which is used for ASN1_STRING_print_ex*() and X509_NAME_print_ex*().

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6118)

Clarify BN_mod_exp docs

Specifically this is not supported with an even modulus and
BN_FLG_CONSTTIME.

Fixes #5082

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6138)

(cherry picked from commit 966b3df182b9c8644c154f1c3e686027d37acb05)

Fix some errors and missing info in the CMS docs

Fixes #5063

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6136)

Update version docs

Make it clear that you should not attempt to get the version before the
first handshake is complete.

Fixes #2893

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6015)

Skip the CMS test if configured 'no-cms'

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6108)

fix: BN_swap mishandles flags (1.0.2)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6102)

s_server: fix warnings unused-but-set-variable (no-dh)

This patch fixes the following two warnings when OpenSSL is built with no-dh option:

s_server.c: In function 's_server_main':
s_server.c:1105:25: warning: variable 'no_dhe' set but not used [-Wunused-but-set-variable]
     int no_tmp_rsa = 0, no_dhe = 0, no_ecdhe = 0, nocert = 0;
                         ^
s_server.c:1101:11: warning: variable 'dhfile' set but not used [-Wunused-but-set-variable]
     char *dhfile = NULL;
           ^
CLA: trivial
Signed-off-by: Cristian Stoica <cristian.stoica@nxp.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6087)

fix warning unused-but-set-variable 'alg_k' (no-dh and no-ec)

This patch fixes the following warning when OpenSSL is configured with
no-dh and no-ec:

./Configure no-ec no-dh linux-x86_64

...
s3_lib.c: In function 'ssl3_get_req_cert_type':
s3_lib.c:4234:19: warning: variable 'alg_k' set but not used [-Wunused-but-set-variable]
     unsigned long alg_k;

CLA: trivial
Signed-off-by: Cristian Stoica <cristian.stoica@nxp.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6087)