Document that HMAC() with a NULL md is not thread safe

Fixes #3541

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3554)

Prepare for 1.1.0g-dev

Reviewed-by: Stephen Henson <steve@openssl.org>

Prepare for 1.1.0f release

Reviewed-by: Stephen Henson <steve@openssl.org>

Update CHANGES and NEWS for new release

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3547)

Document that PKCS#12 functions assume UTF-8 for passwords

Part of issue #3531

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3535)
(cherry picked from commit cff85f39e438830cb5bc8bf7cb2ed6458670c5be)

Clarify what character encoding is used in the returned UI strings

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3532)
(cherry picked from commit 789d6dddecbc231f5c858d203aab318cf81676fd)

Fix typo in doc/man3/EVP_EncrypInit.pod

In the example section.

CLA: trivial

Signed-off-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3520)
(cherry picked from commit 719b289d62d32fe45226e8bc5b4fb0d76f6a1b5d)

[1.1.0 backport] set entry type on SCTs from X.509 and OCSP extensions

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3519)

Fix infinite loops in secure memory allocation.

Remove assertion when mmap() fails.
Only give the 1<<31 limit test as an example.

Fix the small arena test to just check for the symptom of the infinite
loop (i.e. initialized set on failure), rather than the actual infinite
loop. This avoids some valgrind errors.

Backport of:
PR #3512 commit fee423bb68869de02fceaceefbc847e98213574b
PR #3510 commit a486561b691d6293a901b412172ca0c6d1ffc0dc
PR #3455 commit c8e89d58a5d44b9dd657d6d13a5a10d1d4d30733
PR #3449 commit 7031ddac94d0ae616d1b0670263a9265ce672cd2

Issue 1:

sh.bittable_size is a size_t but i is and int, which can result in
freelist == -1 if sh.bittable_size exceeds an int.

This seems to result in an OPENSSL_assert due to invalid allocation
size, so maybe that is "ok."

Worse, if sh.bittable_size is exactly 1<<31, then this becomes an
infinite loop (because 1<<31 is a negative int, so it can be shifted
right forever and sticks at -1).

Issue 2:

CRYPTO_secure_malloc_init() sets secure_mem_initialized=1 even when
sh_init() returns 0.

If sh_init() fails, we end up with secure_mem_initialized=1 but
sh.minsize=0. If you then call secure_malloc(), which then calls,
sh_malloc(), this then enters an infite loop since 0 << anything will
never be larger than size.

Issue 3:

That same sh_malloc loop will loop forever for a size greater
than size_t/2 because i will proceed (assuming sh.minsize=16):
i=16, 32, 64, ..., size_t/8, size_t/4, size_t/2, 0, 0, 0, 0, ....
This sequence will never be larger than "size".

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3453)

Fix srp app missing NULL termination with password callback

The password_callback() function does not necessarily NULL terminate
the password buffer, the caller must use the returned length but the
srp app uses this function as if it was doing NULL termination.

This made the -passin and -passout options of "openssl srp"
fail inexpicably and randomly or even crash.

Fixed by enlarging the buffer by one, so that the maximum password length
remains unchanged, and adding NULL termination upon return.

[Rearrange code for coding style compliance in process.]

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3475)

(cherry picked from commit 0e83981d61fc435f42d4bb4d774272b69556b7bc)

test/run_tests.pl: don't mask test failures.

Switch to TAP::Harness inadvertently masked test failures.
Test::Harness::runtests was terminating with non-zero exit code in case
of failure[s], while TAP::Harness apparently holds caller responsible
for doing so.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

-inkey can be an identifier, not just a file

update pkcs12, smime, ts apps.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3507)
(cherry picked from commit 48b5352212d8c68f7fd071ca9f38822b7e954c5a)

Fix endless loop on srp app when listing users

With the -list option the srp app loops on the main while() endlessly,
whether users were given on the command line or not. The loop should
be stopped when in list mode and there are no more users.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3489)

(cherry picked from commit 5ec3210fc106ecc6badb48db6cfb1493a7607763)

Fix ASN1_TIME_to_generalizedtime to take a const ASN1_TIME

Fixes #1526

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3360)
(cherry picked from commit 9bfeeef8ee2220339e601a028fa991c30d296ed4)

Make SSL_is_server() accept a const SSL

Fixes #1526

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3360)
(cherry picked from commit 6944311688015ad293bd788ce78f3226738ebf00)

Remove notification settings from appveyor.yml

Notifications can be (and should be) configured on account basis on
the CI web site.  This avoids getting emails to openssl-commits for
personal accounts that also build OpenSSL stuff.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3484)
(cherry picked from commit 7a94f5b0f7c878b1056a08f659ce23aa97bfa3ad)

INSTALL: Remind people to read more if they added configuration options

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3469)
(cherry picked from commit 62f218cb8d31851935b8113a2a2236493b3510cc)

INSTALL: clarify a bit more how Configure treats "unknown" options

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3468)
(cherry picked from commit a3cb4cfc6af3f5fc1cd81ccd264daaa79d1c0a46)

Clean up SSL_OP_* a bit

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3439)
(cherry picked from commit 80a2fc4100daf6f1001eee33ef2f9b9eee05bedf)

Clean away needless VMS check

BIO_socket_ioctl is only implemented on VMS for VMS version 7.0 and
up, but since we only support version 7.1 and up, there's no need to
check the VMS version.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3448)
(cherry picked from commit b57f0c598bde43e147a886c9ffb0d6fdb3141d72)

Cleanup - use e_os2.h rather than stdint.h

Not exactly everywhere, but in those source files where stdint.h is
included conditionally, or where it will be eventually

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3447)
(cherry picked from commit 74a011ebb5e9028ef18982d737a434a8ff926a95)

Fix strict-warnings build

crypto/asn1/a_strex.c: Type of width variable in asn1_valid_host
function  needs to be changed from char to signed char to avoid
build error due to '-Werror=type-limits'.

Signed-off-by: Patrick Steuer <psteuer@mail.de>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
CLA: trivial
(cherry picked from commit 34657a8da2ead453460d668771984432cc767044)

Fix regression in openssl req -x509 behaviour.

Allow conversion of existing requests to certificates again.
Fixes the issue #3396

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3437)
(cherry picked from commit 888adbe064556ff5ab2f1d16a223b0548696614c)

Remove dead code.

The second BN_is_zero test can never be true.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3434)
(cherry picked from commit 3f97052392cb10fca5309212bf720685262ad4a6)

Ignore MSVC warnings (via Gisle Vanem)

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3420)
(cherry picked from commit 05004f366ecd6b4fcc968739648be9b7fd87df3f)

Add a test for SNI in conjunction with custom extensions

Test that custom extensions still work even after a change in SSL_CTX due
to SNI. See #2180.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3426)

Copy custom extension flags in a call to SSL_set_SSL_CTX()

The function SSL_set_SSL_CTX() can be used to swap the SSL_CTX used for
a connection as part of an SNI callback. One result of this is that the
s->cert structure is replaced. However this structure contains information
about any custom extensions that have been loaded. In particular flags are
set indicating whether a particular extension has been received in the
ClientHello. By replacing the s->cert structure we lose the custom
extension flag values, and it appears as if a client has not sent those
extensions.

SSL_set_SSL_CTX() should copy any flags for custom extensions that appear
in both the old and the new cert structure.

Fixes #2180

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3426)

Prefer TAP::Harness over Test::Harness

TAP:Harness came along in perl 5.10.1, and since we claim to support
perl 5.10.0 in configuration and testing, we can only load it
conditionally.

The main reason to use TAP::Harness rather than Test::Harness is its
capability to merge stdout and stderr output from the test recipes,
which Test::Harness can't.  The merge gives much more comprehensible
output when testing verbosely.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3424)
(cherry picked from commit 76e0d0b21cc4e8a879d54f4d78a392826dadb1d1)

Remove support for OPENSSL_SSL_TRACE_CRYPTO

This trace option does not appear in Configure as a separate option and is
undocumented. It can be switched on using "-DOPENSSL_SSL_TRACE_CRYPTO",
however this does not compile in master or in any 1.1.0 released version.

(cherry picked from commit eee2750bd3d25265bb44d029877434d2cc80970c)

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3413)

Added a new Makefile in demos/evp directory
Fixed compilation warning in file aesgcm.c

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3406)
(cherry picked from commit d396da33130aba2e77478d00fd369eb8d34bd8bf)

sha/sha512.c: fix formatting.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit ce1932f25f784bc5df3505c5de8b6b53436202a3)

perlasm/x86_64-xlate.pl: work around problem with hex constants in masm.

Perl, multiple versions, for some reason occasionally takes issue with
letter b[?] in ox([0-9a-f]+) regex. As result some constants, such as
0xb1 came out wrong when generating code for MASM. Fixes GH#3241.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3385)

(cherry picked from commit c47aea8af1e28e46e1ad5e2e7468b49fec3f4f29)

Fix pathname errors in errcode file

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3388)
(cherry picked from commit 1d3235f85c4b0e51b6baf7d8b89089c6c77f6928)

Don't leave stale errors on queue if DSO_dsobyaddr() fails

The init code uses DSO_dsobyaddr() to leak a reference to ourselves to
ensure we remain loaded until atexit() time. In some circumstances that
can fail and leave stale errors on the error queue.

Fixes #3372

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3383)
(cherry picked from commit 689f112d9806fa4a0c2f8c108226639455bc770d)

Convert uses of snprintf to BIO_snprintf

Fixes #2360
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3366)

(cherry picked from commit c41048ffe359ae18cb9c8f840ca970e367d97c37)

Fix some error path logic in i2v_AUTHORITY_INFO_ACCESS and i2v_GENERAL_NAME

Fixes #1653 reported by Guido Vranken

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3362)
(cherry picked from commit 75a3e39288feeeefde5ed1f96ff9faeba0d2b233)

Fix URL links in comment

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3365)
(cherry picked from commit dea0eb2c5452cd4c2160a64a6868e79efeca6e9d)

Fix time offset calculation.

ASN1_GENERALIZEDTIME and ASN1_UTCTIME may be specified using offsets,
even though that's not supported within certificates.

To convert the offset time back to GMT, the offsets are supposed to be
subtracted, not added. e.g. 1759-0500 == 2359+0100 == 2259Z.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3335)

asn1/a_int.c: fix "next negative minimum" corner case in c2i_ibuf.

"Next" refers to negative minimum "next" to one presentable by given
number of bytes. For example, -128 is negative minimum presentable by
one byte, and -256 is "next" one.

Thanks to Kazuki Yamaguchi for report, GH#3339

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit 1e93d619b78832834ae32f5c0c1b0e466267f72d)

Check fflush on BIO_ctrl call

Bug found and fix suggested by Julian Rüth.
Push error if fflush fails

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3266)
(cherry picked from commit 595b2a42375427a254ad5a8c85870efea839a9b9)

Ensure blank lines between tests.

Also add a comment describing the file format.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3337)
(cherry picked from commit 9a837f220ad1320c51ad2b57b2466fbe28670a45)

TLSProxy: When in debug mode, show the exact subprocess commands

When you want to debug a test that goes wrong, it's useful to know
exactly what subprocess commands are run.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3342)
(cherry picked from commit 46d5e2b42e00ec392bf3326743519bc25136db09)

Remove (broken) diagnostic print

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3336)
(cherry picked from commit f2150cd74f45d49545d0787725f99723642b7c2c)

Remove unnecessary loop in pkey_rsa_decrypt.

It is not necessary to remove leading zeros here because
RSA_padding_check_PKCS1_OAEP_mgf1 appends them again. As this was not done
in constant time, this might have leaked timing information.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3313)
(cherry picked from commit 237bc6c997e42295eeb32c8c1c709e6e6042b839)

CT_POLICY_EVAL_CTX_set_time expects milliseconds, but given seconds

This resulted in the SCT timestamp check always failing, because the
timestamp appeared to be in the future.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3260)

Add SSL tests for certificates with embedded SCTs

The only SSL tests prior to this tested using certificates with no
embedded Signed Certificate Timestamps (SCTs), which meant they couldn't
confirm whether Certificate Transparency checks in "strict" mode were
working.

These tests reveal a bug in the validation of SCT timestamps, which is
fixed by the next commit.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3260)

Add and use function test_pem to work out test filenames.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3260)

asn1/a_int.c: clean up asn1_get_int64.

Trouble was that integer negation wasn't producing *formally* correct
result in platform-neutral sense. Formally correct thing to do is
-(int64_t)u, but this triggers undefined behaviour for one value that
would still be representable in ASN.1. The trigger was masked with
(int64_t)(0-u), but this is formally inappropriate for values other
than the problematic one. [Also reorder branches to favour most-likely
paths and harmonize asn1_string_set_int64 with asn1_get_int64].]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3231)

(cherry picked from commit 786b6a45fbecc068d0fb8b05252a9228e0661c63)

asn1/a_int.c: don't write result if returning error.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3192)

(cherry picked from commit b997adb3a518b065240e70acf38ec5f77a937f53)

asn1/a_int.c: simplify asn1_put_uint64.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3192)

(cherry picked from commit 6d4321fc242829490e1e7a36358eb12874c9b9e0)

asn1/a_int.c: remove code duplicate and optimize branches,

i.e. reduce amount of branches and favour likely ones.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3192)

(cherry picked from commit a3ea6bf0ef703b38a656245931979c7e53c410b7)

Fix some variable references in init_client

We were incorrectly using "res" when we meant "ai"

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3287)

Fix problem with SCTP close_notify alerts

In SCTP the code was only allowing a send of a close_notify alert if the
socket is dry. If the socket isn't dry then it was attempting to save away
the close_notify alert to resend later when it is dry and then it returned
success. However because the application then thinks that the close_notify
alert has been successfully sent it never re-enters the DTLS code to
actually resend the alert. A much simpler solution is to just fail with a
retryable error in the event that the socket isn't dry. That way the
application knows to retry sending the close_notify alert.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3287)

Don't attempt to send fragments > max_send_fragment in DTLS

We were allocating the write buffer based on the size of max_send_fragment,
but ignoring it when writing data. We should fragment handshake messages
if they exceed max_send_fragment and reject application data writes that
are too large.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3287)

Remove special case code for SCTP reneg handling

There was code existing which attempted to handle the case where application
data is received after a reneg handshake has started in SCTP. In normal DTLS
we just fail the connection if this occurs, so there doesn't seem any reason
to try and work around it for SCTP. In practice it didn't work properly
anyway and is probably a bad idea to start with.

Fixes #3251

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3287)

openssl enc: Don't unbuffer stdin

 - unbuffer causes single-byte reads from stdin and poor performance

Fixes #3281
CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3299)
(cherry picked from commit 65d62488b8c808350f440d2276034f5223b391ad)

check length sanity before correcting in EVP_CTRL_AEAD_TLS1_AAD

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3290)

Typo fix in Configurations/descrip.mms.tmpl

... on the theme "I could have sworn I saved that fix!"

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3285)
(cherry picked from commit cb2a6eb76ef7304a1b76ac5e2578cd9d5f59af28)

Annotate ASN.1 attributes of the jurisdictionCountryName NID

EV Guidelines section 9.2.5 says jurisdictionCountryName follows the
same ASN.1 encoding rules as countryName.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3284)
(cherry picked from commit e8d542121b10158c1bdd5fbe02e0d17c59470cae)

Port Ben's parallell Makefile hack to VMS

As far as I know, there is no MMS / MMK with parallellism today.
However, it might be added in the future (perhaps in MMK at least), so
we may as well prepare for it now.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3282)
(cherry picked from commit 757264207ad8650a89ea903d48ad89f61d56ea9c)

Port Ben's parallell Makefile hack to Windows

jom is an nmake clone that does parallell building, via the same -j
argument as GNU make.  To make it work, we need to apply the same
dependeency build up as done in 27c40a93175d4dcb559325db9354910b3d16cd4e

Fixes #3272

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3277)
(cherry picked from commit f919c12f5c8b92f0318c650573e774fe6522c27c)

Fixed typo in X509_STORE_CTX_new description

 'X509_XTORE_CTX_cleanup' ->  'X509_STORE_CTX_cleanup'
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3271)
(cherry picked from commit 7643a1723ef8eb4e542e57dfadeeb8eff788ec05)

Numbers greater than 1 are usually non-negative.

BN_is_prime_fasttest_ex begins by rejecting if a <= 1. Then it goes to
set A := abs(a), but a cannot be negative at this point.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3275)
(cherry picked from commit 8b24f94209676bbe9933affd2879a686b1ed044d)

Document Next Protocol Negotiation APIs

Add callback function prototypes, fix description

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3084)

(cherry picked from commit 87b81496fec2f969371b3167dea3b6aaed9f9f9d)

update docs because depth refers only to intermediate certs

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3132)
(cherry picked from commit 800b5dac006344896a3aa947ab13cd9f63e3fc4c)

Catch EC_R_UNKNOWN_GROUP in check_unsupported()

If EC support is enabled we should catch also EC_R_UNKNOWN_GROUP as an hint to
an unsupported algorithm/curve (e.g. if binary EC support is disabled).

Before this commit the issue arise for example if binary EC keys are added in
evptests.txt, and the test is run when EC is enabled but EC2m is disabled.

E.g. adding these lines to evptests.txt would reproduce the issue:

~~~

PrivateKey=KAS-ECC-CDH_K-163_C0
-----BEGIN PRIVATE KEY-----
MGMCAQAwEAYHKoZIzj0CAQYFK4EEAAEETDBKAgEBBBUAZlO2B3OY+tx79eYBWBcB
SMPcRSehLgMsAAQHH4sod9YCfZwa3kJE8t6hJpLvI9UFwV7ndiIccrhLNHzjg/OA
Z7icPpo=
-----END PRIVATE KEY-----

PublicKey=KAS-ECC-CDH_K-163_C0-PUBLIC
-----BEGIN PUBLIC KEY-----
MEAwEAYHKoZIzj0CAQYFK4EEAAEDLAAEBx+LKHfWAn2cGt5CRPLeoSaS7yPVBcFe
53YiHHK4SzR844PzgGe4nD6a
-----END PUBLIC KEY-----

PublicKey=KAS-ECC-CDH_K-163_C0-Peer-PUBLIC
-----BEGIN PUBLIC KEY-----
MEAwEAYHKoZIzj0CAQYFK4EEAAEDLAAEBXQjbxQoxDITCUZ4Ols6q7bCfqXWB5CM
JRuNoCHLrCgfEj969PrFs9u4
-----END PUBLIC KEY-----

Derive=KAS-ECC-CDH_K-163_C0
PeerKey=KAS-ECC-CDH_K-163_C0-Peer-PUBLIC
Ctrl=ecdh_cofactor_mode:1
SharedSecret=04325bff38f1b0c83c27f554a6c972a80f14bc23bc

~~~

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3226)

(cherry picked from commit 0c44545c0f5781b1d2401b0d3c93c0376399c8e5)

Reformat evptests.txt

When compiling without EC support the test fails abruptly reading some keys.
Some keys merged in commit db040557c8bf4449e0a0b43510a2b30d89d58a83 start with
------BEGIN EC PRIVATE KEY-----

this format is not supported without EC support.

This commit reformat those keys with the generic format. After this change the
test simply skips the unsupported EC keys when EC is disabled, without parsing
errors.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3222)

Remove ecdhtest.c

All tests from ecdhtest.c have been ported to evptests.txt

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3218)

ecdhtest.c: move co-factor ECDH KATs to evptests

move NIST SP800-56A co-factor ECDH KATs from ecdhtest.c to evptests.txt

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3218)

ASN.1: adapt SSL_SESSION_ASN1 by explicitely embedding INTxx et al

Fixes #3191

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3200)

ASN.1: change INTxx, UINTxx and Z variants to be embedable

Fixes #3191

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3200)

ASN.1: extend the possibilities to embed data instead of pointers

Also, when "allocating" or "deallocating" an embedded item, never call
prim_new() or prim_free().  Call prim_clear() instead.

Fixes #3191

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3200)

Remove more stale code in ecdhtest.c

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3188)

ecdhtest.c: move KATs to evptests.txt

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3188)

Remove stale code in ecdhtest.c

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3188)

ecdhtest.c: move NAMED CURVES TESTS to evptests.txt

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3188)

Added error checking for OBJ_create

fixes segmentation fault in case of not enough memory for object creation

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3157)
(cherry picked from commit 487a73def65aa6ffedf5baf353e912fe61aa3c7c)

Fix x_int64.c

Clearing a misunderstanding.  The routines c2i_uint64_int() and
i2c_uint64_int() expect to receive that internal values are absolute
and with a separate sign flag, and the x_int64.c code handles values
that aren't absolute and have the sign bit embedded.  We therefore
need to convert between absolute and non-absolute values for the
encoding of negative values to be correct.

[extended tests]

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3160)
(cherry picked from commit 429223d198aabacd129cf6dde5a4203b5af41737)

Remove ECDH(E) ciphers from SSLv3

SSLv3 does not support TLS extensions, and thus, cannot provide any
curves for ECDH(E). With the removal of the default (all) list of curves
being used for connections that didn't provide any curves, ECDHE is no
longer possible.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3181)
(cherry picked from commit fe55c4a20f79c77c64a082c5df2c5e8a61317162)

Additional check to handle BAD SSL_write retry

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3123)

Reject decoding of an INT64 with a value >INT64_MAX

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3159)
(cherry picked from commit 0856e3f167964f58c26796331eab9d8b0a883921)

Allow an ALPN callback to pretend to not exist

RFC 7301 mandates that the server SHALL respond with a fatal
"no_application_protocol" alert when there is no overlap between
the client's supplied list and the server's list of supported protocols.
In commit 062178678f5374b09f00d70796f6e692e8775aca we changed from
ignoring non-success returns from the supplied alpn_select_cb() to
treating such non-success returns as indicative of non-overlap and
sending the fatal alert.

In effect, this is using the presence of an alpn_select_cb() as a proxy
to attempt to determine whether the application has configured a list
of supported protocols.  However, there may be cases in which an
application's architecture leads it to supply an alpn_select_cb() but
have that callback be configured to take no action on connections that
do not have ALPN configured; returning SSL_TLSEXT_ERR_NOACK from
the callback would be the natural way to do so.  Unfortunately, the
aforementioned behavior change also treated SSL_TLSEXT_ERR_NOACK as
indicative of no overlap and terminated the connection; this change
supplies special handling for SSL_TLSEXT_ERR_NOACK returns from the
callback.  In effect, it provides a way for a callback to obtain the
behavior that would have occurred if no callback was registered at
all, which was not possible prior to this change.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3158)
(cherry picked from commit 8313a787d770ac1d7ddafcbc41b13e7fb5841eae)

rand/rand_lib.c: keep fixing no-engine configuration.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit ce57ac431964f7692af674fd124103d6fe07af2e)

asn1/x_long.c: remove conditions in inner loops and dependency on BN.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3152)

(cherry picked from commit e128f891de71bbdba8391355af8d6d47d20b1969)

e_os.h: omit PRIu64.

PRIu64 is error-prone with BIO_printf, so introduce and stick to custom
platform-neutral macro. 'll' allows to print 64-bit values on *all*
supported platforms, but it's problematic with -Wformat -Werror. Hence
use 'l' in identifiable LP64 cases.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3148)

bio/b_print.c: drop dependency on BN config.

This might seem controversial, but it doesn't actually affect anything.
Or rather it doesn't make worse cases when it was problematic [with code
additions to 1.1.0]. One of such rare cases is 32-bit PA-RISC target
with *vendor* compiler.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3148)

Standardize on =over 4 and check for it.

(cherry picked from commit dfc63ccebd8262648640fc6e35fffa132102b967)

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3117)

Use 'over 2' for bullet lists.

(cherry picked from commit 7ea8ceab4b21d93eb272a89bff73958010d5c2c4)

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3117)

Fix rand_lib.c for no-engine configuration

When configured no-engine, we still refered to rand_engine_lock.
Rework the lock init code to avoid that.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3145)
(cherry picked from commit 2f881d2d9065342454fe352eac9e835cefa0ba90)

Don't try to clean up RAND from ENGINE

This is especially harmful since OPENSSL_cleanup() has already called
the RAND cleanup function

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3137)
(cherry picked from commit 789a2b6250d5e05dfde6ce259e79ef8c172c9f3f)

Make getting and setting the RAND default method thread safe

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3137)
(cherry picked from commit 87975cfa91c3f06a52f2550ed2ef632644be6dde)

e_os2.h: Refine OSSL_SSIZE definition under UEFI environment

Under UEFI build environment, we may encounter the OSSL_SSIZE macro
re-definition error in e_os2.h if any module call OpenSSL API directly
by including "openssl/xxxx.h" (caused by the predefined _WIN32/_WIN64
macro, which should have been un-defined under OPENSSL_SYS_UEFI).

Though it's not one recommended usage, this patch could still eliminate
the possible build issue by refining the OSSL_SSIZE definition under
OPENSSL_SYS_UEFI.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3121)

(cherry picked from commit 005f6766284155645fa760cf764f85031e209f6c)

In rand_cleanup_int(), don't go creating a default method

If no default method was yet given, RAND_get_rand_method() will set it
up.  Doing so just to clean it away seems pretty silly, so instead,
use the default_RAND_meth variable directly.

This also clears a possible race condition where this will try to init
things, such as ERR or ENGINE when in the middle of a OPENSSL_cleanup.

Fixes #3128

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3136)
(cherry picked from commit 5006b37b31c5a520c3065048bd8aba8cc3ff128d)

Configurations/README: reword bn_ops description.

Fixes GH#3116.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit 1bc563caa144a568fc505f210568f5ac5b66315e)

Fix faulty check of padding in x_long.c

Bug uncovered by test

[extended tests]

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3120)

Fix a possible integer overflow in long_c2i

Credit to OSS-Fuzz for finding this.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3120)

Convert SSL_SESSION_ASN1 to use size specific integers

This increases portability of SSL_SESSION files between architectures
where the size of |long| may vary.  Before this, SSL_SESSION files
produced on a 64-bit long architecture may break on a 32-bit long
architecture.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3120)

make update

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3120)

Implement internal ASN.1 types INT32, UINT32, INT64, UINT64

Also Z varieties.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3120)

Ensure dhparams can handle X9.42 params in DER

dhparams correctly handles X9.42 params in PEM format. However it failed
to correctly processes them when reading/writing DER format.

Fixes #3102

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3111)
(cherry picked from commit 18d20b5eb66fda0ada2e65c2b19aeae75827bdf8)