Make OPENSSL_malloc_init() a no-op

Making this a no-op removes a potential infinite loop than can occur in
some situations.

Fixes #2865

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8167)

(cherry picked from commit ef45aa14c5af024fcb8bef1c9007f3d1c115bd85)

Remove unnecessary trailing whitespace

Trim trailing whitespace. It doesn't match OpenSSL coding standards,
AFAICT, and it can cause problems with git tooling.

Trailing whitespace remains in test data and external source.

Backport-of: https://github.com/openssl/openssl/pull/8092

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8134)

Make some simple getters take const SSL/SSL_CTX

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8145)

(cherry picked from commit 3499327bad401eb510d76266428923d06c9c7bb7)

Fix Invalid Argument return code from IP_Factory in connect_to_server().

Fixes #7732

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8158)

(cherry picked from commit 66a60003719240399f6596e58c239df0465a4f70)

Android build: fix usage of NDK home variable ($ndk_var)

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8153)

(cherry picked from commit adc7e221f12462c6e10bc7c2c7afaf52490cb292)

Add an entry to the CHANGES for the d2i_X509_PUBKEY fix

The commit 5dc40a83c74be579575a512b30d9c1e0364e6a7b forgot
to add a short description to the CHANGES file.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8144)

(cherry picked from commit b2aea0e3d9a15e30ebce8b6da213df4a3f346155)

Fix end-point shared secret for DTLS/SCTP

When computing the end-point shared secret, don't take the
terminating NULL character into account.
Please note that this fix breaks interoperability with older
versions of OpenSSL, which are not fixed.

Fixes #7956

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7957)

(cherry picked from commit 09d62b336d9e2a11b330d45d4f0f3f37cbb0d674)

Fix a crash in reuse of i2d_X509_PUBKEY

If the second PUBKEY is malformed there is use after free.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8122)

(cherry picked from commit 5dc40a83c74be579575a512b30d9c1e0364e6a7b)

Fixed d2i_X509 in-place not re-hashing the ex_flags

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8116)

(cherry picked from commit 53649022509129bce8036c8fb4978dbce9432a86)

Fix a memory leak with di2_X509_CRL reuse

Additionally avoid undefined behavior with
in-place memcpy in X509_CRL_digest.

Fixes #8099

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8112)

(cherry picked from commit a727627922b8a9ec6628ffaa2054b4b3833d674b)

Better phrasing around 1.1.0

Fixes #8129

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8130)

(cherry picked from commit 62b563b9df161a992fde18a0cb0d1a0969158412)

VMS: force 'pinshared'

VMS doesn't currently support unloading of shared object, and we need
to reflect that.  Without this, the shlibload test fails

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8131)

(cherry picked from commit d1dd5d6f4c2f13478aa45557b4546febd51f0cb3)

Fix error message for s_server -psk option

Previously if -psk was given a bad key it would print "Not a hex
number 's_server'".

CLA: Trivial

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/8113)

(cherry picked from commit e57120128fa4e2afa4bda5022a77f73a1e3a0b27)

Reuse already defined macros

instead of duplicity the code.

CLA: trivial

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8127)

(cherry picked from commit c4734493d7da404b1747195a805c8d536dbe6910)

Complain if -twopass is used incorrectly

The option -twopass to the pkcs12 app is ignored if -passin, -passout
or -password is used. We should complain if an attempt is made to use
it in combination with those options.

Fixes #8107

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8114)

(cherry picked from commit 40b64553f577716cb4898895f5fd4530a6266c75)

Fix no-dso builds

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8111)

(cherry picked from commit 522b11e969cbdc82eca369512275f227080a86fa)

Don't leak memory from ERR_add_error_vdata()

If the call the ERR_set_error_data() in ERR_add_error_vdata() fails then
a mem leak can occur. This commit checks that we successfully added the
error data, and if not frees the buffer.

Fixes #8085

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/8105)

(cherry picked from commit fa6b1ee1115c1e5e3a8286d833dcbaa2c1ce2b77)

Android build: use ANDROID_NDK_HOME rather than ANDROID_NDK

It apepars that ANDROID_NDK_HOME is the recommended standard
environment variable for the NDK.

We retain ANDROID_NDK as a fallback.

Fixes #8101

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8103)

(cherry picked from commit 6e826c471b7f0431391a4e9f9484f6ea2833774a)

clarify which functions are the CMS functions which must have CMS_PARTIAL set

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7960)

(cherry picked from commit 61e033308b1c004bd808352fb1d786547dcdf62b)

crypto/bn: fix return value in BN_generate_prime

When the ret parameter is NULL the generated prime
is in rnd variable and not in ret.

CLA: trivial

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8076)

(cherry picked from commit 3d43f9c809e42b960be94f2f4490d6d14e063486)

s_client: fix not to send a command letter of R

Before 1.1.0, this command letter is not sent to a server.

CLA: trivial
(cherry picked from commit bc180cb4887c2e82111cb714723a94de9f6d2c35)

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8081)

(cherry picked from commit 5478e2100260b8d6f9df77de875f37763d8eeec6)

Remove stray -modulus option from the ec manual page.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8082)

(cherry picked from commit d7bcbfd0828616f33008e711eabc6ec00b32e87b)

Add "weak" declarations of symbols used in safestack.h and lhash.h

Only for SunCC for now.

It turns out that some compilers to generate external variants of
unused static inline functions, and if they use other external
symbols, those need to be present as well.  If you then happen to
include one of safestack.h or lhash.h without linking with libcrypto,
the build fails.

Fixes #6912

Signed-off-by: Matthias Kraft <Matthias.Kraft@softwareag.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8087)

(cherry picked from commit 6638b2214761b5f30300534e0fe522448113c6cf)

X509_STORE: fix two misspelled compatibility macros

Fixes #8084

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8086)

(cherry picked from commit 2c75f03b39de2fa7d006bc0f0d7c58235a54d9bb)

Cleanup vxworks support to be able to compile for VxWorks 7

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7569)

(cherry picked from commit 5c8b7b4caa0faedb69277063a7c6b3a8e56c6308)

Revert "Keep the DTLS timer running after the end of the handshake if appropriate"

This commit erroneously kept the DTLS timer running after the end of the
handshake. This is not correct behaviour and shold be reverted.

This reverts commit f7506416b1311e65d5c440defdbcfe176f633c50.

Fixes #7998

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8047)

(cherry picked from commit bcc1f3e2baa9caa83a0a94bd19fb37488ef3ee57)

Make sure we trigger retransmits in DTLS testing

During a DTLS handshake we may need to periodically handle timeouts in the
DTLS timer to ensure retransmits due to lost packets are performed. However,
one peer will always complete a handshake before the other. The DTLS timer
stops once the handshake has finished so any handshake messages lost after
that point will not automatically get retransmitted simply by calling
DTLSv1_handle_timeout(). However attempting an SSL_read implies a
DTLSv1_handle_timeout() and additionally will process records received from
the peer. If those records are themselves retransmits then we know that the
peer has not completed its handshake yet and a retransmit of our final
flight automatically occurs.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8047)

(cherry picked from commit 80c455d5ae405e855391e298a2bf8a24629dd95d)

Update NOTES.ANDROID

Minor typo fix to `adjustment` in the line:
"In such case you have to pass matching target
 name to Configure and shouldn't use -D__ANDROID_API__=N. PATH adjustment
 becomes simpler, $ANDROID_NDK/bin:$PATH suffices."

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8054)

(cherry picked from commit 52bcd4afc84d75f9d22866a3cefaf9ae4e9ff997)

Make ca command silently use default if .attr file does not exist

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7286)

(cherry picked from commit ac454d8d4663e2fcf8a8437fab8aefd883091c37)

PPC: Try out if mftb works before using it

If this fails try out if mfspr268 works.

Use OPENSSL_ppccap=0x20 for enabling mftb,
OPENSSL_ppccap=0x40 for enabling mfspr268,
and OPENSSL_ppccap=0 for enabling neither.

Fixes #8012

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8043)

(cherry picked from commit c8f370485c43729db44b680e41e875ddd7f3108c)

Fix a memory leak in the mem bio

If you use a BIO and set up your own buffer that is not freed, the
memory bio will leak the BIO_BUF_MEM object it allocates.

The trouble is that the BIO_BUF_MEM is allocated and kept around,
but it is not freed if BIO_NOCLOSE is set.

The freeing of BIO_BUF_MEM was fairly confusing, simplify things
so mem_buf_free only frees the memory buffer and free the BIO_BUF_MEM
in mem_free(), where it should be done.

Alse add a test for a leak in the memory bio
Setting a memory buffer caused a leak.

Signed-off-by: Corey Minyard <minyard@acm.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8051)

(cherry picked from commit c6048af23c577bcf85f15122dd03b65f959c9ecb)

Reduce inputs before the RSAZ code.

The RSAZ code requires the input be fully-reduced. To be consistent with the
other codepaths, move the BN_nnmod logic before the RSAZ check.

This fixes an oft-reported fuzzer bug.
https://github.com/google/oss-fuzz/issues/1761

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7187)

(cherry picked from commit 3afd537a3c2319f68280804004e9bf2e798a43f7)

apps/verify.c: Change an old comment to clarify what the callback does

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7922)

(cherry picked from commit 9b10986d7742a5105ac8c5f4eba8b103caf57ae9)

crypto/armcap.c, crypto/ppccap.c: stricter use of getauxval()

Having a weak getauxval() and only depending on GNU C without looking
at the library we build against meant that it got picked up where not
really expected.

So we change this to check for the glibc version, and since we know it
exists from that version, there's no real need to make it weak.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/8028)

(cherry picked from commit 5f40dd158cbfa0a3bd86c32f7a77fec8754bb245)

crypto/uid.c: use own macro as guard rather than AT_SECURE

It turns out that AT_SECURE may be defined through other means than
our inclusion of sys/auxv.h, so to be on the safe side, we define our
own guard and use that to determine if getauxval() should be used or
not.

Fixes #7932

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7933)

(cherry picked from commit aefb980c45134d84f1757de1a9c61d699c8a7e33)

Don't get the mac type in TLSv1.3

We don't use this information so we shouldn't fetch it. As noted in the
comments in #8005.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/8020)

(cherry picked from commit ea09abc80892920ee5db4de82bed7a193b5896f0)

Add missing entries in ssl_mac_pkey_id

Fixes #8005

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/8020)

(cherry picked from commit 7fe0ed75e3e7760226a0a3a5a86cf3887004f6e4)

Check more return values in the SRP code

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8019)

(cherry picked from commit d63bde7827b0be1172f823baf25309b54aa87e0f)

Check a return value in the SRP code

Spotted by OSTIF audit

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8019)

(cherry picked from commit 0a5bda639f8fd59e15051cf757708e3b94bcf399)

Don't artificially limit the size of the ClientHello

We were setting a limit of SSL3_RT_MAX_PLAIN_LENGTH on the size of the
ClientHello. AFAIK there is nothing in the standards that requires this
limit.

The limit goes all the way back to when support for extensions was first
added for TLSv1.0. It got converted into a WPACKET max size in 1.1.1. Most
likely it was originally added to avoid the complexity of having to grow
the init_buf in the middle of adding extensions. With WPACKET this is
irrelevant since it will grow automatically.

This issue came up when an attempt was made to send a very large
certificate_authorities extension in the ClientHello.

We should just remove the limit.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7424)

(cherry picked from commit 7835e97b6ff5cd94a10c5aeac439f4aa145a77b2)

Fix CID 1434549: Unchecked return value in test/evp_test.c

5. check_return: Calling EVP_EncodeUpdate without checking return value
(as is done elsewhere 4 out of 5 times).

Fix CID 1371695, 1371698: Resource leak in test/evp_test.c

- leaked_storage: Variable edata going out of scope leaks the storage it
points to.

- leaked_storage: Variable encode_ctx going out of scope leaks the
storage it points to

Fix CID 1430437, 1430426, 1430429 : Dereference before null check in test/drbg_cavs_test.c

check_after_deref: Null-checking drbg suggests that it
may be null, but it has already been dereferenced on all paths leading
to the check

Fix CID 1440765: Dereference before null check in test/ssltestlib.c

check_after_deref: Null-checking ctx suggests that it may be null, but
it has already been dereferenced on all paths leading to the check.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7993)

(cherry picked from commit 760e2d60e62511a6fb96f547f6730d05eb5f47ec)

More configurable crypto and ssl library initialization

1.  In addition to overriding the default application name,
    one can now also override the configuration file name
    and flags passed to CONF_modules_load_file().

2.  By default we still keep going when configuration file
    processing fails.  But, applications that want to be
    strict about initialization errors can now make explicit
    flag choices via non-null OPENSSL_INIT_SETTINGS that omit
    the CONF_MFLAGS_IGNORE_RETURN_CODES flag (which had so far
    been both undocumented and unused).

3.  In OPENSSL_init_ssl() do not request OPENSSL_INIT_LOAD_CONFIG
    if the options already include OPENSSL_INIT_NO_LOAD_CONFIG.

4.  Don't set up atexit() handlers when called with opts equal to
    OPENSSL_INIT_BASE_ONLY (this flag should only be used alone).

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7969)

Update generator copyright year.

Some Travis builds appear to fail because generated objects get
2019 copyrights now, and the diff complains.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7969)

Add a test for correct handling of the cryptopro bug extension

This was complicated by the fact that we were using this extension for our
duplicate extension handling tests. In order to add tests for cryptopro
bug the duplicate extension handling tests needed to change first.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7984)

(cherry picked from commit 9effc496ad8a9b0ec737c69cc0fddf610a045ea4)

Don't complain if we receive the cryptopro extension in the ClientHello

The cryptopro extension is supposed to be unsolicited and appears in the
ServerHello only. Additionally it is unofficial and unregistered - therefore
we should really treat it like any other unknown extension if we see it in
the ClientHello.

Fixes #7747

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7984)

(cherry picked from commit 23fed8ba0ec895e1b2a089cae380697f15170afc)

doc/man1/x509.pod: fix typo

This looks like a copy&paste error from req.pod to x509.pod.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7995)

(cherry picked from commit 67ee899cb51d3e3d7b5f00b878f8f82a097b93f0)

Restore compatibility with GOST2001 implementations.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7985)

(cherry picked from commit 673e0bbbe4b9cbd19a247c0b18c171bb0421915a)

Fix no-cmac

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7979)

(cherry picked from commit 87bbbfb1e4fc2035e8f9ec1d6313a41c410a3218)

Support _onexit() in preference to atexit() on Windows

This enables cleanup to happen on DLL unload instead of at process exit.

[extended tests]

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7983)

Introduce a no-pinshared option

This option prevents OpenSSL from pinning itself in memory.

Fixes #7598

[extended tests]

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7983)

Test atexit handlers

Test that atexit handlers get called properly at process exit, unless we
have explicitly asked for them not to be.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7983)

Don't link shlibloadtest against libcrypto

The whole point of shlibloadtest is to test dynamically loading and
unloading the library. If we link shlibloadtest against libcrypto then that
might mask potential issues.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7983)

Implement OPENSSL_INIT_NO_ATEXIT

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7983)

Fix a RUN_ONCE bug

We have a number of instances where there are multiple "init" functions for
a single CRYPTO_ONCE variable, e.g. to load config automatically or to not
load config automatically. Unfortunately the RUN_ONCE mechanism was not
correctly giving the right return value where an alternative init function
was being used.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7983)

Fix shlibloadtest to properly execute the dso_ref test

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7983)

Eliminate unused buffers from ssl3_change_cipher_state

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7971)

(cherry picked from commit 9c5ef4ea486f675f33592b34775c3e453f60ee69)

Remove unused variables from tls1_change_cipher_state

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7971)

(cherry picked from commit d072eea2e39c4444ecce3598556053a4c552d9a2)

make update

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/7973)

Fix cert with rsa instead of rsaEncryption as public key algorithm

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/7962)

(cherry picked from commit 1f483a69bce11c940309edc437eee6e32294d5f2)

Fix a minor nit in the hkdflabel size

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7913)

(cherry picked from commit 0b4233f5a4a181a6dcb7c511cd2663e500e659a4)

Fixed minor typo in the req.pod docs

CLA: Trivial

Fixes #7928.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7973)

(cherry picked from commit 005247af76aa8d623fadbf3a82aa7b8ef449989f)

fixed grammar typo

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7959)

(cherry picked from commit fff1470cda05dedd98d2e8e18c3bc320635365d4)

Do not complain when /dev/crypto does not exist.

FreeBSD does not enable cryptodev(4) by default.  OpenBSD disabled support
for /dev/crypto by default from 4.9 and removed it from 5.7.  Now the engine
is properly enabled by default on BSD platforms (see #7885), it continuously
complains:

Could not open /dev/crypto: No such file or directory

Hide the nagging error message behind ENGINE_DEVCRYPTO_DEBUG.

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7896)

(cherry picked from commit c79a022da973140c141eeebff9170ca5702be0f9)

EVP_PKEY_size declared to take a const parameter

CLA: trivial
Function EVP_PKEY_size has been modified to take a const parameter

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7892)

(cherry picked from commit 47ec2367ebf6082abb103e66e609feb5c128d358)

Coverity fix in some crypto/asn1 code

Call to i2d method returns an int value.

Fix:
CID 1338183 (#1 of 1): Improper use of negative value (NEGATIVE_RETURNS)
CID 1371691 (#1 of 1): Improper use of negative value (NEGATIVE_RETURNS)
CID 1371692 (#1 of 1): Improper use of negative value (NEGATIVE_RETURNS)

[extended tests]

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7359)

(cherry picked from commit da84249be6492ccfc5ecad32ac367fd06e9bdbef)

Coverity fix in apps/oscp

CID 1440002 (#1 of 1): Use after free (USE_AFTER_FREE)
Not a deadly error, because error was just before app exit.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7359)

(cherry picked from commit 39fc4c17c49d248e0757bac9aa8863d205c7ad12)

Admit unknown pkey types at security level 0

The check_key_level() function currently fails when the public key
cannot be extracted from the certificate because its algorithm is not
supported.  However, the public key is not needed for the last
certificate in the chain.

This change moves the check for level 0 before the check for a
non-NULL public key.

For background, this is the TPM 1.2 endorsement key certificate.
I.e., this is a real application with millions of certificates issued.
The key is an RSA-2048 key.

The TCG (for a while) specified

     Public Key Algorithm: rsaesOaep

rather than the commonly used

     Public Key Algorithm: rsaEncryption

because the key is an encryption key rather than a signing key.
The X509 certificate parser fails to get the public key.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7906)

Fix function names in ct(7) documentation

The correct function name is SSL_CTX_enable_ct, not SSL_CTX_ct_enable.

Signed-off-by: Christian Heimes <christian@python.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7916)

(cherry picked from commit 6f8b858d054c4eb1112531e39da9ceb5fa37e5f1)

man: harmonize the various formulations in the HISTORY sections

While stereotyped repetitions are frowned upon in literature, they
serve a useful purpose in manual pages, because it is easier for
the user to find certain information if it is always presented in
the same way. For that reason, this commit harmonizes the varying
formulations in the HISTORY section about which functions, flags,
etc. were added in which OpenSSL version.

It also attempts to make the pod files more grep friendly by
avoiding to insert line breaks between the symbol names and the
corresponding version number in which they were introduced
(wherever possible). Some punctuation and typographical errors
were fixed on the way.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7854)

ERR: preserve system error number in a few more places

It turns out that intialization may change the error number, so we
need to preserve the system error number in functions where
initialization is called for.
These are ERR_get_state() and err_shelve_state()

Fixes #7897

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7902)

(cherry picked from commit 91c5473035aaf2c0d86e4039c2a29a5b70541905)

Add missing OPENSSL_clear_free before using ec->key

Fixes #7657

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7877)

(cherry picked from commit 4128136a28c3b7d3878daed728c49f18eb950adc)

add missing check for BN_mod_inverse

Fixes #7650

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7876)

(cherry picked from commit 55833a8de70589a5000044b6291e190f5a3826ae)

fix inconsistent flen check in rsa_pk1 and rsa_oaep

Fixes #7117

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7880)

(cherry picked from commit 4fea7005c3d08ed0d575bdea5082b7b0ce355237)

VMS build: better treatment of .S -> .obj compilation

It turned out that .S files aren't to be treated as lightly as I
thought.  They need to go through a preprocessing step, which .s files
don't need to.

Corrects #7703

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7889)

(cherry picked from commit e436664828429a23bf210710ede34203bdfdc2b8)

doc/man3: remove copy&paste leftover

Fixes #7883

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7884)

(cherry picked from commit 00eb879f74971e3c048286ef44f6f544676f90d7)

Use the correct function name

CLA: trivial

Signed-off-by: Jakub Jelen <jjelen@redhat.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7887)

(cherry picked from commit 829800b0735ab99a0962418180cb076ff8081028)

Fix error in processing $target{enable}

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7885)

(cherry picked from commit 7a8a35ff0ce0c25a36b494a4c7e3f5f390679e6e)

Fixed typo (vi leftover).

There was a trailing :w at a line, which didn't make sense in context
of the sentence/styling. Removed it, because I think it's a leftover
vi command.

CLA: trivial
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7875)

(cherry picked from commit 143b631639f95822e5e00768254fa35c787f6396)

Add an Ed448 malleability test

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/7748)

(cherry picked from commit 6de98b4fb6265f8a4b2e5b599d6714ff937dca6b)

Disallow Ed448 signature malleability

Check that s is less than the order before attempting to verify the
signature as per RFC8032 5.2.7

Fixes #7706

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/7748)

(cherry picked from commit 08afd2f37a4465c90b9b9e2081c9e8df4726db89)

eng_devcrypto: make sure digest can do copy

Digest must be able to do partial-state copy to be used.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7585)

(cherry picked from commit 16e252a01b754a13e83d5e5e87afbe389997926b)

eng_devcrypto: fix ctr mode

Make CTR mode behave like a stream cipher.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7585)

(cherry picked from commit b5015e834aa7d3f0a5d7585a8fae05cecbdbb848)

eng_devcrypto: add cipher CTX copy function

The engine needs a custom cipher context copy function to open a new
/dev/crypto session.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7585)

(cherry picked from commit 6d99e238397859f2df58c60e28905193b2dd6762)

eng_devcrypto: close session on cleanup, not final

Close the session in digest_cleanup instead of digest_final.  A failure
in closing the session does not mean a previous successful digest final
has failed as well.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7585)

(cherry picked from commit a67203a19d379a8cc8b369587c60c46eb4e19014)

eng_devcrypto: fix copy of unitilialized digest

If the source ctx has not been initialized, don't initialize the copy
either.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7585)

(cherry picked from commit ae8183690fa53b978d4647563f5a521c4cafe94c)

eng_devcrypto: expand digest failure cases

Return failure when the digest_ctx is null in digest_update and
digest_final, and when md is null in digest_final.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7585)

(cherry picked from commit 4d9f99654441e36fdcb49540a1dbc9d4c70ccb68)

eng_devcrypto: don't leak methods tables

Call functions to prepare methods after confirming that /dev/crytpo was
sucessfully open and that the destroy function has been set.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7585)

(cherry picked from commit d9d4dff5c640990d45af115353fc9f88a497a56c)

INSTALL: add note about devcrypto engine

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7585)

(cherry picked from commit eb3fe0ee5411449230cff46b7f13ebef37aa950e)

Document the num_tickets s_server option

Fixes #7727

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7756)

(cherry picked from commit 7ffb7fbe8c1c1bc467993389478d2ea6f00e1860)

Preserve errno on dlopen

For the same reasons as in the previous commit we must preserve errno
across dlopen calls. Some implementations (e.g. solaris) do not preserve
errno even on a successful dlopen call.

Fixes #6953

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7680)

(cherry picked from commit 3cb4e7dc1cf92022f62b9bbdd59695885a1265ff)

Make sure build_SYS_str_reasons() preserves errno

This function can end up being called during ERR_get_error() if we are
initialising. ERR_get_error() must preserve errno since it gets called via
SSL_get_error(). If that function returns SSL_ERROR_SYSCALL then you are
supposed to inspect errno.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7680)

(cherry picked from commit 71b1ceffc4c795f5db21861dd1016fbe23a53a53)

make update

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7852)

(cherry picked from commit f2f734d4f9e34643a1d3e5b79d2447cd643519f8)

Prevent calling decryption in an encryption context and vice versa

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7852)

(cherry picked from commit a8bf2f8f24bda57c860e4356f31264fff63fe8b9)

err/err.c: improve err_clear_last_constant_time's portability.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7850)

(cherry picked from commit 91d0fd1c2753f0f7d6e0953eed3cfb6eb96d8ff4)

Make EVP_PKEY_asn1_add0() stricter about its input

It turns out that the strictness that was implemented in
EVP_PKEY_asn1_new() (see Github openssl/openssl#6880) was badly placed
for some usages, and that it's better to do this check only when the
method is getting registered.

Fixes #7758

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7847)

(cherry picked from commit a86003162138031137727147c9b642d99db434b1)

Doc: fix reference to deprecated methods.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7705)

(cherry picked from commit 56bc3ea0a49b8aa99a4b031e7cb48ea87fe28e2d)

curve25519.c: improve formula alignment

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7750)

(cherry picked from commit 425dde5d75a6ffff9a1027107711b99da39e3519)

curve25519.c: reformat code to follow coding guidelines

Fixes #7698

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7750)

(cherry picked from commit 3a17b9a46ecf54a30073534dff04e3223ff98813)

Fix some SSL_export_keying_material() issues

Fix some issues in tls13_hkdf_expand() which impact the above function
for TLSv1.3. In particular test that we can use the maximum label length
in TLSv1.3.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7755)

(cherry picked from commit 0fb2815b873304d145ed00283454fc9f3bd35e6b)

Revert "Reduce stack usage in tls13_hkdf_expand"

This reverts commit ec0c5f5693e39c5a013f81e6dd9dfd09ec65162d.

SSL_export_keying_material() may use longer label lengths.

Fixes #7712

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7755)

(cherry picked from commit ed371b8cbac0d0349667558c061c1ae380cf75eb)