Use the possibility to have test results in a different directory

RESULT_D can be used to provide a separate directory for test results.
Let's use that to separate them from other files.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4507)

Fix util/perl/OpenSSL/Test.pm input variable overwrite

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4507)

Fix util/find-doc-nits to correctly parse function signature typedefs

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4499)

Correct some typedef documentation

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4499)

Since return is inconsistent, I removed unnecessary parentheses and
unified them.
- return (0); -> return 0;
- return (1); -> return 1;
- return (-1); -> return -1;

Reviewed-by: Stephen Henson <steve@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4500)

Document that lhash isn't thread safe under any circumstances and
indicate the level of locking required for various operations.

Remove the lock and atomics from the lhash code.  These we're not complete
or adequate.

Refer to #4418 and #4427 for details.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4429)

Fix doc for i2d/d2i private/public key

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4491)

Anchor the regexp match

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4483)

Rewrite some code

Rewrite the -req-nodes flag from CA.pl (idea from Andy)
Rewrite ERR_string_error_n

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4478)

Merge tls1_check_curve into tls1_check_group_id

The function tls_check_curve is only called on clients and contains
almost identical functionaity to tls1_check_group_id when called from
a client. Merge the two.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4475)

Change curves to groups where relevant

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4475)

Use separate functions for supported and peer groups lists

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4475)

doc/man1/openssl.pod: Add missing commands and links

Fixes #4471 and more

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4472)

Fixes #4459 "issuserAltName" documentation typo.

See crypto/objects/objects.txt:767 -- field is "issuerAltName"
CLA: trivial

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4460)

stack/stack.c: various cleanups.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4455)

Remove some commented out code

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4462)

Remove an incorrect comment

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4456)

Configurations/windows-makefile.tmpl: canonicalise configured paths

This avoids issues that can come with an ending backslash, among other.

Fixes #4458

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4461)

Session resume broken switching contexts

When an SSL's context is swtiched from a ticket-enabled context to
a ticket-disabled context in the servername callback, no session-id
is generated, so the session can't be resumed.

If a servername callback changes the SSL_OP_NO_TICKET option, check
to see if it's changed to disable, and whether a session ticket is
expected (i.e. the client indicated ticket support and the SSL had
tickets enabled at the time), and whether we already have a previous
session (i.e. s->hit is set).

In this case, clear the ticket-expected flag, remove any ticket data
and generate a session-id in the session.

If the SSL hit (resumed) and switched to a ticket-disabled context,
assume that the resumption was via session-id, and don't bother to
update the session.

Before this fix, the updated unit-tests in 06-sni-ticket.conf would
fail test #4 (server1 = SNI, server2 = no SNI).

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/1529)

Use more pre-allocation

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4379)

Postpone allocation of STACK internal storage ... until a first push(),
insert() or an explicit call to OPENSSL_sk_reserve

Factorise STACK item deletion code

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4379)

nistp521: add a comment to the P+P exceptional case in point_add.

This change adds a comment to the exceptional case in point_add that
handles the case of a doubling, which explains when this case may occur
during normal processing.

Thanks go to Antonio Sanso for noting this.

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4424)

Fix the return type of felem_is_zero_int which should be int.
Change argument type of xxxelem_is_zero_int to const void*
to avoid the need of type casts.

Fixes #4413

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4450)

Added const-time flag to DSA key decoding to avoid potential leak of privkey

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4440)

doc: note that the BN_new() initialises the BIGNUM

BN_new() and BN_secure_new() not only allocate memory, but also
initialise it to deterministic value - 0.

Document that behaviour to make it explicit

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4438)

Put back the #include <openssl/safestack.h> lines in public headers.
the latter includes the former.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4437)

Remove unnecessary #include <openssl/lhash.h> directives.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4431)

Use safestack.h exclusively internally.
Remove all stack headers from some includes that don't use them.

Avoid a genearic untyped stack use.

Update stack POD file to include the OPENSSL_sk_ API functions in the notes
section.  They were mentioned in the name section but not defined anywhere.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4430)

Add stack space reservations.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4386)

Add a reserve call to the stack data structure.

This allows the caller to guarantee that there is sufficient space for a
number of insertions without reallocation.

The expansion ratio when reallocating the array is reduced to 1.5 rather than 2.

Change bounds testing to use a single size rather than both INT_MAX and
SIZE_MAX.  This simplifies some of the tests.

Switch the stack pointers to data from char * to void *

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4386)

BN_copy now propagates BN_FLG_CONSTTIME

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4377)

Fixed error in propagating BN_FLG_CONSTTIME flag through BN_MONT_CTX_set, which could lead to information disclosure on RSA primes p and q.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4377)

Allow DH_set0_key with only private key.

The pub_key field for DH isn't actually used in DH_compute_key at all.
(Note the peer public key is passed in as as BIGNUM.) It's mostly there
so the caller may extract it from DH_generate_key. It doesn't
particularly need to be present if filling in a DH from external
parameters.

The check in DH_set0_key conflicts with adding OpenSSL 1.1.0 to Node.
Their public API is a thin wrapper over the old OpenSSL one:
https://nodejs.org/api/crypto.html#crypto_class_diffiehellman

They have separate setPrivateKey and setPublicKey methods, so the public
key may be set last or not at all. In 1.0.2, either worked fine since
operations on DH objects generally didn't use the public key.  (Like
with OpenSSL, Node's setPublicKey method is also largely a no-op, but so
it goes.) In 1.1.0, DH_set0_key prevents create a private-key-only DH
object.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4384)

Add and use function tls1_in_list to avoid code duplication.

[extended tests]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/=4412)

Use tls1_group_id_lookup in tls1_curve_allowed

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/=4412)

Rename tls1_get_curvelist.

Rename tls1_get_curvelist to tls1_get_grouplist, change to void as
it can never fail and remove unnecessary return value checks. Clean
up the code.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/=4412)

Rewrite compression and group checks.

Replace existing compression and groups check with two functions.

tls1_check_pkey_comp() checks a keys compression algorithms is consistent
with extensions.

tls1_check_group_id() checks is a group is consistent with extensions
and preferences.

Rename tls1_ec_nid2curve_id() to tls1_nid2group_id() and make it static.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/=4412)

New function ssl_generate_param_group

Setup EVP_PKEY structure from a group ID in ssl_generate_param_group,
replace duplicate code with this function.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/=4412)

Replace tls1_ec_curve_id2nid.

Replace tls1_ec_curve_id2nid() with tls_group_id_lookup() which returns
the TLS_GROUP_INFO for the group.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/=4412)

Rename tls_curve_info to TLS_GROUP_INFO, move to ssl_locl.h

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/=4412)

Return group id in tls1_shared_group

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/=4412)

Return correct Suite B curve, fix comment.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/=4412)

Make sure that a cert with extensions gets version number 2 (v3)

Fixes #4419

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4420)

Update comments to match function parameter names

CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4407)

Use size of entries, not size of the pointer.

Reviewed-by: Andy Polyakov <appro@openssl.org>
GH: #4410

Use curve_id not the nid

Found by OSS-Fuzz and the tests

Reviewed-by: Andy Polyakov <appro@openssl.org>
GH: #4410

Remove dhparam from SSL_CONF list.

Avoid duplicate assertion by removing dhparam from SSL_CONF parameter list:
dhparam is handled manually by s_server.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4408)

Fix 'key' option in s_server can be in ENGINE keyform

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4405)

Store groups as uint16_t

Instead of storing supported groups in on-the-wire format store
them as parsed uint16_t values. This simplifies handling of groups
as the values can be directly used instead of being converted.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4406)

Configure: add -Wmisleading-indentation to strict warnings flags.

The warning flag in question was added in GCC version 6, hence
addition has to be conditional.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4401)

Guard against DoS in name constraints handling.

This guards against the name constraints check consuming large amounts
of CPU time when certificates in the presented chain contain an
excessive number of names (specifically subject email names or subject
alternative DNS names) and/or name constraints.

Name constraints checking compares the names presented in a certificate
against the name constraints included in a certificate higher up in the
chain using two nested for loops.

Move the name constraints check so that it happens after signature
verification so peers cannot exploit this using a chain with invalid
signatures. Also impose a hard limit on the number of name constraints
check loop iterations to further mitigate the issue.

Thanks to NCC for finding this issue. Fix written by Martin Kreichgauer.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4393)

Cleanup whitespace in ssl_lib.c (tabs to spaces)

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4383)

Fix strict-warnings build

Compilation failed due to -Werror=misleading-indentation.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4395)

Configure: unify clang's -Qunused-arguments option treatment.

Detect clang even if it's disguised, e.g. cross-compiler or invoked by
explicit path name, and add the option based on that.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4383)

Reenable s_server -dhparam option

This option was lost when converting to a table-driven option parser
in commit 7e1b7485706c2b11091b5fa897fe496a2faa56cc.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4398)

Add RSA-PSS certificate type TLS tests

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4368)

Add RSA-PSS test certificates

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4368)

Allow use of RSA-PSS certificates in TLS 1.2

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4368)

Allow RSA certificates to be used for RSA-PSS

Allo RSA certificate to be used for RSA-PSS signatures: this needs
to be explicit because RSA and RSA-PSS certificates are now distinct
types.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4368)

Add RSA-PSS key certificate type.

Recognise RSA-PSS certificate algorithm and add a new certificate
type.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4368)

Fix overflow in c2i_ASN1_BIT_STRING.

c2i_ASN1_BIT_STRING takes length as a long but uses it as an int.  Check
bounds before doing so. Previously, excessively large inputs to the
function could write a single byte outside the target buffer. (This is
unreachable as asn1_ex_c2i already uses int for the length.)

Thanks to NCC for finding this issue. Fix written by Martin Kreichgauer.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4385)

Stack sorting safety

Use the defined typechecking stack method to sort the compression methods stack
rather than using the generic function and apply type casts.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4382)

Null pointer used.
Address coverity report of null pointer being dereferenced.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4381)

Provide getters for min/max proto version

OpenSSL 1.1.0 made SSL_CTX and SSL structs opaque and introduced a new
API to set the minimum and maximum protocol version for SSL_CTX with
TLS_method(). Add getters to introspect the configured versions:

  int SSL_CTX_get_min_proto_version(SSL_CTX *ctx);
  int SSL_CTX_get_max_proto_version(SSL_CTX *ctx);
  int SSL_get_min_proto_version(SSL *ssl);
  int SSL_get_max_proto_version(SSL *ssl);

NOTE: The getters do not resolv the version in case when the minimum or
maxium version are configured as '0' (meaning auto-select lowest and
highst version number).

Signed-off-by: Christian Heimes <christian@python.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4364)

Avoid signed vs unsigned comparison error.
Introduced by #4372

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4374)

Add explanatory comment about fitting into a size_t.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4373)

Remote unrequited casts
Also use strndup instead of a malloc/memcpy pair.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4372)

Replace malloc+strcpy with strdup

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4371)

Fix function name in ECerr call

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4371)

Fix no-ec no-dh build

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4369)

Fix doc nits

[to be squashed]
[skip ci]

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4356)

Support EVP_PKEY_meth_remove and pmeth internal cleanup

1. make app pkey methods cleanup internal
2. add EVP_PKEY_meth_remove

Fixes travis-ci failure in #4337

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4356)

Ensure that the requested memory size cannot exceed the limit imposed by a
size_t variable.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4357)

Manually revert "Ensure allocation size fits into size_t"

This reverts commit cc9c56894606fdf324933cd8090d9a54d967bf5b for the file
pbe_scrypt.c instead of scrypt.c

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4357)

Revert "Reuse strndup(), simplify code"

This reverts commit 1caaea133873d549fa52fbf265298d2d35442477.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4357)

Revert "GH614: Use memcpy()/strdup() when possible"

This reverts commit a89c9a0d855bce735116acfe147b24e386f566ba.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4357)

Fix typo in comment

The peek result is now called 'err', not just 'e'.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4362)

Support key check in EVP interface

A new method is added to EVP_PKEY_METH as:

    int (*check) (EVP_PKEY_CTX *ctx);

and to EVP_PKEY_ASN1_METHOD as:

    int (*pkey_check) (EVP_PKEY_CTX *ctx);

This is used to check the validity of a specific key.

The order of calls is:
EVP_PKEY_check -> pmeth.check -> ameth.pkey_check.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4337)

Always use $ as shell prompt in example

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4359)

Document the recursive option

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4336)

Add a recursive option to 'openssl storeutl'

Simply put, any NAME type OSS_STORE_INTO is a new object that can be
looked into, and potentially lead to a whole tree of data to dive
into.  The recursive option allows someone to view the whole tree and
its data in one go.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4336)

Disable the EGD seeding meachanism when stdio is disabled

crypto/rand/rand_egd.c makes extensive use of stdio functions.  When
they are disabled, it makes sense to disable egd as well.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4358)

rsa/rsa_pmeth.c: recognize SHA3 as supported RSA digest.

Reviewed-by: Richard Levitte <levitte@openssl.org>

evp/m_sha3.c: wire SHA3 to rsaEncryption.

Reviewed-by: Richard Levitte <levitte@openssl.org>

objects/obj_xref.txt: cross-reference SHA3 and rsaEncryption.

Reviewed-by: Richard Levitte <levitte@openssl.org>

objects/objects.txt: fix RSA-SHA3-284 typo.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Update rsautl.pod for typo

Fixes the typo
CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4354)

SSL Trace improvements

A fix formatting fixes.

SSL Trace: internal constification
-  static trace tables
-  trace methods arguments

SSL Trace: enhance error message when tracing an invalid extension packet
... instead of just "Message length parse error!".

SSL trace: add Maximum-Fragment-Length TLS extension log support

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4353)

sha/asm/keccak1600-armv8.pl: fix return value buglet and ...

... script data load.

On related note an attempt was made to merge rotations with logical
operations. I mean as we know, ARM ISA has merged rotate-n-logical
instructions which can be used here. And they were used to improve
keccak1600-armv4 performance. But not here. Even though this approach
resulted in improvement on Cortex-A53 proportional to reduction of
amount of instructions, ~8%, it didn't exactly worked out on
non-Cortex cores. Presumably because they break merged instructions
to separate μ-ops, which results in higher *operations* count. X-Gene
and Denver went ~20% slower and Apple A7 - 40%. The optimization was
therefore dismissed.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Introduce named constants for the ClientHello callback.

It is otherwise unclear what all the magic numbers mean.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4349)

Fix test documentation.

The instructions don't work.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4349)

Rename SSL_CTX_set_early_cb to SSL_CTX_set_client_hello_cb.

"Early callback" is a little ambiguous now that early data exists.
Perhaps "ClientHello callback"?

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4349)

Add UI functions to set result with explicit length and to retrieve the length

This allows completely arbitrary passphrases to be entered, including
NUL bytes.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3821)

Document default section and library configuration

It is talked around but not explicitly stated in one part of the
documentation that you should put library configuration lines at the
start of the configuration file.

CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3959)

Allow an endpoint to read the alert data before closing the socket

If an alert gets sent and then we close the connection immediately with
data still in the input buffer then a TCP-RST gets sent. Some OSs
immediately abandon data in their input buffer if a TCP-RST is received -
meaning the alert data itself gets ditched. Sending a TCP-FIN before the
TCP-RST seems to avoid this.

This was causing test failures in MSYS2 builds.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4333)

Fix error handling/cleanup

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4326)

Fix test_client_hello with no-tls1_2_method

The extensions not sent when TLS 1.2 is not used caused the message
length to be 109, which is less than the 127 threshold needed
to activate the F5 workaround.  Add another 20 bytes of dummy ALPN
data do push it over the threshold.

Also, fix the definition of the (unused) local macro indicating
the threshold.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4346)

Restore historical behavior for absent ServerHello extensions

In OpenSSL 1.1.0, when there were no extensions added to the ServerHello,
we did not write the extension data length bytes to the end of the
ServerHello; this is needed for compatibility with old client implementations
that do not support TLS extensions (such as the default configuration of
OpenSSL 0.9.8).  When ServerHello extension construction was converted
to the new extensions framework in commit
7da160b0f46d832dbf285cb0b48ae56d4a8b884d, this behavior was inadvertently
limited to cases when SSLv3 was negotiated (and similarly for ClientHellos),
presumably since extensions are not defined at all for SSLv3.  However,
extensions for TLS prior to TLS 1.3 have been defined in separate
RFCs (6066, 4366, and 3546) from the TLS protocol specifications, and as such
should be considered an optional protocol feature in those cases.

Accordingly, be conservative in what we send, and skip the extensions block
when there are no extensions to be sent, regardless of the TLS/SSL version.
(TLS 1.3 requires extensions and can safely be treated differently.)

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4296)

Add checks for alloc failing.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4341)

add callback handler for setting DTLS timer interval

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4011)