Various README updates

Close GH Issue 69
Close GH PR 457
Some other updates

By Rich Salz, Alessandro Ghedini, Steve Marquess, Collin Anderson

Reviewed-by: Kurt Roeckx <kurt@openssl.org>

Remove des_ver.h; broke build.

Reviewed-by: Matt Caswell <matt@openssl.org>

Replace "SSLeay" in API with OpenSSL

All instances of SSLeay (any combination of case) were replaced with
the case-equivalent OpenSSL.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove some SSLv2 references

There were a few remaining references to SSLv2 support which are no longer
relevant now that it has been removed.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Add SRP and PSK to disallowed CertificateRequest ciphersuites

There was a discrepancy between what ciphersuites we allowed to send a
CertificateRequest, and what ciphersuites we allowed to receive one. So
add PSK and SRP to the disallowed ones.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove the inline keyword

Some functions were marked as inline in statem_srvr.c where they probably
didn't need to be, so remove it.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove superfluous check

|tls_process_finished| was checking that |peer_finish_md_len| was
non-negative. However neither |tls1_final_finish_mac| or
|ssl3_final_finish_mac| can ever return a negative value, so the check is
superfluous.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Change SSL_state_string return strings to start with a T

Due the rest of the state machine changes it makes sense to change the
SSL_state_string return strings from 3* to T*. They are not SSL3 specific

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Change snprintf to memcpy

There was a few uses of snprintf in the DTLS SCTP code which made more
sense to be a memcpy.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Add ossl_statem prefix to various functions

Add the ossl_statem prefix to various funtions to avoid name clashes.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Convert enums to typedefs

Various enums were introduced as part of the state machine rewrite. As a
matter of style it is preferred for these to be typedefs.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Make dtls1_link_min_mtu static

The function dtls1_link_min_mtu() was only used within d1_lib.c so make
it static.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix a bogus clang warning

Clang with --strict-warnings was complaining about an uninitalised
variable. In reality it will never be used uninitialised but clang can't
figure out the logic, so just init it anyway to silence the warning.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix empty NewSessionTicket processing

Rebasing the state machine code introduced a problem with empty
NewSessionTicket processing. The return value from the
tls_process_new_session_ticket() is supposed to be an enum, but a bare
integer was being used. Unfortunately this is valid C so the compiler
doesn't pick it up.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix uninitialised variable

Fix another instance of |al| being unitialised in certain error scenarios.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

make update

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix various error codes

A number of error codes were wrong due to a rebase of the state machine
code.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Move in_handshake into STATEM

The SSL variable |in_handshake| seems misplaced. It would be better to have
it in the STATEM structure.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix unitialised variable problem

tls_process_client_hello() failed to initialise the |al| variable in some
(error) scenarios. This could cause issues with creating the alert.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix DTLSv1_listen following state machine changes

Adding the new state machine broke the DTLSv1_listen code because
calling SSL_in_before() was erroneously returning true after DTLSv1_listen
had successfully completed. This change ensures that SSL_in_before returns
false.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Add a function to get the info_callback

Remove repeated blocks of checking SSL and then SSL_CTX for the
info_callback.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove the old state defines

Remove all the defines for the old state machines states. Mapping old to new
is probably going to cause more problems than it solves so it is probably
better to just remove them.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Change SUB_STATE_RETURN into a typedef

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Rename STATEM to OSSL_STATEM

Add an OSSL_ prefix onto STATEM

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove SSL_state and SSL_set_state

SSL_state has been replaced by SSL_get_state and SSL_set_state is no longer
supported.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Move |no_cert_verify| into state machine

The |no_cert_verify| should be in the state machine structure not in SSL

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove extraneous parens

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Change statem prefix to ossl_statem

Change various state machine functions to use the prefix ossl_statem
instead.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Don't depend on SSL structure internals

The macros SSL_in_connect_init() and SSL_in_accept_init() inadvertently
depended on SSL structure internals. This fixes it to use public API calls
instead.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Change HANDSHAKE_STATE to OSSL_HANDSHAKE_STATE

Rename the enum HANDSHAKE_STATE to OSSL_HANDSHAKE_STATE to ensure there are
no namespace clashes, and convert it into a typedef.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix some client side transition logic

Fixed some issues in the logic for determining whether an SKE should be
expected or not. In particular only allow an SKE for RSA if its export and
the key size is not allowed. Also fix the ephemeral ciphersuite checks and
add in a missing call to ssl3_check_cert_and_algorithm().

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Minor documentation tweak

Update the return type for SSL_state in the documentation.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Update CHANGES

Update the CHANGES file for the state machine rewrite

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove a call to SSL_set_state from s_server

s_server was (ab)using SSL_set_state to force a renegotiation. This is a
bad way to do things and does not work with the new state machine code, so
we need to do it a different way.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix a comment

Fix an out of date reference to old state machine code in a comment

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove some unused variables

The next_state variable is no longer needed in the new state machine.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Add a state machine README

Add some documentation on the thinking behind the state machine.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

make update

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

More state machine reorg

Move some function definitions around within the state machine to make sure
they are in the correct files. Also create a statem_locl.h header for stuff
entirely local to the state machine code and move various definitions into
it.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Reorganise state machine files

Pull out the state machine into a separate sub directory. Also moved some
functions which were nothing to do with the state machine but were in state
machine files. Pulled all the SSL_METHOD definitions into one place...most
of those files had very little left in them any more.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove ssl_get_message from ssl_method_st

ssl_get_message is no longer used so it should be removed from
ssl_method_st

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Move PACKET creation into the state machine

Previously each message specific process function would create its own
PACKET structure. Rather than duplicate all of this code lots of times we
should create it in the state machine itself.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove the SSL state variable

The SSL structure contained a "state" variable that kept track of the state
machine in the old code. The new state machine does not use this so it can
be removed.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove the type variable

The SSL structure contained a "type" variable that was set to either
SSL_ST_ACCEPT or SSL_ST_CONNECT depending on whether we are the server or
the client. This duplicates the capability of the "server" variable and was
actually rarely used.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Convert DTLSv1_listen to use new state machine code

The DTLSv1_listen code set the state value explicitly to move into init.
Change to use state_set_in_init() instead.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Redefine old state values

ssl.h and ssl3.h have a number of defines for the various states in the old
state machine code. Since this is public API it is not desirable to just
remove them. Instead redefine them to the closest equivalent state in the
new state machine code. If an application calls SSL_state then the return
value can still be compared against these old values if necessary. However
not all values have an equivalent state in the new code, so these are just
redefined to a dummy value.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove redundant code

Clean up and remove lots of code that is now no longer needed due to the
move to the new state machine.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Move server side DTLS to new state machine

Implement all of the necessary changes to make DTLS on the server work
with the new state machine code.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Move server side TLS to new state machine

Implement all of the necessary changes for moving TLS server side
processing into the new state machine code.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Split TLS server functions

Split the TLS server ssl3_get_* and ssl3_send_* functions into two ready
for the migration to the new state machine code.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Delete unused functions

Remove all the functions and dead code that is now no longer required as
a result of the DTLS client move into the new state machine code.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Implement DTLS client move to new state machine

Move all DTLS client side processing into the new state machine code. A
subsequent commit will clean up the old dead code.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

dtls_get_message changes for state machine move

Create a dtls_get_message function similar to the old dtls1_get_message but
in the format required for the new state machine code. The old function will
eventually be deleted in later commits.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Client TLS state machine rewrite cleanup

Remove redundant code following moving client side TLS handling to the new
state machine implementation.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Implement Client TLS state machine

This swaps the implementation of the client TLS state machine to use the
new state machine code instead.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Split client message reading and writing functions

The new state machine code will split up the reading and writing of
hanshake messages into discrete phases. In order to facilitate that the
existing "get" type functions will be split into two halves: one to get
the message and one to process it. The "send" type functions will also have
all work relating to constructing the message split out into a separate
function just for that. For some functions there will also be separate
pre and post "work" phases to prepare or update state.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Add initial state machine rewrite code

This is the first drop of the new state machine code.

The rewrite has the following objectives:
- Remove duplication of state code between client and server
- Remove duplication of state code between TLS and DTLS
- Simplify transitions and bring the logic together in a single location
  so that it is easier to validate
- Remove duplication of code between each of the message handling functions
- Receive a message first and then work out whether that is a valid
  transition - not the other way around (the other way causes lots of issues
  where we are expecting one type of message next but actually get something
  else)
- Separate message flow state from handshake state (in order to better
  understand each)
  - message flow state = when to flush buffers; handling restarts in the
    event of NBIO events; handling the common flow of steps for reading a
    message and the common flow of steps for writing a message etc
  - handshake state = what handshake message are we working on now
- Control complexity: only the state machine can change state: keep all
  the state changes local to a file

This builds on previous state machine related work:
- Surface CCS processing in the state machine
- Version negotiation rewrite

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Split ssl3_get_message

The function ssl3_get_message gets a whole message from the underlying bio
and returns it to the state machine code. The new state machine code will
split this into two discrete steps: get the message header and get the
message body. This commit splits the existing function into these two
sub steps to facilitate the state machine implementation.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

BN_GF2m_mod_inv(): check bn_wexpand return value

Signed-off-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@akamai.com>
MR #1276, RT #4107

Remove SSLeay history, etc., from docs

If something was "present in all versions" of SSLeay, or if it was
added to a version of SSLeay (and therefore predates OpenSSL),
remove mention of it.  Documentation history now starts with OpenSSL.

Remove mention of all history before OpenSSL 0.9.8, inclusive.

Remove all AUTHOR sections.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Remove HAMC_cleanup

Old API for use with OpenSSL-0.9.6.  Remove it.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Move crypto/threads to demo/threads

Reviewed-by: Richard Levitte <levitte@openssl.org>

Probably fix travis (wine build).

Reviewed-by: Stephen Henson <steve@openssl.org>

Improve make depend.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix undeclared variable warnings.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>

Remove useless code

RT#4081

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Check memory allocation

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix references to various RFCs

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix typos

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Set salt length after the malloc has succeeded

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix memory leaks and other mistakes on errors

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Replace malloc+strlcpy with strdup

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Do not treat 0 return value from BIO_get_fd() as error

0 is a valid file descriptor.

RT#4068

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Remove bugs/ and crypto/threads/

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Clarify return values for EVP_DigestVerifyFinal.

Previous language was unclear.  New language isn't pretty but I believe
it is more accurate.

Signed-off-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Laurie <ben@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Only include SRP headers when OPENSSL_NO_SRP is undefined

[fixes github issue #447]

Reviewed-by: Tim Hudson <tjh@openssl.org>

Make Configure die when unsupported options are given

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add an explicit list of options that can be disabled, enabled, ...

Configure has, so far, had no control at all of which 'no-' options it
can be given.  This means that, for example, someone could configure
with something absurd like 'no-stack' and then watch the build crumble
to dust...  or file a bug report.

This introduces some sanity into the possible choices.

The added list comes from looking for the explicit ones used in
Configure, and from grepping after OPENSSL_NO_ in all source files.

Reviewed-by: Rich Salz <rsalz@openssl.org>

make update

Reviewed-by: Matt Caswell <matt@openssl.org>

Don't forget to load the CT error strings

Reviewed-by: Matt Caswell <matt@openssl.org>

Handle CT error macros separately

Because the default error macro generator assumes the header file with
error macros is in include/openssl and therefore generates a C file
with error texts that include <openssl/{name}.h>, we need to generate
the error macros and texts for CT separately, since the CT module
doesn't follow the default criteria.

Reviewed-by: Matt Caswell <matt@openssl.org>

Don't use SSLv23_server_method in an example

The function SSLv23_server_method() is an old name. New code should use
TLS_server_method() instead. Therefore don't use SSLv23_server_method() in
an example in the docs.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Avoid undefined behaviour in PACKET_buf_init

Change the sanity check in PACKET_buf_init to check for excessive length
buffers, which should catch the interesting cases where len has been cast
from a negative value whilst avoiding any undefined behaviour.

RT#4094

Reviewed-by: Richard Levitte <levitte@openssl.org>

ct_locl.h moved, reflect it in crypto/ct/Makefile

Reviewed-by: Matt Caswell <matt@openssl.org>

make update

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add crypto/include/internal to the directories to scan for stack declarations

Reviewed-by: Rich Salz <rsalz@openssl.org>

Because ct_locl.h is used between modules, move it to internal headers

Rename it to ct_int.h

Reviewed-by: Rich Salz <rsalz@openssl.org>

Move auto Host adding to query_responder

Check for Host header in query_responder instead of process_responder. This
also fixes a memory leak in the old code if the headers was NULL.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix memory leak with -issuer option.

Reviewed-by: Richard Levitte <levitte@openssl.org>

set string type when embedding

Reviewed-by: Tim Hudson <tjh@openssl.org>

Move contributing info to CONTRIBUTING

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>

Run tests on Travis for mingw builds as well

Signed-off-by: Rich Salz <rsalz@akamai.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>

Fix error message when loading engines from config

When using command line applications errors occur when trying to
load engines specified in a config file. Introduced by commit
a0a82324f96

RT#4093

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Remove Obsolete engines

There are a number of engines in the OpenSSL source code which are now
obsolete. The following engines have been removed: 4758cca, aep, atalla,
cswift, nuron, sureware.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix self signed handling.

Don't mark a certificate as self signed if keyUsage is present and
certificate signing not asserted.

PR#3979

Reviewed-by: Matt Caswell <matt@openssl.org>

embed CRL serial number and signature fields

Reviewed-by: Rich Salz <rsalz@openssl.org>

embed certificate serial number and signature fields

Reviewed-by: Rich Salz <rsalz@openssl.org>

embed value field of X509_EXTENSION

Reviewed-by: Rich Salz <rsalz@openssl.org>

add CHANGES entry for embed

Reviewed-by: Rich Salz <rsalz@openssl.org>

Handle embed flag in ASN1_STRING_copy().

Reviewed-by: Rich Salz <rsalz@openssl.org>