From 491c35324c9f4392ec3d59bb260bf9b7e588f881 Mon Sep 17 00:00:00 2001 From: Richard Levitte Date: Sat, 5 May 2018 00:06:07 +0200 Subject: [PATCH] Docs: add general document on how pass phrases are handled Reviewed-by: Andy Polyakov (Merged from https://github.com/openssl/openssl/pull/6179) --- doc/man7/passphrase-encoding.pod | 182 +++++++++++++++++++++++++++++++ 1 file changed, 182 insertions(+) create mode 100644 doc/man7/passphrase-encoding.pod diff --git a/doc/man7/passphrase-encoding.pod b/doc/man7/passphrase-encoding.pod new file mode 100644 index 0000000000..bc777c90c6 --- /dev/null +++ b/doc/man7/passphrase-encoding.pod @@ -0,0 +1,182 @@ +=pod + +=encoding utf8 + +=head1 NAME + +password encoding +- How diverse parts of OpenSSL treat pass phrases character encoding + +=head1 DESCRIPTION + +In a modern world with all sorts of character encodings, the treatment of pass +phrases has become increasingly complex. +This manual page attempts to give an overview over how this problem is +currently addressed in different parts of the OpenSSL library. + +=head2 The general case + +The OpenSSL library doesn't treat pass phrases in any special way as a general +rule, and trusts the application or user to choose a suitable character set +and stick to that throughout the lifetime of affected objects. +This means that for an object that was encrypted using a pass phrase encoded in +ISO-8859-1, that object needs to be decrypted using a pass phrase encoded in +ISO-8859-1. +Using the wrong encoding is expected to cause a decryption failure. + +=head2 PKCS#12 + +PKCS#12 is a bit different regarding pass phrase encoding. +The standard stipulates that the pass phrase shall be encoded as an ASN.1 +BMPString, which consists of the code points of the basic multilingual plane, +encoded in big endian (UCS-2 BE). + +OpenSSL tries to adapt to this requirements in one of the following manners: + +=over 4 + +=item 1. + +Treats the received pass phrase as UTF-8 encoded and tries to re-encode it to +UTF-16 (which is the same as UCS-2 for characters U+0000 to U+D7FF and U+E000 +to U+FFFF, but becomes an expansion for any other character), or failing that, +proceeds with step 2. + +=item 2. + +Assumes that the pass phrase is encoded in ASCII or ISO-8859-1 and +opportunistically prepends each byte with a zero byte to obtain the UCS-2 +encoding of the characters, which it stores as a BMPString. + +Note that since there is no check of your locale, this may produce UCS-2 / +UTF-16 characters that do not correspond to the original pass phrase characters +for other character sets, such as any ISO-8859-X encoding other than +ISO-8859-1 (or for Windows, CP 1252 with exception for the extra "graphical" +characters in the 0x80-0x9F range). + +=back + +OpenSSL versions older than 1.1.0 do variant 2 only, and that is the reason why +OpenSSL still does this, to be able to read files produced with older versions. + +It should be noted that this approach isn't entirely fault free. + +A passphrase encoded in ISO-8859-2 could very well have a sequence such as +0xC3 0xAF (which is the two characters "LATIN CAPITAL LETTER A WITH BREVE" +and "LATIN CAPITAL LETTER Z WITH DOT ABOVE" in ISO-8859-2 encoding), but would +be misinterpreted as the perfectly valid UTF-8 encoded code point U+00EF (LATIN +SMALL LETTER I WITH DIARESIS) I. +A pass phrase that contains this kind of byte sequence will give a different +outcome in OpenSSL 1.1.0 and newer than in OpenSSL older than 1.1.0. + + 0x00 0xC3 0x00 0xAF # OpenSSL older than 1.1.0 + 0x00 0xEF # OpenSSL 1.1.0 and newer + +On the same accord, anything encoded in UTF-8 that was given to OpenSSL older +than 1.1.0 was misinterpreted as ISO-8859-1 sequences. + +=head2 OSSL_STORE + +L acts as a general interface to access all kinds of objects, +potentially protected with a pass phrase, a PIN or something else. +This API currently doesn't stipulate any specific encoding of pass phrases, but +uses the underlying routines with their behaviours. +This means that when using the built-in C scheme loader, the pass phrase +to unlock a PKCS#12 file will be treated as described for PKCS#12 above, and +the pass phrase for a PEM files will be treated as the general case described +above, since that loader uses the same underlying routines. +I. + +=head1 RECOMMENDATIONS + +This section assumes that you know what pass phrase was used for encryption, +but that it may have been encoded in a different character encoding than the +one used by your current input method. +For example, the pass phrase may have been used at a time when your default +encoding was ISO-8859-1 (i.e. "naïve" resulting in the byte sequence 0x6E 0x61 +0xEF 0x76 0x65), and you're now in an environment where your default encoding +is UTF-8 (i.e. "naïve" resulting in the byte sequence 0x6E 0x61 0xC3 0xAF 0x76 +0x65). +Whenever it's mentioned that you should use a certain character encoding, it +should be understood that you either change the input method to use the +mentioned encoding when you type in your pass phrase, or use some suitable tool +to convert your pass phrase from your default encoding to the target encoding. + +Also note that the sub-sections below discuss human readable pass phrases. +This is particularly relevant for PKCS#12 objects, where human readable pass +phrases are assumed. +For other objects, it's as legitimate to use any byte sequence (such as a +sequence of bytes from `/dev/urandom` that's been saved away), which makes any +character encoding discussion irrelevant; in such cases, simply use the same +byte sequence as it is. + +=head2 Creating new objects + +For creating new pass phrase protected objects, make sure the pass phrase is +encoded using UTF-8. +This is default on most modern Unixes, but may involve an effort on other +platforms. +Specifically for Windows, setting the environment variable +C will have anything entered on [Windows] console prompt +converted to UTF-8 (command line and separately prompted pass phrases alike). + +=head2 Opening existing objects + +For opening pass phrase protected objects where you know what character +encoding was used for the encryption pass phrase, make sure to use the same +encoding again. + +For opening pass phrase protected objects where the character encoding that was +used is unknown, or where the producing application is unknown, try one of the +following: + +=over 4 + +=item 1. + +Try the password that you have as it is in the character encoding of your +environment. +It's possible that its byte sequence is exactly right. + +=item 2. + +Convert the pass phrase to UTF-8 and try with the result. +Specifically with PKCS#12, this should open up any object that was created +according to the specification. + +=item 3. + +Do a naïve (i.e. purely mathematical) ISO-8859-1 to UTF-8 conversion and try +with the result. +This differs from the previous attempt because ISO-8859-1 maps directly to +U+0000 to U+00FF, which other non-UTF-8 character sets do not. + +This also takes care of the case when a UTF-8 encoded string was used with +OpenSSL older than 1.1.0. +(for example, C<ï>, which is 0xC3 0xAF when encoded in UTF-8, would become 0xC3 +0x83 0xC2 0xAF when re-encoded in the naïve manner. +The conversion to BMPString would then yield 0x00 0xC3 0x00 0xA4 0x00 0x00, the +erroneous/non-compliant encoding used by OpenSSL older than 1.1.0) + +=back + +=head1 SEE ALSO + +L, +L, +L, L, +L, +L, L, +L + +=head1 COPYRIGHT + +Copyright 2018 The OpenSSL Project Authors. All Rights Reserved. + +Licensed under the OpenSSL license (the "License"). You may not use +this file except in compliance with the License. You can obtain a copy +in the file LICENSE in the source distribution or at +L. + +=cut -- 2.25.1