openssl dgst, openssl enc: check for end of input

The input reading loop in 'openssl dgst' and 'openssl enc' doesn't
check for end of input, and because of the way BIO works, it thereby
won't detect that the end is reached before the read is an error.
With the FILE BIO, an error occurs when trying to read past EOF, which
is fairly much ok, except when the command is used interactively, at
least on Unix.  The result in that case is that the user has to press
Ctrl-D twice for the command to terminate.

The issue is further complicated because both these commands use
filter BIOs on top of the FILE BIO, so a naïve attempt to check
BIO_eof() doesn't quite solve it, since that only checks the state of
the source/sink BIO, and the filter BIO may have some buffered data
that still needs to be read.  Fortunately, there's BIO_pending() that
checks exactly that, if any filter BIO has pending data that needs to
be processed.

We end up having to check both BIO_pending() and BIO_eof().

Thanks to Zsigmond Lőrinczy for the initial effort and inspiration.

Fixes #9355

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9668)

(cherry picked from commit 8ed7bbb411d2a9e0edef928958ad955e0be3d6dd)

diff --git a/apps/dgst.c b/apps/dgst.c
index d158a0ccb2843af199467d4dde07b66d02aaf1ae..6e86fa410a749cc5bf1f009c0fbb403e029ad624 100644 (file)
--- a/apps/dgst.c
+++ b/apps/dgst.c
@@ -421,7 +421,7 @@ int do_fp(BIO *out, unsigned char *buf, BIO *bp, int sep, int binout,
     size_t len;
     int i;
 
-    for (;;) {
+    while (BIO_pending(bp) || !BIO_eof(bp)) {
         i = BIO_read(bp, (char *)buf, BUFSIZE);
         if (i < 0) {
             BIO_printf(bio_err, "Read Error in %s\n", file);

diff --git a/apps/enc.c b/apps/enc.c
index de33e57a5f49a5ba3cabd3276d668920e7c9c184..d1772f3eb9f25e47a233441d020e6c3371347034 100644 (file)
--- a/apps/enc.c
+++ b/apps/enc.c
@@ -586,7 +586,7 @@ int enc_main(int argc, char **argv)
     if (benc != NULL)
         wbio = BIO_push(benc, wbio);
 
-    for (;;) {
+    while (BIO_pending(rbio) || !BIO_eof(rbio)) {
         inl = BIO_read(rbio, (char *)buff, bsize);
         if (inl <= 0)
             break;