read-byte ignores eof-error-p inside SBCL gray stream

user_1218261 0
Viewed 129

So, I'm specializing read-byte method using gray streams on SBCL. I've run into a peculiar behaviour where eof-error-p argument seems to be ignored. It could be I'm missing some trivial mistake in my code, but I've looked through it a dozen times already, and I just don't see it.

Suppose there is a file test.txt with a single byte, say

echo -n 7 > test.txt

I expect the following code to return :eof

(defclass binary-input-stream (fundamental-binary-input-stream)
  ((stream :initarg :stream :reader stream-of)))

(defmethod stream-read-byte ((stream binary-input-stream))
  (format t "~a~%" "It's me, all right")
  (read-byte (stream-of stream) nil :eof))

(defun make-binary-input-stream (stream)
  (make-instance 'binary-input-stream :stream stream))

(with-open-file (in "test.txt" :element-type '(unsigned-byte 8))
  (setq in (make-binary-input-stream in))
  (read-byte in)
  (read-byte in))

However, SBCL throws an END-OF-FILE exception. What is going on here?

sbcl common-lisp lisp stream
2 Answers

I can reproduce the example, and the debugger shows (under Slime):

Backtrace:
  0: (READ-BYTE #<BINARY-INPUT-STREAM {1039A8D933}> T NIL)
  1: ((LAMBDA ()))
  ...

By moving the cursor to frame 0 (i.e. read-byte) and pressing v, the editor shows the definition of read-byte in sbcl/src/code/stream.lisp (I build it from source):

(defun read-byte (stream &optional (eof-error-p t) eof-value)
  (declare (explicit-check))
  (if (ansi-stream-p stream)
      (ansi-stream-read-byte stream eof-error-p eof-value nil)
      ;; must be Gray streams FUNDAMENTAL-STREAM
      (let ((byte (stream-read-byte stream)))
        (if (eq byte :eof)
            (eof-or-lose stream eof-error-p eof-value) ;; <<<< CURSOR HERE
            (the integer byte)))))

It turns out :eof is already used by SBCL to indicate the enf of line, and since the top-level call to read-byte does not ignore errors, this cause the condition to be signaled.

Replacing the :eof keyword by another one, say :my-eof, is not good either since the returned value is not a byte. But if you return -1, the test passes (the source stream is a stream of unsigned-bytes, but your wrapper may return -1 without errors).

user_1243190

This is an addition to coredump's answer. In particular I think that the behaviour of your code is correct: SBCL is doing the right thing in its implementation of Gray streams, and it should indeed be signalling an exception here.

In your code the pattern of calls is:

  • read-byte (with no optional arguments) on your binary-input-stream calls stream-read-byte on the same stream;
  • your method on stream-read-byte directly calls read-byte on the stream you have wrapped, asking for no error and a return on EOF of :eof, and returns the value of that call without further inspection;
  • that in turn presumably goes through the stream-read-byte method of the wrapped stream, but we don't need to worry about that.

So, what should this do? Well, I am not sure where the right place for the definitive documentation of Gray streams is, but here is something which may be close to it.

In that document, stream-read-byte is defined:

STREAM-READ-BYTE  stream            [Generic Function]

    Used by READ-BYTE; returns either an integer, or the symbol :EOF if the
    stream is at end-of-file.

read-byte is then defined as:

(defun READ-BYTE (binary-input-stream &optional (eof-errorp t) eof-value)
  (check-for-eof (stream-read-byte binary-input-stream) 
                 binary-input-stream eof-errorp eof-value))

Finally check-for-eof is defined as:

(defun check-for-eof (value stream eof-errorp eof-value)
  (if (eq value :eof)
      (report-eof stream eof-errorp eof-value)
    value))

(I think the last two definitions really mean that 'the implementation needs to do something whose behaviour is equivalent to this', and in particular it needs to do that in the case where the stream is a Gray stream.)

So any method on stream-read-byte must not return :eof unless the stream is at end of file, and in particular doing so will cause an exception to be signalled (or will cause report-eof to be invoked, anyway, and it may or may not signal an exception). And :eof is the only special value that stream-read-byte can return.

Well, your method on stream-read-byte does indeed return :eof to indicate end of file, having carefully suppressed the exception that the inner call yo read-byte would otherwise signal, so it's a well-behaved method.

But then the outer call to read-byte, as defined above, sees this EOF value and dutifully raises an exception for you. Which is, in fact, what you asked for, because you did not ask to suppress the exception in those calls.

If you don't want an exception you need to make sure that the outer calls to read-byte ask for one not to happen, by, for instance:

(with-open-file (in "test.txt" :element-type '(unsigned-byte 8))
  (with-open-stream (bin (make-binary-input-stream in))
    (values (read-byte bin nil ':eof)
            (read-byte bin nil ':eof))))

For a one-byte file this should return the byte in the file and :eof.

Admin1
Related