Is it possible to open a binary file with os.open and os.fdopen on Unix?

Question

I'm trying to stream a file to clients with Python, and I need to add the HTTP header fields in the response, namely Content-Length and Last-Modified. I found that I can access these fields from the file using os.fstat, which returns a stat_result object, giving me st_size and st_mtime that I can use in the response header.

Now this os.fstat takes a file descriptor, which is provided by os.open. This works:

import os

file_name = "file.cab"

fd = os.open(file_name, os.O_RDONLY)
stats = os.fstat(fd)

print("Content-Length", stats.st_size) # Content-Length 27544
print("Last-Modified", stats.st_mtime) # Last-Modified 1650348549.6016183

Now to actually open this file and have a file object (so I can read and stream it), I can use os.fdopen, which takes the file descriptor provided by os.open.

f = os.fdopen(fd)
print(f) # <_io.TextIOWrapper name=3 mode='r' encoding='UTF-8'>

We can see that the return object has encoding set to UTF-8. However, when I try to read the file, it gives an error:

print(f.read())
Traceback (most recent call last):

  File "{redacted}/stream.py", line 10, in 
    print(f.read())
  File "/usr/lib/python3.9/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x82 in position 60: invalid start byte

Now there's this flag called os.O_BINARY, but it's mentioned in the document that

The above constants are only available on Windows.

And sure enough, since I'm running on a Unix machine, if I execute os.open with this flag, it gives an AttributeError:

fd = os.open(file_name, os.O_RDONLY | os.O_BINARY)

Traceback (most recent call last):
  File "{redacted}/stream.py", line 5, in 
    fd = os.open(file_name, os.O_RDONLY | os.O_BINARY)
AttributeError: module 'os' has no attribute 'O_BINARY'

So is it possible to open a binary file with os.open and os.fdopen on Unix?

Note that this problem doesn't occur if I just use the built-in open function:

file_name = "file.cab"

f = open(file_name, 'rb')
print(f) # <_io.BufferedReader name='file.cab'>
print(f.read()) # throws up the file in my terminal

But I have to open it with the os module, because I need to provide those HTTP header fields I mentioned.

Edit: As mentioned by tripleee, this is an example of an XY problem. I can get the result I want by using os.stat, which doesn't necessarily take a file descriptor and can be used with just the file path. So I can do something like this:

import os

file_name = "file.cab"

f = open(file_name, 'rb')
stats = os.stat(file_name)

print(f) # <_io.BufferedReader name='file.cab'>
print(stats) # os.stat_result(...)

So at this point, I'm only wondering how, or if, it's possible to do the same with os.open and os.fdopen.

Is it possible to open a binary file with os.open and os.fdopen on Unix?

0 Answers