Sending back a file stream from GRPC Python Server

user_373784 0
Viewed 5239

I have a service that needs to return a filestream to the calling client so I have created this proto file.

service Sample {
     rpc getSomething(Request) returns (stream Response){}
}

message Request {

}

message Response {
    bytes data = 1;
}

When the server receives this, it needs to read some source.txt file and then write it back to the client as a byte stream. Just would like to ask is this the proper way to do this in a Python GRPC server?

fileName = "source.txt"
with open(file_name, 'r') as content_file:
    content = content_file.read()
    response.data = content.encode()
    yield response

I cannot find any examples related to this.

grpc-python grpc python
2 Answers

That looks mostly correct, though it's hard to be sure since you haven't shared with us all of your service-side code. A few tweaks I'd suggest would be (1) reading the file as binary content in the first place, (2) exiting the with statement as early as possible, (3) constructing the response message only after you've constructed the value of its data field, and (4) making a module-scope module-private constant out of the file name. Something like:

with open(_CONTENT_FILE_NAME, 'rb') as content_file:
    content = content_file.read()
yield my_generated_module_pb2.Response(data=content)

. What do you think?

user_46211640

One option would be to lazily read in the binary and yield each chunk. Note, this is untested code:

def read_bytes(file_, num_bytes):
    while True:
        bin = file_.read(num_bytes)
        if len(bin) != num_bytes:
            break
        yield bin   

class ResponseStreamer(Sample_pb2_grpc.SampleServicer):
    def getSomething(request, context):
        with open('test.bin', 'rb') as f:
            for rec in read_bytes(f, 4):
                yield Sample_pb2.Response(data=rec)

Downside is that you'll have the file opened while the stream is open.

user_89419260
Related