How can I stream arrays with Grpc?

user_16705333 0
Viewed 596

I want to transport an array of int64.
I looked up how to use it. In my proto-file I either need a stream:

service myService {
    rpc GetValues(myRequest) returns (stream myResponse);
}

message myRequest {
}

message myResponse {
    int64 values = 1;
}

or a repeated response:

message myRepeatedResponse {
    repeated int64 value = 1;
}

Is one option better than the other?
My use case is that I want to read the latest x entries from my Database and send these values as an array to my client.
But I didn't get how I am supposed to do it, because when assigning the values in the overridden function of MyService.MyService.Base I can only pass values of type 'long' and not 'long[]'.

grpc
1 Answers

For the stream vs repeated question, the answer is: it depends.

The distinction between the two is that:

  • streaming sends one or more messages (each message possibly containing repeated fields)
  • unary sends a single message containing a repeated field

So, I think your decision is based upon:

  • how the server obtains the repeated field.
  • the size of the message (including the repeated field that's being sent)
  • the 'integrity' of the message

If the server is unable to obtain the entirety of the repeated field in one go, then your answer is simpler; the server will need to stream the messages (including the repeated field) as it obtains them.

By 'integrity' of the message, is there some reason why decomposing the message into many (to stream) is problematic. If the repeated field must be transmitted as a single chunk, almost as a transactional unit, then I think you may prefer to not stream the message as chunks.

You should also consider the consequence on your client(s). Are your clients able to receive one larger message or, would many smaller messages be preferred, e.g. an IoT SoC device that's resource constrained.

Otherwise, if individual messages are large1, then you'd want to decompose them into smaller 'bites' and stream them.

1: Large Data Set and note that there is a hard limit of 2GB/message.

user_6092900
Related