we are running multiple instances of a GRPC streaming service on our infrastructure. Currently we are using nginx to reverse-proxy and load balance requests to our instances.
However, as this streaming service can create high load on our servers we now want to add more cloud based instances. For that we would need a reverse-proxy that proxies requests based on the GRPC message payload from which we are able to estimate the load.
Example Service:
service HelloService {
rpc SayHello (HelloRequest) returns (stream HelloResponse);
}
message HelloRequest {
string greeting = 1;
}
message HelloResponse {
string reply = 1;
}
For the proxy we now would need something like this
// ...
if len(request.greeting) > getCurrentThreshold() {
proxy_to("long-name.service.cloud", request)
} else {
proxy_to("short-name.service.local", request)
}
//...
That means we would either need an existing proxy service (such as nginx) being able to parse GRPC requests, do dynamic lookups to figure out the correct endpoint and then proxy the request to it. Or we would need to implement our own proxy server (preferably in Go), similar to this package: https://github.com/mwitkow/grpc-proxy but I haven't seen a way to extract the GRPC payload on the UnknownServiceHandler which is used by that package.
Unfortunately as this service is used by a bunch of customers it's very difficult to change anything on the client side (for example adding headers which are then used on the server side to select the instance instead of the message parsing).
Any hints on how to create such a proxy?