Using google PubSub with a Cloud Run gRPC service

Viewed 142

I have a Cloud Run gRPC service and I want it to listen to PubSub topic. The only way to do it from the official documentations is to use a trigger but this only works with REST server that accept http requests. I couldn't find any good example on the web of how to use pubsub with cloud run grpc services. My service is built with Go following the official grpc instructions

main.go:

func main() {
    ctx := context.Background()

    pubSubClient, err := pubsub.NewClient(ctx, 'project-id')
    if err != nil {
        log.Fatal(err)
    }
    pubSubSubscription := pubSubClient.Subscription("subscription-id")
    go func(sub *pubsub.Subscription) {
        err := sub.Receive(ctx, func(ctx context.Context, m *pubsub.Message) {
            defer m.Ack()
            log.Printf("sub.Receiv called %+v", m)
        })
        if err != nil {
            // handle error
        }
    }(pubSubSubscription)

    ....

    lis, err := net.Listen("tcp", ":"+port)
    if err != nil {
        log.Fatalf("failed to listen: %v", err)
    }

    s, _ := server.New(ctx, ...)
    grpcServer := grpc.NewServer()

    grpcpkg.RegisterReportServiceServer(grpcServer, s)
    if err := grpcServer.Serve(lis); err != nil {
        log.Fatalf("failed to serve: %s", err)
    }
}

This will work as long as there is at least one instance up and listening. But if there is no instances active, and a new pubsub message publishes, cloud run won't "wake up" the insane because it's autoscaling is utilized based on http requests. The only way to make the above code work is by setting min-instance 1 in cloud run, but this can be not efficient for a lot of use cases (i.e the service is not active at night)

Is there any work around for this? Is there any way to trigger a grpc cloud run service from a pubsub message?

1 Answers

The sub.Receive method you are attempting to use on the client library is meant for Pull message delivery and not Push delivery. You can read about how these two delivery types compare to make a choice.

If you wish to receive messages as incoming requests, you must use Push delivery and can follow the the Cloud Run Pubsub usage guide. Note that Cloud Pub/Sub only delivers messages as HTTPS POST requests with the specified JSON payload. To handle these requests from a gRPC server, you can use the gRPC Gateway plugin to generate a reverse proxy that exposes an HTTP API.

Once your Cloud Run instance is accepting HTTP traffic using the reverse proxy, it will be autoscaled with incoming requests, so you won't need to keep 1 instance always running. Note that this may mean that message processing latency is upto 10s on a cold start.

Related