we have a microservices architecture developed on Google cloud. Actually the microservices are all running on cloud run and talk each other with rest (sync) or with pub/sub (async).
It is an event-driven pattern so when a service publish something happened (like "user_created") on the right pub/sub topic, many services receive that event with a push subscription on their http endpoint.
Now we are moving to kafka for message ordering and replaying features. Unfortunately kafka consumers are pull based so we need to change the way services are receiving events.
Since cloud run is a serverless solutions that scale to zero, we cannot make it listen to kafka topic, because the service could shut down during the night because no request arrive.
We have different services which can safely be updated with a scheduled cron, so every one hour as example, we make a get request to service, which download all new kafka messages and update itself accordingly.
But many other services need a near real time update to accomplish their role.
So which product of google cloud platform is best suited to consume kafka topic in this architecture?
Thanks!