Google Pub/Sub performance (latency under low load)?

user_685764 0
Viewed 9040

We are about to pick Google Cloud Pub/Sub as our message bus. Some of the messages need to be dispatched in low latency durations like less than 100-200ms.

I read somewhere that in high volumes of messages, it performs well, but for lesser amounts like hundreds of messages per second, Pub/Sub may yield high latencies.

And in that previous question, the topic is not clear as well. what's google cloud pub/sub latency from 2016.

Are there any defined metrics about Pub/Sub latencies and relationship with the volume of messages?

google-cloud-pubsub latency message-queue
1 Answers

Cloud Pub/Sub was designed with high volumes of messages in mind, so it's expected for it to struggle sometimes if the message throughput is too low. This has to do with how Pub/Sub balances the pull requests between listeners.

Back to your question, there are no defined metrics regarding this behaviour, also not minimal latency or a magic number of messages that you can say it's considered "high load". In reality it depends on factors like the number of workers, batch size, processing time of each message, etc.

I've found this other StackOverflow answer that talks a bit more about a valid use case for Pub/Sub (messages in the order of thousands per second) and why high loads are recommended for Cloud Pub/Sub.

Edit:

Expanding on this topic, adding an asynchronous layer to the Pub/Sub implementation can help improve its performance, including latency in low message volume scenarios. Spotify, when working with Pub/Sub, realized the need for this and created their own Async Cloud Pub/Sub Client. They now recommend using Google's java client, but is worth taking this information into account

user_99082510
Related