Cosmos DB Change Feed - What's the delay/latency?

user_9041274 0
Viewed 1804

What is the "normal" latency between when a document is created/updated and when the Cosmos DB Change Feed processor picks it up?

Some operations we do are two-phase: Create, then a few milliseconds later, Update.

I'm aware that only the latest version of a document will appear in the change feed. But if the change feed is super-duper speedy, I'll end up processing both versions of the document. Double RU usage than necessary because I only care about the "final" version.

Sure, I'll design my processor so that it doesn't care. And sure, I doubt there's any set-in-stone guarantees. But I am still curious, and wonder if anyone's had any experience (and paid attention to this particular detail) before. In a few weeks, I might be able to come post my own experiences, too.

EDIT: Digging around, I found FeedPollDelay. Looks like 5 seconds by default. So I suppose the answer is "the delay/latency is whatever I want it to be". Which is convenient in terms of RU usage, but mildly disappointing just because it's a polling architecture. Makes sense though :)

azure-cosmosdb
1 Answers

Your question is actually two-fold. The Change Feed is a feature in Cosmos DB that will publish changes as they happen, as described here though:

Only the most recent change for a given item is included in the change log. Intermediate changes may not be available.

So it could occur that if your insert and update operation happen in-between your checks for changes, you'll get the update version instead of 2 separate changes.

On the other hand, it seems you are using the Change Feed Processor, which is a library that helps you consume this endpoint (it's one of several available options). As you mentioned, the CFP Library works as a polling mechanism under the hood:

  1. It polls for changes and sends it to your ProcessChangesAsync implementation, so it feels as a push model from the developer point of view.
  2. After your ProcessChangesAsync implementation finishes, it will poll right away for more changes, with no delay.
  3. If there are more changes, it goes back to #1. If there are no more changes, it will hold for the configured FeedPollTime as you described, and poll again.

This difference is important because if your changes are continuous, then the only delay in the change-detection comes from the speed your ProcessChangesAsync implementation can deal with them.

user_56415980
Related