Scenario:
We're synching part of our Firestore database to an ElasticSearch instance. We're using external versioning with some timestamp in milliseconds as version number.
What we're doing:
When a new document is created in Firestore, or when we need to completely overwrite a document, we're synching by calling the INDEX API, which works as expected. https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/api-reference.html#_index Note it does include parameters for
versionandversion_type(which we're setting asexternal).When we only need to update a specific field, we're calling the UPDATE API: https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/api-reference.html#_update We expected it to include
versionandversion_typeas well, but it does not include those parameters. Not setting external versioning is causing the call to return an error:Validation Failed: 1: internal versioning can not be used for optimistic concurrency control. Please use if_seq_no and if_primary_term instead
The problem:
We don't have seq_no nor primary_term at hand. We're counting on optimistic concurrency control based on external versioning by the Firestore update timestamp in milliseconds.
The specific field to update is a calculation that's not even part of the original document in Firestore, so simply overwriting the whole document to ES every time is not an option. We do need to update only one field in ES.
And... the questions:
- Why doesn't the update API include
versionandversion_type? Maybe we're missing something about the concept and this is not the correct approach at all. - If
versionandversion_typeare out of the update API on purpose... what could be the correct approach for this situation? Since we're using external versioning,seq_noandprimary_termwere not supposed to be relevant. Saving information back to Firebase from ES is just cumbersome for what we need.
The client is @elastic/elasticsearch: 7.9.0 in Node, in case that matters.
Thank you very much in advance.