I have an application that receives events on different threads, each event is processed, formatted into different attributes, and saved into ElasticSearch. I would like to add a new entry in my DB when a specific attribute value first appears on any event (each event reads all previous events and verifies the value did not yet appear). Due to the multithreaded nature of the application i need some way of validating that I am adding the entry when i first encounter the attribute, and only the once. The best solution that comes to my mind is some kind of optimistic locking (making sure no other thread has added an entry for this attribute before i commit it). Is there any ElasticSearch support for this?
1 Answers
Yes, Elasticsearch provides supports for dealing with conflicts, And you can read the official Elasticsearch https://www.elastic.co/guide/en/elasticsearch/guide/master/version-control.html docs which explain this in details.
If we don't deal properly with conflicts then, it can lead to lost update problem and how to deal with them explained below
In the database world, two approaches are commonly used to ensure that changes are not lost when making concurrent updates:
Pessimistic concurrency control: Widely used by relational databases, this approach assumes that conflicting changes are likely to happen and so blocks access to a resource in order to prevent conflicts. A typical example is locking a row before reading its data, ensuring that only the thread that placed the lock is able to make changes to the data in that row.
Optimistic concurrency control Used by Elasticsearch, this approach assumes that conflicts are unlikely to happen and doesn’t block operations from being attempted. However, if the underlying data has been modified between reading and writing, the update will fail. It is then up to the application to decide how it should resolve the conflict. For instance, it could reattempt the update, using the fresh data, or it could report the situation to the user.
Good news for you is that Elasticsearch supports optimistic locking hence doesn't lock all the documents and provides a better performance, You can read their official doc https://www.elastic.co/guide/en/elasticsearch/guide/master/optimistic-concurrency-control.html on how to achieve that.
Let me know if it's clear to you and have any doubts.
Related