RavenDB: How to properly update a document at the same time? (two simultaneous API request to the same endpoint)

Viewed 122

I have a C# REST API with an upload endpoint that has the sole purpose to process a binary file and add its metadata (as an Attachment model) to a List<Attachment> property of a different entity.

When I call the endpoint from my web application in a sequential manner like below (pseudo code), the endpoint does as intended and processes each binary file and adds a single Attachment to the provided entity.

const attachments = [Attachment, Attachment, Attachment];

for(const attachment of attachments) {
    await this.api.upload(attachment);
}

But when I try to upload the attachments in a parallel manner like below (pseudo code), each binary file gets processed properly, but only one Attachment metadata object gets added to the entity.

const attachments = [Attachment, Attachment, Attachment];

const requests = attachments.map((a) => this.api.upload(a));
await Promise.all(requests);

The endpoint basically does the following (simplified):

var attachment = new Attachment() 
{
    // Metadata is collected from the binary (FormFile)
};

using (var session = Store.OpenAsyncSession())
{
    var entity = await session.LoadAsync<Entity>(entityId);

    entity.Attachments.Add(attachment);

    await session.StoreAsync(entity);                   
    await session.SaveChangesAsync();
};

I suspect that the problem is that the endpoint is called at the same time. Both request open (at the same time) a database session and query the entity into memory. They each add the Attachment to the entity and update it in the database. The saved attachment you see in the database is from the request that finishes last, e.g. the request that takes the longest.

I've tried to recreate the issue by creating this example. When you open the link, the example runs right away. You can see the created entities on this database server.

Open the Hogwarts database and after that open the contact Harry Potter and you see two attachments added. When you open the contact Hermione Granger you only see the one attachment added (the Second.txt), although it should also have both attachments.

What is the best approach to solve this issue? I prefer not having to send the files as a batch to the endpoint. Appreciate any help!

PS: You might need to run the example manually by clicking on Run. If the database doesn't exist on the server (as the server gets emptied automatically) you can create it manually with the Hogwarts name. And because it looks like a race condition, sometimes both Attachment items are added properly. So you might need to run the example a few times.

1 Answers

That is a a fairly classic example of a race condition in writing to the database, you are correct.

The sequence of event is:

  1. Req 1 load doc Attachments = []
  2. Req 1 load doc Attachments = []
  3. Req 1 Attachments.Push()
  4. Req 2 Attachments.Push()
  5. Req 1 SaveChanges()
  6. Req 2 SaveChanges()

The change in 5 overwrites the change in 4, so you are losing data.

There are two ways to handle this scenario. You can enable optimistic concurrency for this particular scenario, see the documentation on the topic:

https://ravendb.net/docs/article-page/4.2/csharp/client-api/session/configuration/how-to-enable-optimistic-concurrency#enabling-for-a-specific-session

Basically, you can do session.Advanced.UseOptimisticConcurrency = true; to cause the transaction to fail if the document was updated behind the scenes.

You can then retry the transaction to make it work (make sure to create a new session).

Alternatively, you can use the patching API, which will allow you to add an item to the document concurrently safely. Here is the relevant documentation:

https://ravendb.net/docs/article-page/4.2/csharp/client-api/operations/patching/single-document#add-item-to-array

Note that there is a consideration here, you shouldn't care what the order of the operations are (because they can happen in any order). If there is a business usecase behind the order, you probably cannot use the patch API easily and need to go with the full transaction route.

Related