Mongo throwing write conflicts for concurrent sessions

user_2804884 0
Viewed 839
 session.StartTransaction();
 await mongo.Collection1.UpdateOneAsync(session, filter1, update1);
 await mongo.Collection2.BulkWriteAsync(session, updatesToDifferentDocs);
 await mongo.Collection3.UpdateOneAsync(session, filter2, update2);
 await session.CommitTransactionAsync();

The above code is running concurrently on multiple threads. The final update for Collection3 has a high chance of writing on the same document by multiple threads. I wanted the transactions across the 3 collections to be atomic which is why I put them in one session, which is what I thought session is essentially used for, however, I'm not familiar with the details of its inner-workings.

Without knowing much about the built-in features of Mongo. It's pretty obvious why this is giving me a write conflict. I simply can't write to the same document in Collection3 at the same time on multiple threads.

However, I tried Googling a bit and it seems like Mongo >= 3.2 has WiredTiger Storage Engine by default which has Document level locks that doesn't need to be used by the developer. I've read that it automatically retries the transaction if the document was initially locked.

I don't really know if I'm using session incorrectly here, or I just have to manually implement some kind of lock/semaphore/queue system. Another option would be to manually check for write conflict and re-attempt the entire session. But it feels like I'm just reinventing the wheel here if Mongo is already supposed to have concurrency support.

mongodb .net c#
2 Answers

the automatic retry behavior is different for transactions in mongodb. by default a transactional write operation will wait for 5ms and aborts the transaction if a lock cannot be aquired.

you could try increasing the 5ms timeout by running the following admin command on your mongodb instance:

db.adminCommand( { setParameter: 1, maxTransactionLockRequestTimeoutMillis: 100 } )

more details about that can be found here.

alternatively you could manually implement some retry logic for failed transactions.

user_43684850

Should have updated this thread earlier but nonetheless, here is what I ended up doing to solve my problem. While MongoDB does have automatic retries to transactions along with some locking mechanisms, I couldn't find a clean way to do leverage this for my specific problematic session. Kept getting write conflicts even though I thought I'd acquired locks on all the colliding documents start of each session.

Having to maintain atomicity for a session that reads and writes across multiple collections not just documents, I thought it was cleaner to simply wrap it in custom retry logic. I followed the example bottom of page here and used a timeout that I thought was reasonable for my use case.

I decided to use a timeout-based retry logic because I knew most of the collisions would be tightly packed together temporally. For less predictable collisions, maybe some queue-based mechanism would be better.

user_28048840
Related