I am trying to implement an atomic distributed counter using Redis. My usecase is like this:
- There is a publishing application which publishes messages to
RabbitMQ. There can be multiple instances of the publisher running, hence they can publish in parallel. These publishers do not know in advance how many messages they are going to publish. - There is a consuming applications which subscribes to these messages from
RabbitMQ. There can be multiple instances of consumer running which can consume in parallel.
My problem is, once the last message is consumed by consumer application I want to take certain action (such as publishing one more message).
The solution I cameup with is to implement a sort of distributed reference counting using Redis. The idea is that publisher will increment the count in Redis as soon as it publishes the message and consumer will decrement the count in Redis as soon as it finishes processing. After decrement if the count is 0 that means it is the last message.
However, when I tried to implement a sample application to test this out, I see the increments are not working as expected. This is my sample code:
const string key = "MyKey";
ConnectionMultiplexer redis = ConnectionMultiplexer.Connect("localhost");
var db = redis.GetDatabase();
var guid = Guid.NewGuid().ToString();
Parallel.For(0, 1000, async (_) =>
{
var transaction = db.CreateTransaction();
var v = transaction.StringIncrementAsync(key + "-Count").ConfigureAwait(false);
await transaction.ExecuteAsync();
await v;
});
I expected the value in the Redis to be "1000" after the completion. However, it has value such as "nil" or "4". Am I missing something? Is it possible to do this with Redis? Note that correct value of the counter is important to me and cannot miss increment/decrement operations. I am using StackExchange Redis library as client.
EDIT
I could get it working by changing body of loop to
db.ScriptEvaluate(@"redis.call('incr', KEYS[1])", new RedisKey[] {key});.
Can anyone comment whether this is guaranteed to work or its working purely by luck?