How to replace sets in Redis

userVaty 0
Viewed 25

I have a use case where I have to update sets in Redis every one hour.

Suppose the sets were created with following commands

127.0.0.1:6379> sadd k1 m11 m12 m13 m14
(integer) 4
127.0.0.1:6379> sadd k2 m21 m22 m23 m24
(integer) 4
127.0.0.1:6379> sadd k3 m31 m32 m33
(integer) 3
127.0.0.1:6379> sadd k4 m41
(integer) 1

After 1 hour, I want to update the data to

k1 = {m11, m13, m15}
k2 = {m21, m25}
k5 = {m51, m52}

I found that this can be done using Redis transactions

multi
del k1
sadd k1 m11 m13 m15
exec

multi
del k2
sadd k2 m21 m25
exec

multi
del k5
sadd k5 m51 m52
exec

But now k3 and k4 are still present in Redis. If I don't remove them, they pile up over time. I am solving this by adding 2 hour expiry to every key.

But found one problem with the above approach.Transactions in Redis don't have roll back if a command fails after exec.

So, if del k1 passed and sadd k1 m11 m13 m15 failed, k1 will be removed from Redis entirely. I would prefer to have old data instead of having nothing.

Is there a way to avoid deleting a key when sadd fails?

redis
1 Answers

  1. The possible problem you describe is not that probable, if del k1 is executed then the probability of sadd k1 m11 m13 m15 being executed is pretty high.

  2. There is a solution that involves SUNIONSTORE and SINTERSTORE.

sadd k1 m11 m12 m13 m14 // Setup

multi
sadd kt m11 m13 m15     // Create a temporary key
sunionstore k1 k1 kt    // Store union of k1/kt in k1
sinterstore k1 k1 kt    // Store intersection of k1/kt in k1
del kt                  // Clean kt
exec

You have to note that :

  • It can be pretty CPU intensive/long depending on the length of the sets because of the time complexity of these commands
  • sunionstore and sinterstore commands are interchangeable, the order depends on what you want to optimise. The order I gave is to have the less data loss (your requirement) and the other is to optimize time complexity (the O(N+M) command being executed first) but at the cost of some possible data loss.
JeanJacquesGourdin0
Related