How to atomically delete keys matching a pattern using Redis

Viewed 530653

In my Redis DB I have a number of prefix:<numeric_id> hashes.

Sometimes I want to purge them all atomically. How do I do this without using some distributed locking mechanism?

32 Answers

Other answers may not work if your key contains special chars - Guide$CLASSMETADATA][1] for instance. Wrapping each key into quotes will ensure they get properly deleted:

redis-cli --scan --pattern sf_* | awk '{print $1}' | sed "s/^/'/;s/$/'/" | xargs redis-cli del

I just had the same problem. I stored session data for a user in the format:

session:sessionid:key-x - value of x
session:sessionid:key-y - value of y
session:sessionid:key-z - value of z

So, each entry was a seperate key-value pair. When the session is destroyed, I wanted to remove all session data by deleting keys with the pattern session:sessionid:* - but redis does not have such a function.

What I did: store the session data within a hash. I just create a hash with the hash id of session:sessionid and then I push key-x, key-y, key-z in that hash (order did not matter to me) and if I dont need that hash anymore I just do a DEL session:sessionid and all data associated with that hash id is gone. DEL is atomic and accessing data/writing data to the hash is O(1).

// TODO

You think it's command not make sense bu some times Redis command like DEL not working correct and comes to the rescue of this

redis-cli KEYS "*" | xargs -i redis-cli EXPIRE {} 1 it's life hack

Please use this command and try :

redis-cli --raw keys "$PATTERN" | xargs redis-cli del

I succeeded this with the simplest variant of EVAL command:

EVAL "return redis.call('del', unpack(redis.call('keys', 'my_pattern_here*')))" 0

where I replaced my_pattern_here with my value.

Adding to this answer:

To find first 1000 keys:

EVAL "return redis.call('scan', 0, 'COUNT', 1000, 'MATCH', ARGV[1])" 0 find_me_*

To delete them:

EVAL "return redis.call('del', unpack(redis.call('SCAN', 0, 'COUNT', 1000, 'MATCH', ARGV[1])[2]))" 0 delete_me_*

I tried most of methods mentioned above but they didn't work for me, after some searches I found these points:

  • if you have more than one db on redis you should determine the database using -n [number]
  • if you have a few keys use del but if there are thousands or millions of keys it's better to use unlink because unlink is non-blocking while del is blocking, for more information visit this page unlink vs del
  • also keys are like del and is blocking

so I used this code to delete keys by pattern:

 redis-cli -n 2 --scan --pattern '[your pattern]' | xargs redis-cli -n 2 unlink 

Below command worked for me.

redis-cli -h redis_host_url KEYS "*abcd*" | xargs redis-cli -h redis_host_url DEL

If you have spaces in your key names, this will work with MacOS

redis-cli --scan --pattern "myprefix:*" | tr \\n \\0 | xargs -0 redis-cli unlink

This one worked for me but may not be atomic:

redis-cli keys "stats.*" | cut -d ' ' -f2 | xargs -d '\n' redis-cli DEL

Ad of now, you can use a redis client and perform first SCAN (supports pattern matching) and then DEL each key individually.

However, there is an issue on official redis github to create a patter-matching-del here, go show it some love if you find it useful!

If you are using Redis version below 4 you might try

redis-cli -h 127.0.0.1 -p 26379 -a `yourPassword` --scan --pattern data:* | xargs redis-cli del

and if you are using the above 4 versions, then

redis-cli -h 127.0.0.1 -p 26379 -a `yourPassword` --scan --pattern data:*| xargs redis-cli unlink

for checking your version enter your Redis terminal by using the following command

redis-cli -h 127.0.0.1 -p 26379 -a `yourPassword

then type

> INFO

# Server
redis_version:5.0.5
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:da75abdfe06a50f8
redis_mode:standalone
os:Linux 5.3.0-51-generic x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:7.5.0
process_id:14126
run_id:adfaeec5683d7381a2a175a2111f6159b6342830
tcp_port:6379
uptime_in_seconds:16860
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:15766886
executable:/tmp/redis-5.0.5/src/redis-server
config_file:

# Clients
connected_clients:22
....More Verbose

If we want to make sure of atom operation we can try to write a Lua script.

If your Redis version support SCAN and UNLINK which is higher than 4.0.0,I prefer to use SCAN and UNLINK instead of Key and DEL in the production environment, because Key and DEL commands might block

they can be used in production without the downside of commands like KEYS or SMEMBERS that may block the server for a long time (even several seconds) when called against big collections of keys or elements.

EVAL "local cursor = 0 repeat local result = redis.call('SCAN', cursor, 'MATCH', ARGV[1])    for _,key in ipairs(result[2]) do  redis.call('UNLINK', key)   end  cursor = tonumber(result[1]) until cursor == 0 " 0 prefix:*

We can change prefix:* as we want.

If you use windows environment please follow this steps and it will definitely works:

  1. Download GOW from here - https://github.com/bmatzelle/gow/wiki (because xargs command doesn't works in windows)

  2. Download redis-cli for Windows (detailed explanation is here - https://medium.com/@binary10111010/redis-cli-installation-on-windows-684fb6b6ac6b)

  3. Run cmd and open directory where redis-cli stores (example: D:\Redis\Redis-x64-3.2.100)

  4. if you want to delete all keys which start with "Global:ProviderInfo" execute this query (it's require to change bold parameters (host, port, password, key) and write yours, because of this is only example):

    redis-cli -h redis.test.com -p 6379 -a redispassword --raw keys "Global:ProviderInfo*" | xargs redis-cli -h redis.test.com -p 6379 -a redispassword del

this is the easiest way that comes to mind without using any xargs magic

pure bash!

redis-cli DEL $(redis-cli KEYS *pattern*)
Related