Is there any way to get Redis keys sorted by number of occurences?

user_4281999 0
Viewed 72

I have this set of keys and values that I need to eventually sort by the number of keys' occurences. I'm aware that Redis isn't suppose to work like this, but hoping there may be some smart workaround*.

Schema requirements:

  1. Allow each key to occur any number of times.
  2. Have each value expire after a certain amount of time (and the key with it).
  3. Keep each full pair unique.

Known constraints:

  1. No inbuilt way to expire values, just keys.
  2. Keys can't be duplicated even when they have different values (or can they?)
  3. Using sets or other methods doesn't allow easy counting either (tried that too...)

So apparently the requirements can only be met by grouping both key and value in the Redis key (while assigning them with null/random values and ttls), like this...

Input keys:

"apples:123"
"oranges:123"
"bananas:456"
"apples:456"
"oranges:789"
"apples:789"
[then maybe another hundred or so such pairs]

Expected output: apples, oranges, bananas

[or apples(3), oranges(2), bananas(1) – but I'll then ditch the numbers anyway.]

* while it can be done in app's logic, I think it loses in efficiency as it needs to get all data at once and cycle through each item, when all I need is a rather limited subset. So right now I'd have to do it like this (node.js)...

client.keys('*').then(response => {     
  let occurences = {}
  response.forEach(function (pair){
    let fruit = pair.split(':')[0]
    occurences[fruit] = (occurences[fruit] || 0) + 1
    })
  let topfruits = Object.keys(occurences).sort((a, b) => occurences[a] - occurences[b]).reverse().slice(0, 3)
  console.log(topfruits)
})
// (client.scan in production, which makes it more complicated and doesn't help that much for this use case)

...migrating from a SQL query that does it in one line:

let topfruits = 'SELECT fruit, number, count (fruit) AS occurences FROM fruits GROUP BY fruit ORDER BY occurences DESC LIMIT 3'
redis
1 Answers

This is a great Redisearch Aggregation problem

  1. you can have multiple rows of fruits and counts
  2. They can be expired by using TTLs (EXPIRE command)
  3. These can all be unique (I used order# but it could be a UUID or some other generated informatio)
127.0.0.1:6379> FT.CREATE fruitIndex ON HASH PREFIX 1 fruit_order: SCHEMA fruit TEXT quantity NUMERIC
OK
127.0.0.1:6379> HSET fruit_order:100 fruit bananas quantity 2
(integer) 2
127.0.0.1:6379> HSET fruit_order:101 fruit bananas quantity 200
(integer) 2
127.0.0.1:6379> HSET fruit_order:103 fruit apples quantity 12
(integer) 2
127.0.0.1:6379> FT.AGGREGATE fruitIndex "*" GROUPBY 1 @fruit REDUCE SUM 1 quantity as totals SORTBY 2 @totals DESC
1) (integer) 3
2) 1) "fruit"
   2) "bananas"
   3) "totals"
   4) "202"
3) 1) "fruit"
   2) "oranges"
   3) "totals"
   4) "25"
4) 1) "fruit"
   2) "apples"
   3) "totals"
   4) "12"
127.0.0.1:6379> EXPIRE fruit_order:101 5
(integer) 1

## Wait 5 seconds and re-run the query and you can see that order drop out

127.0.0.1:6379> FT.AGGREGATE fruitIndex "*" GROUPBY 1 @fruit REDUCE SUM 1 quantity as totals SORTBY 2 @totals DESC
1) (integer) 3
2) 1) "fruit"
   2) "oranges"
   3) "totals"
   4) "25"
3) 1) "fruit"
   2) "apples"
   3) "totals"
   4) "12"
4) 1) "fruit"
   2) "bananas"
   3) "totals"
   4) "2"
user_131246290
Related