Is there a way for reading a redis list in bulks?

user_9039096 0
Viewed 386

Assume we have a redis set with hundreds thousands elements in it. As smember command does eager-loading, it fetches all of the elements just by this one command and consequently it consumes too much time. I want to know is there a way to read redis data as bulks or maybe as a stream?

batch-processing redis python
1 Answers

Bulks

Data from Redis Set data structure can be read in bulks using SSCAN command.

> sadd set a
(integer) 1
> sadd set b
(integer) 1
> sadd set c
(integer) 1

> SMEMBERS set
1) "c"
2) "b"
3) "a"

> SSCAN set 0 COUNT 2
1) "1"                  // <-- cursor for following SSCAN
2) 1) "b"
   2) "a"
> SSCAN set 1 COUNT 2   // <-- cursor from previous SSCAN
1) "0"                  // <-- cursor=0 indicating complete iteration
2) 1) "c"

Stream

There is no direct support from Redis to stream Set structure data.

user_40218020
Related