Should I use BLPOP, or yield-based busy waiting on hundreds of redis keys?

b1sub 0
Viewed 72

The former sounds good, but I have the following concerns:

  • blocking hundreds of connections might be wasteful, e.g., io multiplexing strategy used, and
  • the redis server should deal with more concurrent connections since each is long-running.

The alternative approach might be the latter:

  • instead of busy waiting indefinitely, yield the thread at every N-th iteration.

Note that the number of connections would increase in proportion to the number of instances. Outside the two, a fixed pool of BLPOP executors can be introduced, but that could easily be the bottleneck if some of the redis list is idle.

nosql architecture multithreading redis performance
1 Answers

I/O multiplexing is not busy waiting

The BLPOP command is an actual command sent to Redis server which involves I/O multiplexing and not busy waiting in the client code

I don't see any point in busy waiting with or without thread yield

For an alternative I suggest using PUB/SUB function supported by Redis, just in case you need to wait on some other events beside list being empty

rostamn7390
Related