Ratelimiting with fairness/sharing

user_4101264 0
Viewed 195

I am looking at any library/implementation which implements rate-limiting like Guava ratelimiter. However I am also trying to ensure fairness.

For example, lets say there are 3 queues Q1, Q2, Q3 each receiving messages. The processing rate of the system can only support 1000 messages per second.

With guava I can configure a token based rate limiter that will gives a constant 1000 tokens per second. However, there is no guarantee that Q1, Q2 and Q3 will receive same number of tokens per second.

A simple way would be to create 3 rate limiters with 1000/3 = 333 tokens per second and assign them to each queue. However, in that case, the system becomes sub-optimal.

Is there any easy or simple way to achieve this without writing code from scratch ? Ideally I would want to limit each queue to a set rate - for example Q1 is allowed upto 100 TPS, Q2 - 200 TPS, Q3: 700 when the system is at full capacity.

rate-limiting guava java
1 Answers

I have come up with one approach taking advantage of the queues being used in this case.

To drive fairness, the processing rate should be proportional to the the number of messages in the queue waiting to be processed. For example, lets say Q1 has 1M messages while Q2 has 9 M messages and lastly Q3 is empty. Then TPS for Q1 should be (1 / (1+9))*1000 = 100 TPS while for Q2 it should be 900.

We can ensure fairness by creating one rate limiter instance per queue. Workers for each queue will use their own rate limiter instance. All rate limiters will initially start with equal rate limits.

A background thread will fetch queue sizes every x minutes (say 30 minutes). The background thread will update the rates of all rate-limiters after every 30 minutes.

user_41012640
Related