How can I get an OkHttpClient to comply with a REST API's rate limits?

user_15420176 0
Viewed 710

I'm writing an Android app that makes frequent requests to a REST API service. This service has a hard request limit of 2 requests per second, after which it will return HTTP 503 with no other information. I'd like to be a good developer and rate limit my app to stay in compliance with the service's requirements (i.e, not retry-spamming the service until my requests succeed) but it's proving difficult to do.

I'm trying to rate limit OkHttpClient specifically, because I can cleanly slot an instance of a client into both Coil and Retrofit so that all my network requests are limited without me having to do any extra work at the callsites for either of them: I can just call enqueue() without thinking about it. And then it's important that I be able to call cancel() or dispose() on the enqueue()ed requests so that I can avoid doing unnecessary network requests when the user changes the page, for example.

I started by following an answer to this question that uses a Guava RateLimiter inside of an OkHttp Interceptor, and it worked perfectly! Up until I realized that I needed to be able to cancel pending requests, and you can't do that with Guava's RateLimiter, because it blocks the current thread when it acquire()s, which then prevents the request from being cancelled immediately.

I then tried following this suggestion, where you call Thread.interrupt() to get the blocked interceptor to resume, but it won't work because Guava RateLimiters block uninterruptibly for some reason. (Note: doing tryAcquire() instead of acquire() and then interruptibly Thread.sleep()ing isn't a great solution, because you can't know how long to sleep for.)

So then I started thinking about scrapping the Guava solution and implementing a custom ExecutorService that would hold the requests in a queue that would be periodically dispatched by a timer, but it seems like a lot of complicated work for something that may or may not work and I'm way off into the weeds now. Is there a better or simpler way to do what I want?

okhttp kotlin android java
3 Answers

Ultimately I decided on not configuring OkHttpClient to be ratelimited at all. For my specific use case, 99% of my requests are through Coil, and the remaining handful are infrequent and done through Retrofit, so I decided on:

  • Not using an Interceptor at all, instead allowing any request that goes through the client to proceed as usual. Retrofit requests are assumed to happen infrequently enough that I don't care about limiting them.
  • Making a class that contains a Queue and a Timer that periodically pops and runs tasks. It's not smart, but it works surprisingly well enough. My Coil image requests are placed into the queue so that they'll call imageLoader.enqueue() when they reach the front, but they can also be cleared from the queue if I need to cancel a request.
  • If, after all that, I somehow exceed the rate limit by mistake (technically possible, but unlikely,) I'm okay with OkHttp occasionally having to retry the request rather than worrying about never hitting the limit.

Here's the (very simple) queue I came up with:

import java.util.*

class RateLimitedQueue(private val millisecondsPerTask: Long) {
    private val timer = Timer()
    private val queue = ArrayDeque<Task>()

    init {
        timer.scheduleAtFixedRate(RunTaskTimerTask(this), 0, millisecondsPerTask)
    }

    private class RunTaskTimerTask(val parent: RateLimitedQueue) : TimerTask() {
        override fun run() {
            synchronized(parent.queue) {
                if (!parent.queue.isEmpty())
                    parent.queue.pop().run()
            }
        }
    }

    fun interface Task {
        fun run()
    }

    fun add(t: Task) {
        synchronized(queue) {
            queue.add(t)
        }
    }

    fun remove(t: Task) {
        synchronized(queue) {
            queue.remove(t)
        }
    }

    fun removeAll(filter: (Task) -> Boolean) {
        synchronized(queue) {
            queue.removeAll(filter)
        }
    }
}

I'm satisfied with this solution, but there's some notable caveats:

  • The Timer+Queue isn't very smart: it literally just checks for a task every X milliseconds, so if a task arrives at an empty queue between a long delay, it might wait unnecessarily. You could come up with something more elegant, but I found it to work well enough for my delay of 500 milliseconds.
  • It's not as clean as an Interceptor, in that it isn't shared between Retrofit and Coil.
  • It's technically possible to exceed the rate limits, but for my case, that's acceptable.
  • You can't cancel tasks after they've reached the front of the queue; you can only remove them from the queue before they are executed.
user_154201760

Create a queue that wraps your comm calls that uses a timer/delayed coroutine that terminates the check for the next calls once the queue is empty (this way you don't have a timer running unnecessarily). I've built logic like this both in mobile and also on the desktop ... simple enough to IMPL and gets the job done.

user_19708960
user_104016980
Related