Ktor delays requests under load

user_289782 0
Viewed 142

I found out that under load my Ktor-Backend processes requests with a long delay. My backend sometimes takes several seconds (can be up to 10-20 seconds) to process a request. I simulate this here:

fun main(args: Array<String>): Unit = EngineMain.main(args)

val logger: Logger = LoggerFactory.getLogger("main")!!

fun Application.health() {
    routing {
        get("/health") {
            val loadString = call.parameters["load"] ?: ""
            logger.info("Health check $loadString")
            Thread.sleep(10000) 
            // This task accesses a db and other services 
            // to get the "Answer"
            call.respond(HttpStatusCode.OK, "Answer")
        }
    }
}

Using a shell script I start multiple such requests simulatenously with a timestamp and index:

i=0
while [ $i -ne 100 ]
do
  i=$(($i+1))
  date=`date +%Y%m%d%H%M%S`
  request="localhost:8080/health?load=$date-$i"
  curl -s -o /dev/null $request&
done

The result in my logs is that for later requests there is an extremely high delay between sending the request and processing it, up to 80 seconds:

11:33:00.222 [nioEventLoopGroup-4-2] INFO  main - Health check 20220525113143-98
11:33:00.222 [nioEventLoopGroup-4-1] INFO  main - Health check 20220525113143-100
11:33:00.222 [nioEventLoopGroup-4-5] INFO  main - Health check 20220525113143-97
11:33:00.222 [nioEventLoopGroup-4-3] INFO  main - Health check 20220525113143-99

Since this backend serves a web frontend I need a strategy to cope with these long delays, eg dropping requests that have not been processed within 15 seconds or better to simply process them sooner. CPU is not an issue, it is always below 10%.

Any suggestions how to deal with this?

ktor kotlin
1 Answers

Thread.sleep actually holds the entire thread. Ktor is built around coroutines, so-called light threads, which allow you not to stop an entire thread but to suspend the current coroutine, so that the current thread can be reused while you wait for the long task to finish.

The analogue of Thread.sleep in the world of coroutines is delay:

delay(10000)
//or
delay(10.seconds)

Note that the task you are going to simulate with such approach should also be non-blocking too - if the real work your server has to do is something like video processing, you need to create a queue and not process many items in parallel, as this will hang your server just like Thread.sleep does.

user_35857960
Related