Ktor, Netty and increasing the number of threads per endpoint

Question

Using Ktor and Kotlin 1.5 to implement a REST service backed by Netty. A couple of things about this service:

"Work" takes non-trivial amount of time to complete.
A unique client endpoint sends multiple requests in parallel to this service.
There are only a handful of unique client endpoints.

The service is not scaling as expected. We ran a load test with parallel requests coming from a single client and we noticed that we only have two threads on the server actually processing the requests. It's not a resource starvation problem - there is plenty of network, memory, CPU, etc. and it doesn't matter how many requests we fire up in parallel - it's always two threads keeping busy, while the others are sitting idle.

Is there a parameter we can configure to increase the number of threads available to process requests for specific endpoints?

user_1820583 · Answer

Netty use what is called Non-blocking IO model (http://tutorials.jenkov.com/java-concurrency/single-threaded-concurrency.html).

In this case you have only a single thread and it can handle a lot of sub-processes in parallel, as long as you follow best practices (not blocking the main thread event loop).

user_473471 · Answer

You might need to check the following configuration options for Netty https://ktor.io/docs/engines.html#configure-engine

connectionGroupSize = x
workerGroupSize = y
callGroupSize = z

Default values usually are set rather low and tweaking them could be useful for the time-consuming 'work'. The exact values might vary depending on the available resources.

Ktor, Netty and increasing the number of threads per endpoint

2 Answers