How to design a spring boot application that can process 1k requests a second

user_6032753 0
Viewed 4417

I have a rest controller which accepts post requests and returns the statuses of whether the operations are successfull or not. It works fine for 100 requests per second as I have multiple operations underlying it which at the end send the response.

There could be hundreds of users trying to send the requests to the controller, thereby its all done using a completable future and http Async invoker. The problem happens when there are a 1000 requests per second and then the controller threads are exhausted as there are already multiple thread processing multiple requests and all are waiting for there future to be complete and then sending the response.

How can I make my rest controller be able to handle 1000 requests per Second without breaking.

completable-future httpclient software-design spring
2 Answers

there are already multiple thread processing multiple requests and all are waiting for there future to be complete and then sending the response.

You can actually make you controllers asynchronous by making them return a CompletableFuture. Just chain the calls on the CompletableFuture returned by your service to convert them into a an appropriate response instead of using get() or join():

@RequestMapping
public CompletableFuture<ResponseEntity<…>> processRequest() {
    return myService.getStatusFuture()
                    .thenApply(status -> convertToResponseEntity(status));
}

Of course for this to work properly you should have a truly asynchronous service. If you are using @Async or submitting tasks with CompletableFuture.supplyAsync(), this will just move the problem from the HTTP threadpool to another threadpool.

user_5250360

It depends on the servlet server you are using. In the application.properties file, you can use the server.* properties to set the parameters you need.

In this link you can find these properties under the EMBEDDED SERVER CONFIGURATION section. If you are using the default tomcat embedded server, check out the server.tomcat.* properties. Especially the server.tomcat.accept-count, server.tomcat.max-connections and server.tomcat.max-threads properties.

user_94461700
Related