Tomcat API : how to improve performance when client are closing connection after each request?

user_7739375 0
Viewed 210

I have a simple Tomcat API and my goal is to manage the higher number of req /sec.

My problem is the following :

Scenario 1: When the client is using some persistent connections I manage to reach around 20000 req/sec using a single instance of the API. The server is loaded and the CPU of the server is almost fully used.

Scenario 2: When the client is closing connections after each request, the API only manages 600 req/sec and the server resources are not used at all. So I guess there is a bottleneck either on the global number of connections, either on the number of connections the server is able to manage per second.

What I want to know is if there is a configuration (on tomcat or on the server) that I can change to improve performance during scenario 2. If not, which kind of resources is limiting? Can I address the problem by deploying many 1 CPU servers?

What I have looked to for the moment :

  • The number of thread and connection in Tomcat config : I have adjusted theses number from default to 200 threads and 2000 connections, I don't see any effect during scenario2.

  • Ulimit is set to unlimited

  • JVM is configured as follow : JAVA_OPTS: -Xmx8g

tomcat connection api http
1 Answers

It was better if you provide more information about your deployment but generally there are some works that can help you to achieve better performance.
First of all you should measure the cost of each request and optimize it as much as you can. For example, if your API with each request, execute a query on the local database and this query is consuming a lot of CPU usages, you should optimize your query.By doing this your server can tolerate more request before its cpu becomes 100%.
Note some tools like JProbe can help you for optimizing your API.
secondly, monitor your resources during the test and find which one of them becomes fully used. You should check Network Connection, Disk, Memory and CPU loads during the test and identify weakness of your resources. Track thread blocks and deadlocks as they are important to performance. You can scale-up your server resources based on this information or decide to implement distributed architecture or add a load-balancer to your solution or add a caching strategy for you project.
In your Tomcat configuration there some settings which can improve your performance such as :
Configuring connectors
set maxThreads to a high enough value
set acceptCount to a high enough value
Configuring cache
set cacheMaxSize attribute to the appropriate value.
Configuring content compression
turning content compression on and using GZIP compression

user_51080010
Related