Latency when establishing 300 tcp connections to server

Viewed 100

We recently integrating some of our old product and doing some load testing, one of the product are using legacy tech .Net remoting (using TCP). And through Process Explorer I discovered each proxy call would create 1 TCP/IP connection.

Both remoting client and server are hosted as Window Service, and we have multiple server and each server at most will only received 4 request at the same time, so I'm expecting the limitation are coming from remoting client, and that's only 1 remoting client are doing all the task, as the client server are designed to be the core middleman.

Here's what I have research and tested so far

  1. Client machine seems good, CPU, memory, Network and IO still have resources to spare
  2. Increased min thread pool
  3. Increased max connection in app.config

The behavior are somewhat similar like thread pool, when active thread are more than the min thread in the setting, all the subsequence task which require thread will be put into the queue, and new thread will be created if thread pool are unable to release free thread on time, but each created thread will introduce approximate 500ms overhead.

The latency started to occur when the it reached around 250 requests, so my question is, does TCP introduce some overhead when too many connection are establishing at the same time just like thread pool?

Being researching this for many days but still no clue yet, any advice or hint will be very much appreciated.

1 Answers

This sounds like a version of the original C10K problem (long since solved).

This happens when all of the connections are dropped into some array and then iterated over regularly to see what needs to be done with them. The original C10K problem was addressed by epoll() and all manner of asynchronous IO trickery, hence.

I'd bet that the legacy .Net code is compiled against Socket.Select() (or something that requires it) and that's the problem you've got.

Check out this tale of woe.

Related