Performance of multithreaded TCP networking

user_6569596 0
Viewed 709

I'm working on a project using the TCP protocol that may have to work with many 100s or more connections at once.

As such, I am uncertain as to what method I should collect and send this data.

I was wondering whether the principal of more threads = more performance applied here.

My reason for doubt is because all data still has to be fed through the network connection, of which most devices only have 1 active at a time. In addition, I know that repeated context switching can reduce performance as well.

However, I've seen from other sources suggesting that multithreading does indeed scale network performance to a point, and if that's true, why?

Currently, I'm using the Non-Boost variant of ASIO to handle networking.

Thanks in advance for any assistance.

asio tcp networking multithreading c++
3 Answers

ASIO is a wrapper around epoll/IOCP, and as such is optimized for high-performance non-blocking I/O. It's possible to achieve hundreds of thousands of simultaneous connections with this setup on a single thread. Indeed, the old-fashioned "a thread per client" setup could never reach this level is performance due to the context switching overhead.

With that said, depending on the protocol used, handling network requests and replies takes some CPU time, and on a high-rate network it might saturate the single CPU core on which the io_service is running. In that case it is possible to parallelize the io_service so that completion routines can run on more than one core. Still no context switching would take place if the number of threads doesn't exceed the number of available CPU cores/hardware threads. Context switching occurs when the same core needs to handle multiple threads and also when switching between user and kernel mode (i.e. twice for each system call).

Benchmark your server to see how many clients it can handle on a single thread. Chances are it will be enough. Parallelizing io_service comes at a cost of having to deal with completion routines running in parallel, which almost always requires additional synchronization, which means additional overhead.

user_4853430

You want about the same number of threads as you have CPU cores, including hypertreaded ones. Not more.

Each thread deals with a subset of the sockets. That way, you maximize CPU parallelism, but minimize overhead.

If you truly need in the 100s of connections and require low latency, you should consider UDP, where a single socket can receive from many remote addresses. But you then have to implement reliability yourself. Still, that's how multi-player AAA games servers are typically run. And there's good reasons for it.

user_44742300

Multi-Threading vs Single-Threading is a hard topic, and I think it all depends on the point of view of your implementation.
If you have a good event-driven system on one thread probably using single thread for low level network IO will be better. Spawning threads have on itself a performance penalty as the system will need to attend them, of course it will be helpful to use the extra processors, but as you said when finally getting into the low level all threads will need some kind of synchronization, penalty again, unless you are using one socket per thread.

One mayor drawback of multi-threading (one socket per thread) on networks is that most of the time you system will be subject to 'slow loris' attacks. Wikipedia for slow loris Computerphile video on slow loris

So, I think you are better using multi-thread for other long waiting or time consuming tasks. Of course you should use non-blocking IO.

user_138367620
Related