Java NIO - Accumulation of messages

user_549007 0
Viewed 209

I'm writing a Java application that uses NIO sockets. It consists of 3 servers and a set of clients. A client can communicate with a server, and a server can communicate with a client and the other servers.

Servers to servers and clients to servers sends Messages that are serialize to a byte[] array. The first byte of each Message contains the size of the message and naturally it is guaranteed that each message does not contain more than 127 (2^8 -1) bytes. You can think of the sending of messages of the servers and the clients as operating in a loop:

Message msg = new Message()
while (true) {
    sendMessage(msg, server or client)
    receiveMessage()
}

Then, the implementation utilizes ByteBuffer. Naturally, as in any Java NIO implementation, each server does selector.select() and then retrieves the SelectionKeys to see if reading (calling handleRead() method), writeing (calling handleWrite()), or accepting (calling handleAccept()) needs to be handled. All the handleXX methods take a finite number of steps and do not block waiting for anything else.

When retrieving the data (handleRead()) for a specific key, I just store the data in a specific map Map<SelectionKey, List<byte[]>> readDataForKey; Then I traverse the list and extract all the messages that have been received.

However, I noticed that sometimes when entering handleRead for some key there are thousands of messages waiting to be processed. I cannot figure out why this would be the case? I would expect handleRead to see a couple of messages and that would be it.

The fact that some times thousand of messages get accumulated before they are being processed. What does that mean? Does it mean that my handleRead or handleWrite or some other part of the NIO implementation takes too long and the underlying buffers get full? Does it mean that once in a while I get a GC even (for ~10ms) and in the meantime the buffers get full? Does it mean that I have code that is perhaps slow in handleRead and therefore the messages get accumulated?

Is it normal for so many messages to accumulate?

server client nio java
2 Answers

Your messages are very small. So send and receive buffers can contain a lot of messages. If you do not explicitely specify the size of send and receive buffers the TCP stack will do its best to optimize the sizes for efficient transmission over your network. While default sizes for most Linux distributions are around 128 KB maximal sizes may be multiple megabytes (e.g. on networks with very high latency). Therefore a single handleRead can easily see hundreds or thousands of messages. TCP tries to use the bandwith of your network as good as possible. Seeing a lot of messages in the receive buffer does not automatically mean that your receiver is overloaded. TCP even avoids that your receiver gets overloaded. The only way to measure, whether your system is "overloaded" is to measure the time it takes to send messages (e.g. queue your messages before they are sent and measure its size)

There are cases where it makes sense to optimize buffer sizes manually (e.g. networks with very high latency) but most of the time the TCP stack does a good job here. There are also very rare cases where it makes sense to disable Nagle's algorithm (e.g. Telnet and SSH) to minimize reaction time on user input, but most of the time, there is absolutely no need for manual intervention. Let TCP do its job, which is optimizing for efficient transmission and avoiding that a receiver gets overloaded.

user_135105620

I can't comment so I'll add my comment here...

Does the problem happens with
1. single client and single server? 2 server only (i.e. no clients)?

  1. Single server with multiple clients?

  2. Regarding:

Does it mean that my handleRead or handleWrite or some other part of the NIO implementation takes too long

You can add some logs to measure the time you spend there (or if you want to go further you could use MAT or VisualVM, which might also hint if you are being delayed due to GC)

user_5149490
Related