How does Kafka identify an idle connection?

Viewed 183

I'm doing some tests on my producer (springboot), sending requests every 30 seconds for 1 hour.

I notice that after a few minutes (~10 minutes), with TRACE enabled, I have the error below:

        2022-07-18 19:15:25.025 DEBUG 12991 --- [streams-kafka-1] o.apache.kafka.common.network.Selector   : [Producer clientId=event-streams-kafka-1] Connection with localhost/127.0.0.1 disconnected

java.io.EOFException: null
    at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:97) ~[kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:452) ~[kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:402) ~[kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:674) [kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:576) [kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.common.network.Selector.poll(Selector.java:481) [kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:551) [kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:328) [kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:243) [kafka-clients-3.0.0.jar:na]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_311]

2022-07-18 19:15:25.027 TRACE 12991 --- [streams-kafka-1] o.apache.kafka.common.network.Selector   : [Producer clientId=event-streams-kafka-1] Read from closing channel failed, ignoring exception

java.io.EOFException: null
    at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:97) ~[kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:452) ~[kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:402) ~[kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:674) [kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.common.network.Selector.maybeReadFromClosingChannel(Selector.java:700) [kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.common.network.Selector.close(Selector.java:935) [kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:625) [kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.common.network.Selector.poll(Selector.java:481) [kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:551) [kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:328) [kafka-clients-3.0.0.jar:na]
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:243) [kafka-clients-3.0.0.jar:na]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_311]

2022-07-18 19:15:25.027 DEBUG 12991 --- [streams-kafka-1] org.apache.kafka.clients.NetworkClient   : [Producer clientId=event-streams-kafka-1] Node -2 disconnected.

I checked in Kafka documentations, that the connections.max.idle.ms parameter could solve my idle disconnects problem.

I changed the connections.max.idle.ms=-1 parameter on the application side and got no improvement.

When I made the change from the broker side, the error stopped occurring.

But my big question is, what makes Kafka understand that my connection is idle? Since my application has been sending requests to Kafka every 30 seconds, is there any way to send a signal to kafka to prevent it from closing my connection?

This is a problem for me, as changing this parameter on the broker side will affect all applications that use Kafka in a productive environment. I don't feel comfortable with that.

1 Answers

The usual occurrence of this issue just because any old KAFKA clients trying to connect to the broker which is having a latest version, for the idle connection scenarios. As you mentioned, the broker side setting idle.ms would eventually fix this issue. But the recommendations from the community, is to upgrade your kafka client to a latest version, which handles the idle connection appropriately behind the scene.

More details here : https://issues.apache.org/jira/browse/KAFKA-3205

Related