Apache Kafka incorrect ISR update

Question

I have problem with updating ISR in my Kafka cluster. I have topics with replication-factor 3 but when one node goes down ISR is wrongly update. My Topic settings:

offsets.topic.replication.factor=3
transaction.state.log.replication.factor=3
transaction.state.log.min.isr=2
default.replication.factor=3

Example:

Topic: test     TopicId: o60ayOJlS1KxroaMfwTbOA PartitionCount: 3       ReplicationFactor: 3    Configs: segment.bytes=1073741824
        Topic: test     Partition: 0    Leader: 1       Replicas: 1,2,3 Isr: 1,2,3
        Topic: test     Partition: 1    Leader: 2       Replicas: 2,3,1 Isr: 1,2,3
        Topic: test     Partition: 2    Leader: 3       Replicas: 3,1,2 Isr: 1,2,3
Topic: test2    TopicId: Iroiup4dT-Kmu-aBWJahhw PartitionCount: 9       ReplicationFactor: 3    Configs: segment.bytes=1073741824
        Topic: test2    Partition: 0    Leader: 1       Replicas: 1,2,3 Isr: 1,2
        Topic: test2    Partition: 1    Leader: 2       Replicas: 2,3,1 Isr: 2,1
        Topic: test2    Partition: 2    Leader: 1       Replicas: 3,1,2 Isr: 1,2
        Topic: test2    Partition: 3    Leader: 1       Replicas: 1,3,2 Isr: 1,2
        Topic: test2    Partition: 4    Leader: 2       Replicas: 2,1,3 Isr: 2,1
        Topic: test2    Partition: 5    Leader: 2       Replicas: 3,2,1 Isr: 2,1
        Topic: test2    Partition: 6    Leader: 1       Replicas: 1,2,3 Isr: 1,2
        Topic: test2    Partition: 7    Leader: 2       Replicas: 2,3,1 Isr: 2,1
        Topic: test2    Partition: 8    Leader: 1       Replicas: 3,1,2 Isr: 1,2
Topic: test3 TopicId: D2vaBgOkT1evW2x8xdaCtA PartitionCount: 3       ReplicationFactor: 3    Configs: segment.bytes=1073741824
        Topic: test3 Partition: 0    Leader: 1       Replicas: 1,2,3 Isr: 1,3
        Topic: test3 Partition: 1    Leader: 3       Replicas: 2,3,1 Isr: 1,3
        Topic: test3 Partition: 2    Leader: 3       Replicas: 3,1,2 Isr: 1,3

In my logs i have only information:

WARN [Partition test1-0 broker=1] Failed to update ISR to PendingExpandIsr(isr=Set(1, 2), newInSyncReplicaId=3, sentLeaderAndIsr=LeaderAndIsr(leader=1, leaderEpoch=1, isr=List(1, 2, 3), zkVersion=1)) due to unexpected UNKNOWN_SERVER_ERROR. Retrying. (kafka.cluster.Partition)

Restarting zookeeper and Kafka broker has no efect. Does anybody have similar problem?

Apache Kafka incorrect ISR update

0 Answers