I have a partition with the following Replicas:
Topic: topicname Partition: 10 Leader: 1 Replicas: 1,2,4,3 Isr: 1,2,3
Where Replica 4 is a non-existent broker. I accidentally added this broker into the replica set as a typo.
I want to remove 4 from the Replica set. but after running kafka-reassign-partitions.sh, the reassignment to remove Replica #4 never finishes.
kafka-reassign-partitions.sh --zookeeper myzookeeperhost:2181 --reassignment-json-file remove4.txt --execute
Where remove4.txt looks like
{ "partitions": [
{ "topic": "topicname", "partition": 2, "replicas": [1,2,3] }
], "version": 1 }
The reassignment is stuck:
kafka-reassign-partitions.sh --zookeeper myzookeeperhost:2181 --reassignment-json-file remove4.txt --verify
Status of partition reassignment:
Reassignment of partition [topicname,10] is still in progress
I checked the controller log, it looks like the reassignment command was picked up, but nothing happens afterwards:
[2017-08-01 06:46:07,653] DEBUG [PartitionsReassignedListener on 101 (the controller broker)]: Partitions reassigned listener fired for path /admin/reassign_partitions. Record partitions to be reassigned {"version":1,"partitions":[{"topic":"topicname","partition":10,"replicas":[1,2,3]}]} (kafka.controller.PartitionsReassignedListener)
Any ideas on what I'm doing wrong? How to I remove broker #4 from the replica set? update: I'm running kafka 10