what happens after a broker is down in a cluster?

Viewed 12967

Suppose a broker is down for long time, now what happens to the followers and leaders that this broker was containing?

  • if broker was containing a leader and one of in-sync replica is chosen as leader, does it create another in-sync replica(in case we have specific replication factor)?

  • if broker was containing a follower, does it create another follower elsewhere in cluster?

  • now suppose broker awakens after long time, now do leaders and followers are restored as it left when it went down?

1 Answers

What happens when a broker is down depends on your configuration. It mostly depends on these configuration settings:

  • min.insync.replicas
  • default.replication.factor
  • unclean.leader.election.enable

Kafka does not create a new replica when a broker goes down.

If the offline broker was a leader, a new leader is elected from the replicas that are in-sync. If no replicas are in-sync it will only elect an out of sync replica if unclean.leader.election.enable is true, otherwise the partition will be offline.

If the offline broker was a follower, it will be marked a out of sync by the leader.

When restarting the broker, it will try to get back in sync. Once done, whether it stays a follower or becomes the leader depends if it is the prefered replica.

Finally if you know a broker will be offline for a long time and still require a replica, you can use the reassignment tool kafka-reassign-partitions.sh to move partitions to online brokers.

Related