How to fix an inconsistent clusterId value running kafka in Docker

Viewed 20

I have a docker-compose file that starts 3 zookeeper containers, and 3 kafka containers. In essence, I have a cluster with three brokers.

When running the docker-compose file, I get the following error:

mcflow_kafka_1 | [2022-09-11 03:50:52,600] INFO [feature-zk-node-event-process-thread]: Starting (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
mcflow_kafka_1 | [2022-09-11 03:50:52,747] INFO Updated cache from existing <empty> to latest FinalizedFeaturesAndEpoch(features=Features{}, epoch=0). (kafka.server.FinalizedFeatureCache)
mcflow_kafka_1 | [2022-09-11 03:50:52,750] INFO Cluster ID = 8Z5JcWoMRUC_qnTWg9q1wQ (kafka.server.KafkaServer)
mcflow_kafka_1 | [2022-09-11 03:50:52,755] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
mcflow_kafka_1 | [2022-09-11 03:50:52,755] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
mcflow_kafka_1 | kafka.common.InconsistentClusterIdException: The Cluster ID 8Z5JcWoMRUC_qnTWg9q1wQ doesn't match stored clusterId Some(maywBQemSJuPZzgIh3BE5A) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong.
mcflow_kafka_1 |    at kafka.server.KafkaServer.startup(KafkaServer.scala:252)
mcflow_kafka_1 |    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
mcflow_kafka_1 |    at kafka.Kafka$.main(Kafka.scala:82)
mcflow_kafka_1 |    at kafka.Kafka.main(Kafka.scala)

This is occurring because some file meta.properties is specifying a different clusterId value than what my container's clusterId is. Problem is, I'm not sure how to correct/locate this meta.properties file, since I'm running kafka in a Docker container.

Below is the docker-compose.yml file I'm using:

---
version: "3.0"
services:

  
  zookeeper1:
    image: confluentinc/cp-zookeeper:6.1.0
    restart: always
    environment:
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_CLIENT_PORT: "2181"
      ZOOKEEPER_TICK_TIME: "2000"
      ZOOKEEPER_SERVERS: "zookeeper1:22888:23888"
    ports:
      - "2181:2181"

  zookeeper2:
    image: confluentinc/cp-zookeeper:6.1.0
    restart: always
    environment:
      ZOOKEEPER_SERVER_ID: 2
      ZOOKEEPER_CLIENT_PORT: "2182"
      ZOOKEEPER_TICK_TIME: "2000"
      ZOOKEEPER_SERVERS: "zookeeper2:22888:23888"
    ports:
      - "2182:2181"

  zookeeper3:
    image: confluentinc/cp-zookeeper:6.1.0
    restart: always
    environment:
      ZOOKEEPER_SERVER_ID: 3
      ZOOKEEPER_CLIENT_PORT: "2183"
      ZOOKEEPER_TICK_TIME: "2000"
      ZOOKEEPER_SERVERS: "zookeeper2:22888:23888"
    ports:
      - "2183:2181"




  kafka1:
    container_name: mcflow_kafka_1
    image: confluentinc/cp-kafka:6.1.0
    depends_on:
      - zookeeper1
      - zookeeper2
      - zookeeper3
    ports:
      # Exposes 29092 for external connections to the broker
      # Use kafka1:9092 for connections internal on the docker network
      # See https://rmoff.net/2018/08/02/kafka-listeners-explained/ for details
      #- "29092:29092"
      - "9092:9092"
    environment:
      KAFKA_ZOOKEEPER_CONNECT: "zookeeper1:2181,zookeeper2:2182,zookeeper3:2183"
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: >-
          PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: >-
          PLAINTEXT://kafka1:9092,PLAINTEXT_HOST://localhost:29092
      KAFKA_BROKER_ID: 1
      KAFKA_BROKER_RACK: "r1"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_DELETE_TOPIC_ENABLE: "true"
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
      KAFKA_SCHEMA_REGISTRY_URL: "schemaregistry1:8085"

  kafka2:
    container_name: mcflow_kafka_2
    image: confluentinc/cp-kafka:6.1.0
    depends_on:
      - zookeeper1
      - zookeeper2
      - zookeeper3
    ports:
      # Exposes 29092 for external connections to the broker
      # Use kafka1:9092 for connections internal on the docker network
      # See https://rmoff.net/2018/08/02/kafka-listeners-explained/ for details
      #- "29092:29092"
      - "9093:9092"
    environment:
      KAFKA_ZOOKEEPER_CONNECT: "zookeeper1:2181,zookeeper2:2182,zookeeper3:2183"
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: >-
          PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: >-
          PLAINTEXT://kafka2:9093,PLAINTEXT_HOST://localhost:29092
      KAFKA_BROKER_ID: 2
      KAFKA_BROKER_RACK: "r1"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_DELETE_TOPIC_ENABLE: "true"
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
      KAFKA_SCHEMA_REGISTRY_URL: "schemaregistry1:8085"

  kafka3:
    container_name: mcflow_kafka_3
    image: confluentinc/cp-kafka:6.1.0
    depends_on:
      - zookeeper1
      - zookeeper2
      - zookeeper3
    ports:
      # Exposes 29092 for external connections to the broker
      # Use kafka1:9092 for connections internal on the docker network
      # See https://rmoff.net/2018/08/02/kafka-listeners-explained/ for details
      #- "29092:29092"
      - "9094:9092"
    environment:
      KAFKA_ZOOKEEPER_CONNECT: "zookeeper1:2181,zookeeper2:2182,zookeeper3:2183"
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: >-
          PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: >-
          PLAINTEXT://kafka3:9094,PLAINTEXT_HOST://localhost:29092
      KAFKA_BROKER_ID: 3
      KAFKA_BROKER_RACK: "r1"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_DELETE_TOPIC_ENABLE: "true"
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
      KAFKA_SCHEMA_REGISTRY_URL: "schemaregistry1:8085"

How can I resolve this issue?

0 Answers
Related