Failed to build multi broker kafka with wurstmeister/kafka docker

Viewed 914

I tried several docker-compose.yml to build kafka cluster with wurstmeister docker image on my owner server but still failed so far. Currently, my docker-compose.yml is:

version: "2"
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
  kafka1:
    image: wurstmeister/kafka
    ports:
      - "9095:9092"
    environment:
      KAFKA_ADVERTISED_LISTENERS: INSIDE://:9092,OUTSIDE://10.227.76.112:9095
      KAFKA_LISTENERS: INSIDE://:9092,OUTSIDE://:9095
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
  kafka2:
    image: wurstmeister/kafka
    ports:
      - "9096:9093"
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: INSIDE://:9093,OUTSIDE://10.227.76.112:9096
      KAFKA_LISTENERS: INSIDE://:9093,OUTSIDE://:9096
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
  kafka3:
    image: wurstmeister/kafka
    ports:
      - "9097:9094"
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: INSIDE://:9094,OUTSIDE://10.227.76.112:9097
      KAFKA_LISTENERS: INSIDE://:9094,OUTSIDE://:9097
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

After starting the containers using docker-compose, all container are up: enter image description here

While I tried to connect to my kafka(producing a mocked message) cluster outside the server by Java API, the errors are like

16:14:23.027 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.227.76.112:9094 (id: -1 rack: null) for sending metadata request
16:14:23.027 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.227.76.112:9094 (id: -1 rack: null) using address /10.227.76.112
16:14:23.044 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.227.76.112 disconnected
java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
    at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
    at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:219)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:530)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:485)
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:544)
    at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:325)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:240)
    at java.lang.Thread.run(Thread.java:748)
16:14:23.045 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node -1 disconnected.
16:14:23.045 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (/10.227.76.112:9094) could not be established. Broker may not be available.
16:14:23.045 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker 10.227.76.112:9094 (id: -1 rack: null) disconnected

BTW, the is no firewall opened on my server.

0 Answers
Related