kafka container - Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available

Question

I'm following this guide in order to start a single kafka container .I'm working on win10 pro and my cli is gitbash.

docker-compose-single-broker.yml :

version: '2'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka 
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_CREATE_TOPICS: "test:1:1"
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

The containers are starting without any problem :

$ docker-compose -f docker-compose-single-broker.yml up -d
Creating kafka-docker_zookeeper_1 ... done
Creating kafka-docker_kafka_1     ... done

$ docker ps
CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                                NAMES
19929c6c3297        wurstmeister/kafka       "start-kafka.sh"         17 seconds ago      Up 15 seconds       0.0.0.0:9092->9092/tcp                               kafka-docker_kafka_1
d343a8ecf7ed        wurstmeister/zookeeper   "/bin/sh -c '/usr/sb…"   17 seconds ago      Up 15 seconds       22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp   kafka-docker_zookeeper_1

However, when I try to run any kafka command(create topic,list topics) from inside the container like the following :

> $KAFKA_HOME/bin/kafka-topics.sh --create --topic test --partitions 1 --replication-factor 1 --bootstrap-server `broker-list.sh`
> $KAFKA_HOME/bin/kafka-topics.sh --describe --topic test --bootstrap-server `broker-list.sh`

I'm getting the following warnings and afterwards a timeout exception :

$ ./start-kafka-shell.sh localhost
bash-4.4# $KAFKA_HOME/bin/kafka-topics.sh --create --topic test --partitions 1 --replication-factor 1 --bootstrap-server `broker-list.sh`
[2020-06-09 11:47:26,241] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (/172.17.54.145:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-06-09 11:47:29,306] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (/172.17.54.145:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

The content of the start-kafka-shell.sh :

$ cat start-kafka-shell.sh
#!/bin/bash
docker run --rm -v //var/run/docker.sock:/var/run/docker.sock -e HOST_IP=$1 -e ZK=$2 -i -t wurstmeister/kafka /bin/bash

user_350613 · Answer

I'm not sure what the purpose of start-kafka-shell.sh is. If you're using Docker then you can interact with it from your Windows shell as follows:

docker exec -it tmp_kafka_1 bash -c '$KAFKA_HOME/bin/kafka-topics.sh --create --topic test4 --partitions 1 --replication-factor 1 --bootstrap-server `broker-list.sh`'


docker exec -it tmp_kafka_1 bash -c '$KAFKA_HOME/bin/kafka-topics.sh --describe --topic test4 --bootstrap-server `broker-list.sh`'

Topic: test4    PartitionCount: 1       ReplicationFactor: 1    Configs: segment.bytes=1073741824
        Topic: test4    Partition: 0    Leader: 1001    Replicas: 1001  Isr: 1001

You just need the correct Docker container name (tmp_kafka_1 in my example above), which you can confirm with docker ps.

kafka container - Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available

1 Answers