kafka started in docker container cannot be accessed by offsetexployer running outside of the container

Viewed 21

I started a kafka service using docker compose:

version: "2"
services:
  zookeeper:
    image: 'bitnami/zookeeper:latest'
    container_name: zookeeper
    ports:
      - 2181:2181
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes

  kafka:
    image: 'bitnami/kafka:2.4.1'
    container_name: kafka
    ports:
      - 29092:9092
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_CFG_LISTENERS=INTERNAL://kafka:9092,EXTERNAL_SAME_HOST://localhost:29092
      - KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://kafka:9092,EXTERNAL_SAME_HOST://localhost:29092
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,EXTERNAL_SAME_HOST:PLAINTEXT
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS=100
    depends_on:
      - zookeeper

Kafka can be started properly and I could access kafka from other service containers defined in the same docker-compose.yml

However when I tried to connect the kafka from an offsetexplorer outside of the docker compose. I've got an error that the broker defined as localhost:29092 is not reachable. zookeeper host is ok.

What I'm trying to accomplish is:

Here's the dilemma, port 9092 is used for other services to access kafka (broker specified as kafka:9092). I need another advertised listener to allow access from e.g. local host instead of services. In this case, I need a different port, I was trying to map the host port 29092 to container port 9092 but it seems like it does not work.


I've found a solution, On the KAFKA_CFG_LISTENERS line: 0.0.0.0 is the key to make it work, localhost or 127.0.0.1 does not:

Kafka access inside and outside docker Follow the above question, I changed my docker-compose.yml file as following and now I can access kafka in side container from offset explorer running on local host outside of the docker compose network.

 kafka:
    image: 'bitnami/kafka:2.4.1'
    container_name: kafka
    ports:
      - '29092:29092'
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_CFG_LISTENERS=INTERNAL://0.0.0.0:9092,OUTSIDE://0.0.0.0:29092
      - KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://kafka:9092,OUTSIDE://localhost:29092
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,OUTSIDE:PLAINTEXT
      - KAFKA_INTER_BROKER_LISTENER_NAME=INTERNAL
      - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
    depends_on:
      - zookeeper
0 Answers
Related