How to access Kafka service in Github action?

Viewed 485

Running the most basic test using a Kafka client results in the error NoBrokersAvailable. My Github workflow setup:

  test:
    runs-on: ubuntu-latest
    services:
      zoo_keeper:
        image: bitnami/zookeeper
        ports:
          - 2181:2181
        env:
          ALLOW_ANONYMOUS_LOGIN: yes
        options: >-
          --health-cmd "echo mntr | nc -w 2 -q 2 localhost 2181"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
      kafka:
        image: bitnami/kafka
        ports:
          - 9092:9092
        options: >-
          --health-cmd "kafka-broker-api-versions.sh --version"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        env:
          KAFKA_CFG_KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
          ALLOW_PLAINTEXT_LISTENER: yes
          KAFKA_CFG_LISTENERS: "CLIENT://:9092,INTERNAL://:9093"
          KAFKA_CFG_ADVERTISED_LISTENERS: "CLIENT://localhost:9092,INTERNAL://kafka:9093"
          KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: "CLIENT:PLAINTEXT,INTERNAL:PLAINTEXT"
          KAFKA_CFG_INTER_BROKER_LISTENER_NAME: INTERNAL
    ...
    steps:
      - name: test
        ...
        run: pytest

Running the same setup locally (on docker-windows) and running the tests locally (not from a container) works and successfully connects to Kafka.

My tests use KAFKA_URL=localhost:9092 and run on Github runner host (not a container). In another test, I successfully connect to postgres (using localhost as the hostname). I am sure it's some misconfiguration, what am I missing?

0 Answers
Related