Generating a unique clientIdPrefix in KafkaListener

user_12444374 0
Viewed 300

I have a Spring boot application running a Kafka consumer that consumes messages from a single Topic having two partitions(partition-0 and 1). The application is deployed as a single Kubernetes pod. I have scaled the application and now it's running in two pods. But the client Id prefix of the consumer running in both pod has the same clientIdPrefix 'client-consumer-0'.

How to generated clientIdPrefix unique to the pod?

spring-kafka spring-boot kubernetes
1 Answers

Set kafka consumer client.id properties for change consumer client name https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html#consumerconfigs_client.id.

You can give property in yml file like below.

spring:
  kafka:
    consumer:
      properties:
        client.id: ${KAFKA_CLIENT_ID:unique-kafka-client-name}

for create unique value for client id you can use pod info https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/#use-pod-fields-as-values-for-environment-variables

i.e metadata.uid

  env:
    - name: KAFKA_CLIENT_ID
      valueFrom:
        fieldRef:
          fieldPath: metadata.uid
user_61710250
Related