how do you get default Kafka configs global and per topic from command line?

user_469718 0
Viewed 49485

How do you get default Kafka configs global and per topic from the command line or other ways?

I know the defaults are available on Broker Configs and you can get per topic "overrides" using describe, but if I make many changes to topic level configuration, it would be nice to get all values including the overrides per topic.

$KAFKA_HOME/bin/kafka-topics.sh --zookeeper $ZK --describe --topic test-topic
$KAFKA_HOME/bin/kafka-configs.sh --zookeeper $ZK --describe --entity-name test-topic --entity-type topics

FYI, I am just getting started with kafka v0.9.0.1.

Apparently, there is a discussion about why this is not possible, so I guess there is no solution.

apache-kafka
4 Answers

Since Kafka 2.5.0 (see https://issues.apache.org/jira/browse/KAFKA-9040), you can now use the --all option to see all (default and overridden) topic configuration:

% kafka-configs --bootstrap-server <KAFKA_SERVERS> --entity-type topics --entity-name <TOPIC_NAME> --describe --all
All configs for topic <TOPIC_NAME> are:
  compression.type=producer sensitive=false synonyms={}
  message.format.version=1.0-IV0 sensitive=false synonyms={}
  file.delete.delay.ms=60000 sensitive=false synonyms={}
  leader.replication.throttled.replicas= sensitive=false synonyms={}
  max.message.bytes=1000012 sensitive=false synonyms={}
  min.compaction.lag.ms=0 sensitive=false synonyms={}
  message.timestamp.type=CreateTime sensitive=false synonyms={}
  min.insync.replicas=1 sensitive=false synonyms={}
  segment.jitter.ms=0 sensitive=false synonyms={}
  preallocate=false sensitive=false synonyms={}
  index.interval.bytes=4096 sensitive=false synonyms={}
  min.cleanable.dirty.ratio=0.5 sensitive=false synonyms={}
  unclean.leader.election.enable=false sensitive=false synonyms={}
  retention.bytes=-1 sensitive=false synonyms={}
  delete.retention.ms=86400000 sensitive=false synonyms={}
  cleanup.policy=delete sensitive=false synonyms={}
  flush.ms=1000 sensitive=false synonyms={}
  follower.replication.throttled.replicas= sensitive=false synonyms={}
  segment.bytes=1073741824 sensitive=false synonyms={}
  retention.ms=172800000 sensitive=false synonyms={}
  segment.ms=604800000 sensitive=false synonyms={}
  message.timestamp.difference.max.ms=9223372036854775807 sensitive=false synonyms={}
  flush.messages=10000 sensitive=false synonyms={}
  segment.index.bytes=10485760 sensitive=false synonyms={}

As you can see in the PR, you can use this option for (at least) broker config too.

Also note that this option does not currently exist in kafka-topics CLI.

user_70689450

in newer versions of kafka zookeeper connection is not needed, to modify topics just go to the broker, using kafka-topics.sh from https://kafka.apache.org/downloads

kafka-topics.sh --bootstrap-server broker_url --describe --topic test-topic

bin/kafka-topics.sh --bootstrap-server broker_url --list
user_106486170
Related