How do I delete/clean Kafka queued messages without deleting Topic

user_8606510 0
Viewed 61506

Is there any way to delete queue messages without deleting Kafka topics?
I want to delete queue messages when activating the consumer.

I know there are several ways like:

  1. Resetting retention time

    $ ./bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic MyTopic --config retention.ms=1000

  2. Deleting kafka files

    $ rm -rf /data/kafka-logs/<topic/Partition_name>

apache-kafka
2 Answers

To delete all messages in a specific topic, you can run kafka-delete-records.sh

For example, I have a topic called test, which has 4 partitions.

Create a Json file , for example j.json:

{

"partitions": [

    {

        "topic": "test",

        "partition": 0,

        "offset": -1

    }, {

        "topic": "test",

        "partition": 1,

        "offset": -1

    }, {

        "topic": "test",

        "partition": 2,

        "offset": -1

    }, {

        "topic": "test",

        "partition": 3,

        "offset": -1

    }

],

"version": 1

}

now delete all messages by this command : 

/opt/kafka/confluent-4.1.1/bin/kafdelete-records --bootstrap-server 192.168.XX.XX:9092 --offset-json-file j.json

After executing the command, this message will be displayed

Records delete operation completed:
partition: test-0   low_watermark: 7
partition: test-1   low_watermark: 7
partition: test-2   low_watermark: 7
partition: test-3   low_watermark: 7
user_44607370
Related