Can i write some bash command pipeline that reads kafka topic, deserializes protobuf contents and writes it to terminal?

Viewed 568

I have a topic with messages serialized to protobuf. I have a .proto file with contract, and i want to write something similar to

read-kafka brokerAddress topicName startOffset | deserialize-proto myContract.proto

and have some text representation of messages dumped to terminal.

Maybe someone knows easiest way to do something similar to what i want ?

1 Answers

You can read you message (using flag --property line.separator='' to avoid separating lines in the data) and pass the output to the binary protoc

kafka-console-consumer --bootstrap-server $YOU_KAFKA_BROKERS --topic $YOU_TOPIC --from-beginning --max-messages 1 --property line.separator='' | protoc --decode=you.proto.package.Message you/proto/path.proto
Related