Kafka: produce/consume structured data

Viewed 25

I am new to Kafka and testing it with Java.

As far I see the messages I produce/consume with Kafka are strings.

Is there some possiblity to do some easy binding to structured data (e.g. a list or map) or is the best way to do so in serializing/deserializing it manually (e.g. in using JSON)?

2 Answers

As far as I see

Unclear what examples you are finding, but Kafka stores bytes. StringSerializer is only one of many Serializer interfaces available and takes a (UTF-8) string to bytes.

do some easy binding to structured data (e.g. a list or map)

Confluent and Apicurio, for example, provide Avro and/or Protobuf serializers for binary serialization of Java objects that integrate with their Schema Registry products.

serializing/deserializing it manually (e.g. in using JSON)?

There is already a built-in JsonSerializer based on Jackson for Kafka clients to use or build into a Serde in Streams. (requires a dependency on connect-api, not just kafka-clients).

Spring-Kafka also has its own JsonSerializer class.

Related