ksqlDB can't get data from Schema Registry

user_4889150 0
Viewed 318

Case: I have topic in Kafka with name some_table_prefix.table_name. Data is serialized with AVRO, but for historical reasons I have record in Schema Registry named table_name-value.

When I'm trying to setup ksqlDB stream

CREATE STREAM some_stream_name
WITH (KAFKA_TOPIC='some_table_prefix.table_name', VALUE_FORMAT='AVRO');

I'm getting error Schema for message values on topic some_table_prefix.table_name does not exist in the Schema Registry.Subject: some_table_prefix.table_name-value.

I have Schema registry integrated correctly, for others topics everything works ok.

So, is it possible to specify Schema Registry record name in ksqlDB stream creation or resolve this issue some other way?

ksqldb confluent-schema-registry confluent-platform apache-kafka streaming
1 Answers

If you have a topic named table_name, that has Avro being produced to it (which would automatically create table_name-value in the Registry), then that's what ksqlDB should consume from. If you'd manually created that subject by posting the schema on your own, without matching the topic name, then that's part of the problem.

As the error says, it's looking for a specific subject in the Registry based on the topic you've provided. To my knowledge, its not possible to use another subject name, so the workaround is to POST the old subject's schemas into the new one

user_23086830
Related