How can I view the failed messages in Ksql stream?

user_16672078 0
Viewed 349

when i use the DESCRIBE EXTENDED in ksqlDB over stream i created, i'm getting the following result:

Local runtime statistics:
consumer-messages-per-sec: 0 | consumer-total-bytes: 101768 | 
consumer-total-messages: 98 | last-message: 2021-08-18T15:27:28.978Z 
| consumer-failed-messages: 1 |consumer-failed-messages-per-sec: 0 | 
   last-failed: 2021-08-18T15:27:29.405Z

The failed message (consumer-failed-messages: 1) is a message that I intentionally sent to the topic and it failed because I sent a different data type than what is defined in the schema to a particular field.

Now i want to view the failed message (getting the actual json that was sent) Is it possible?

The goal is to build a monitor of messages that failed to pass the processing stage.

Thanks, ruth.

ksqldb confluent-schema-registry confluent-platform apache-kafka
1 Answers

Deserialization messages are logged to the ksqlDB processing log: https://docs.ksqldb.io/en/latest/reference/processing-log/

You can set ksql.logging.processing.rows.include to true in order to include the actual messages themselves (encoded as base64) in the processing log message for deserialization errors if you want to.

user_49530790
Related