Need some clarification regarding error handling in kafka connect

user_17015600 0
Viewed 118

I'm looking over the following site KIP:298: Error Handling in Connect

In example 2, what does the following configuration will do? Bit more information or an example can help me out to understand.:-

# retry for at most 10 minutes times waiting up to 30 seconds between consecutive failures
errors.retry.timeout=600000
errors.retry.delay.max.ms=30000

And one more thing is, while dealing with sink connector, when I'm getting some errors due to duplicate records, it keeps on trying for a certain period, how to set our own limit of retries? I tried by setting errors.retry.timeout=0 even though duplicate key error was retrying continuously for certain no.of.times, but if the error is because of schema or serializer it's not retrying.

And finally, errors.log.enable when this is true where does these logs are stored? I was checking in connect log, but not able to find the difference between default log and when the errors.log.enable is set to true.

apache-kafka-connect apache-kafka
1 Answers

Not sure how to fix your problem ,but when errors.log.enable=true, you should see 2 additional topic are created for your connector, yourconnector-error and yourconnector-success, you should be able to see the connector failure message in yourconnector-error topic.

user_190891540
Related