Retry Attempt without data loss when sink side solr is down during runtime

user_14568114 0
Viewed 497
curl -X POST -H "Content-Type: application/json" --data '{
"name": "t1",
"config": {
"tasks.max": "1",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"value.converter.schemas.enable": "false",
"key.converter.schemas.enable": "false",
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
"connector.class": "com.github.jcustenborder.kafka.connect.solr.HttpSolrSinkConnector",
"topics": "TRAN",
"solr.queue.size": "100",
"solr.commit.within": "10",
"solr.url": "http://192.168.2.221:27052/solr/TRAN",
"errors.retry.delay.max.ms":"5000",
"errors.retry.timeout":"600000",
"errors.tolerance":"all",
"errors.log.enable":"true",
"errors.log.include.messages":"false",
"errors.deadletterqueue.topic.name":"DEAD_TRAN",
"errors.deadletterqueue.topic.replication.factor":"1",
"retry.backoff.ms":"1000",
"reconnect.backoff.ms":"5000",
"reconnect.backoff.max.ms":"600000"
}
}' http://localhost:8083/connectors

Need to retry ( without any data loss) based on count from connector config if solr server is down during runtime.

In my case, Its working perfectly whenboth connector and solr are in running state [Active]. But while only solr server is down, there is no retry process until my data passed to the solr leads to data loss..

Error Information shown below enter image description here

Connector Config from the Kafka Connect Log

enter image description here

apache-kafka-connect apache-kafka
1 Answers

I've just checked the SinkTask implementation of that specific connector and it does throw a RetriableException in the put() method.

In theory, and according to your connector configuration, it should block for 10 minutes ("errors.retry.timeout" : "600000"). If your SolR instance recover within the 10 minutes there shouldn't be any problem in terms of data loss.

If you want to fully block your connector until solR is up on his feet, have you tried to set "errors.retry.timeout" : "-1"?

As per the documentation of errors.retry.timeout:

The maximum duration in milliseconds that a failed operation will be reattempted. The default is 0, which means no retries will be attempted. Use -1 for infinite retries.

PS: IMHO this might lead to a deadlock situation if for some reason a single message is permanently failing its sink operation (i.e: if the sink is rejecting the operation).

user_70957510
Related