Sending data from elasticsearch to kafka and finally to influxdb?

Sarindra Thérèse 0
Viewed 33

I would like to know how can I send data from elasticsearch to kafka and then to influxdb? I've already tried using confluent platform with sources connector from elasticsearch and sink connector from influxdb, but the problem is that I'm stuck on sending data from elasticsearch to kafka

moreover once my computer is off I no longer have the backup of the connectors and I have to start from scratch

that's why my questions:

  • How to send data from elasticsearch to kafka? using confluent platform?

  • Do I really have to use confluent platform if I want to use kafka connect?

apache-kafka-connect apache-kafka elasticsearch
1 Answers

Kafka Connect is Apache 2.0 Licensed and is included with Apache Kafka download.

Confluent (among other companies) write plugins for it, such as Sinks to Elasticsearch or Influx.

It appears the Elasticsearch source on Confluent Hub is not built by Confluent, for example.

Related - Use Confluent Hub without Confluent Platform installation

once my computer is off I no longer have the backup of the connectors and I have to start from scratch

Kafka Connect distributed mode stores its config data in Kafka topics... Kafka defaults to store topic data in /tmp... Which is deleted when you shutdown your computer

Similarly, if you are using Docker for any of these systems without mounted volumes, Docker also is not persistent by default

OneCricketeer0
Related