Apologies if my question is all over the place but I am new to Apache Kafka and currently I am trying to import data from a Teradata table (on a VM) into Kafka using my Windows OS. So far I've done the following:
1- Download Apache Kafka
2- Run the Kafka server and Zookeper.
3- Write a producer class in JAVA using IntelliJ IDE.
4- Consume the output of the producer through Kafka's kafka-console-consumer.batscript which comes with the downloaded zip for Apache Kafka.
I have read a few posts and found that to connect Kafka and Teradata I need to use Kafka connect. But a few things confuse me:
1- Most posts I've come across use Confluent. I have read that confluent is for Linux operating systems only. Is there a way to perform my task of ingesting data into Kafka without the use of Confluent?
2- I also know that I have to copy kafka-connect-jdbc jars to Kafka libs folder. Then I have to copy Teradata JDBC jars into Kafka lib folder.
What is the next step for integrating Teradata or any other database with Kafka? How do I create the configuration files in intelliJ and link those with Kafka?