kafka connector HTTP/API source

user_1690863 0
Viewed 4690

I am actually aware on how to capture data from any data source, such as a specific API (e.g HTTP GET request) and ingest them in specific kafka connector. 

{
    "name": "localfileSource", 
    "config": {
        "connector.class": "FileStreamSourceConnector",
        "tasks.max": "1",
        "file": "test.txt",
        "topic": "connectSource"
    }
}

I would need something similar to this(FileStreamSourceConnector), that can be used with API sources.

apache-kafka-connect apache-kafka
2 Answers

It's probably a bit late for the question asker, but I had a similar need and I decided to implement my own connector.

The particularity of the connector is that it implements some sort of Change Data Capture

In case it can help anyone else, you can find it here: castorm/kafka-connect-http

user_9090880
Related