I have a data source provided to me using SignalR, which I’ve never used. I can’t find any documentation on how to ingest it with Spark Streaming, is there a defined process for that?
If not, are there intermediate steps I should take first? Process the data myself from a signal-r client, and create a Kafka producer that’s then reading from that via Structured Streaming?
Alternatively, could try to use airflow but also not sure about that since it’s streaming data.