I'm new with both Spark and Kafka...
I need to create an event listener and then process the data to feed a Power Bi report. The solution would be customer hosted in their Azure environment, so I'm wondering if a local configuration could do or if I need to use a cluster. I would like to keep things as simple as possible.
How many servers would I need as a minimum? The amount of data to process would be around 200Gb per year.
Also, is there a recommended way to deploy the solution to the customer's tenant? It's only purpose would be to work in the backend to get notifications and process the data with Spark.