Im working on an application where I need to choose between Hdinsight and event hubs. I have completely searched the internet but could not find any specific comparison between them. I would like to know it from some experienced azure users.
P.S this isn't an opinion based question.
HDInsight installs and runs Kafka on their servers. So whatever Kafka release they install you get the entire feature set from that very release.
On the other hand, Azure Event Hubs support Kafka protocol on its own implementation of event streaming solution. So basically it is not Kafka running behind the scenes but Kafka clients can still talk to this service since client protocol is binary compliant with Kafka. Please note that Event Hubs don't support all Kafka features today. Here is a list of missing features:
Idempotent producer
Transaction Compression
Size-based retention
Log compaction
Adding partitions to an existing topic
HTTP Kafka API
Kafka Streams
