Kafka security: allowing only certain users to read certain messages, within a single topic

user_122792 0
Viewed 588

I have a requirement where we will be "firehosing" a bunch of different providers' financial price feeds into a single "market data" topic in Kafka. Thing is, some of those providers have finely grained permissioning requirements, right down to individual securities. Without creating different topics for each different permissioning grain, is there a way to prevent Kafka clients from receiving information based on what's inside a message? IE can Kafka do fine-grained intra-topic permissioning?

If not, what is the canonical solution to this?

apache-kafka
1 Answers

No. Out of the box Apache Kafka cannot do “fine-grained intra topic permissioning”.

You will have to write a custom Kafka Serializer/Deserializer (SerDes) library or implement your own Entitlements Server in the middle to provide Role Based Access Control (RBAC) and content based filtering.

If you break the data into different topics then yes, basic Kafka ACL can control access to the topic but that was not the original question and you aren’t going to make 6 million topics in kafka.

The original question was about having one topic that mixes different types of data. For that you need to have a client-side deserializer that reads all the data and then filters out certain messages based on what the entitlement server says they are allowed to see.

If you are concerned that client side should never receive data that it is not allowed to receive, even deep in the client-side stack, then you can encrypt the messages with different keys and you can control who gets the keys.

user_49505280
Related