Currently, I'm researching ways to deliver messages to destination clients in distributed systems. (Last-mile deliver).
I don't want to implement my own message broker because that solution will not be efficient (how it can be with using Kafka/RabbitMQ etc.), but I can't find out which already existing solution will be more suitable for my purposes.
Important criteria for that system are:
- Delivery guarantee. Not perfect, it's not about banking-like transaction systems, but messages 'costs' much more than typical metric/statistic messages. I can't afford just skip messages if the consumer is not available or the broker fell into error.
- Access to messages by specific rules. It means the broker's system will be public-visible in the global network. Every (or most part) consumer isn't a trusted client, and each of them should be authorized for message consumption. With the specified above criteria I can't imagine which broker will satisfy those criteria as much as possible.
With point 2 in the criteria list, I think I should bind every user to a specific resource(s) by ACL.
After a short research, I see a solution with giving access to topics/queues for specific users by built-in ACL (Kafka or RabbitMQ no matter).
In this regard, I have a question is it a suitable practice to have thousands of topics/queues(for every user)?
Is it allowed by Kafka or RabbitMQ or anything else?
Or maybe it is problem of design or solution choosing?