How to check SSL Kafka topic is authorized using keystore

user_9959152 0
Viewed 1838

I have a simple KafkaProducer to produced into SSL Kafka topic using keystore.jks file

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("acks", "all");
props.put("retries", 0);
props.put("batch.size", 16384);
props.put("linger.ms", 1);
props.put("buffer.memory", 33554432);
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("security.protocol", "SSL");
props.put("ssl.key.password", "password");
props.put("ssl.keystore.password", "password");
props.put("ssl.keystore.location", "/keystore.jks");

Producer<String, String> producer = new KafkaProducer<>(props);

producer.send(new ProducerRecord<String, String>("my-topic", key, body));

But since KafkaProducer is a lazy instance and which creates a connection with a Kafka cluster on the arrival of the first message causing failure at runtime if the topic is unauthorized

Error:

org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [my-topic]

Is there a way to check topic authorization either using KafkaAdminClient or writing custom logic before starting KafkaProducer? I do have below code extracting certs & key from keystore file but couldn't find any next steps

KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());  
char[] pwdArray = "password".toCharArray();
keyStore.load(new FileInputStream("/keystore.jks"), pwdArray);
apache-kafka ssl java
2 Answers

Although you did not configure the necessary properties to trust the Kafka server:

ssl.truststore.location=/var/private/ssl/client.truststore.jks
ssl.truststore.password=test1234

probably your SSL setup is fine although, as you indicated, there is some kind of problem related with the authorization of your client:

org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [my-topic]

Please, consider review the Kafka authorization documentation, especially the part that describes how the user name is derived from the CN of your client SSL certificate:

By default, the SSL user name will be of the form "CN=writeuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown". One can change that by setting ssl.principal.mapping.rules to a customized rule in server.properties. This config allows a list of rules for mapping X.500 distinguished name to short name.

Once the user is identified, check that the necessary ACLs are applied for him and the topic my-topic. Please, see the provided examples in the documentation.

For example, to add the above-mentioned user as a producer of the topic my-topic try something similar to:

bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:writeuser --producer --topic my-topic

You can verify the ACLs applied to a resource, my-topic in your use case, with something like:

bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --list --topic my-topic
user_139424480

After long research and with the help of answer provided by @jccampanero, I have created a block of java code to list ACLS using AdminClient for a specific topic

AdminClient adminClient = KafkaAdminClient.create(configProps);

ResourcePatternFilter resourceFilter = new ResourcePatternFilter(ResourceType.TOPIC,"my-topic", PatternType.ANY);

AclBindingFilter aclBindingFilter = new AclBindingFilter(resourceFilter, AccessControlEntryFilter.ANY);

DescribeAclsResult describe = adminClient.describeAcls(aclBindingFilter);

KafkaFuture<Collection<AclBinding>> values = describe.values();

Collection<AclBinding> aclBindings = values.get();

aclBindings.forEach(acl->{
        System.out.println(String.format("%s %s %s",acl.entry().operation().name(),acl.entry().principal(),acl.entry().permissionType().name()));
    });

Output:

DESCRIBE User:CN=test-cert.com,O=Corporation,OU=1234567,L=Newyork,ST=NY,C=US ALLOW
WRITE User:CN=CN=test-cert.com,O=Corporation,OU=1234567,L=Newyork,ST=NY,C=US ALLOW
READ User:CN=test-cert.com,O=Corporation,OU=1234567,L=Newyork,ST=NY,C=US ALLOW

Now you can extract the CN part from AclBinding and then use it as alias to get the Certificate from JSK file

 KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
 char[] pwdArray = "password".toCharArray();
 keyStore.load(new FileInputStream("/keystore.jks"), pwdArray);
       
 Certificate certificate = keyStore.getCertificate(alias);  //alias = test-cert.com
  if (certificate instanceof X509Certificate) {
      X509Certificate x509cert = (X509Certificate) certificate;

          // Get subject
          Principal principal = x509cert.getSubjectDN();
          String subjectDn = principal.getName();
          System.out.println(principal);  //test-cert.com

       }
user_99591520
Related