spring kafka how to configure two jaas of different kerberos

Viewed 1960

I have spring boot application, which is recives data over rest, based on some business logic I need to forwaard the data to two different kafka cluster which have their own kerberos keys menttioned jaas file.

I have written two different producer Instance with below properties in their different object instances.

@Service
public class EventProducer {
    private Logger logger = LoggerFactory.getLogger(EventProducer.class);
 
    
    Producer<String, String> kafkaProducer = null;

    @Autowired
    public Producer<String, String> createProducer() {
        if (kafkaProducer == null) {
            Properties props = getKafkaConfig();
         props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "Cluste_1_hostaddress:9092");
         props.put(ProducerConfig.CLIENT_ID_CONFIG,"usertest");

         props.put(ProducerConfig.ACKS_CONFIG, "all");
         props.put(ProducerConfig.RETRIES_CONFIG, "3");

         props.put(ProducerConfig.BUFFER_MEMORY_CONFIG, 33554432);
         props.put(ProducerConfig.BATCH_SIZE_CONFIG, 1600);

         System.setProperty("javax.security.auth.useSubjectCredsOnly", "true"); 
         System.setProperty("java.security.auth.login.config", "/home/user/clusrter_1_jaas.conf);   
         
         props.put("security.protocol", "SASL_PLAINTEXT");
         props.put("kafka.cluster.SecurityProtocol",PLAINTEXTSASL);
         props.put("sasl.kerberos.service.name",  "kafka"); 
         props.put("sasl.kerberos", "sasl.kerberos.service.namekafka");     
         props.put("security.inter.broker.protocol", "SASL_PLAINTEXT");
         props.put("sasl.mechanism.inter.broker.protocol", "PLAIN");
         props.put("sasl.enabled.mechanisms", "PLAIN");
         props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
             
         kafkaProducer = new KafkaProducer<String, String>(props);
        }
        return kafkaProducer;
    }
 
}

Second producer

@Service
public class MovementProducer {
    private Logger logger = LoggerFactory.getLogger(MovementProducer.class);
    
    Producer<String, String> kafkaProducer = null;

    @Autowired
    public Producer<String, String> createProducer() {
        if (kafkaProducer == null) {
            Properties props = getKafkaConfig();
         props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "Cluste_2_hostaddress:9092");
         props.put(ProducerConfig.CLIENT_ID_CONFIG,"usertest");

         props.put(ProducerConfig.ACKS_CONFIG, "all");
         props.put(ProducerConfig.RETRIES_CONFIG, "3");

         props.put(ProducerConfig.BUFFER_MEMORY_CONFIG, 33554432);
         props.put(ProducerConfig.BATCH_SIZE_CONFIG, 1600);

         System.setProperty("javax.security.auth.useSubjectCredsOnly", "true"); 
         System.setProperty("java.security.auth.login.config", "/home/user/clusrter_2_jaas.conf);   
         
         props.put("security.protocol", "SASL_PLAINTEXT");
         props.put("kafka.cluster.SecurityProtocol",PLAINTEXTSASL);
         props.put("sasl.kerberos.service.name",  "kafka"); 
         props.put("sasl.kerberos", "sasl.kerberos.service.namekafka");     
         props.put("security.inter.broker.protocol", "SASL_PLAINTEXT");
         props.put("sasl.mechanism.inter.broker.protocol", "PLAIN");
         props.put("sasl.enabled.mechanisms", "PLAIN");
            props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
         
            kafkaProducer = new KafkaProducer<String, String>(props);
        }
        return kafkaProducer;
    }
 
}

When I start this as two service with enabling only producer instance it works, but when I enable both instance in a single jar only one producer works and other gets authentication issues.

I feel this is due to System.setProperty("java.security.auth.login.config","") , since it is global system variable so it overrides when i use both in single process, so only one works.

So is there any way to solve this issue other than starting a two process. I have only one spring service and should be able to produce to both different kafka cluster ..

1 Answers

Kafka client's latest version has given options for multiple JAAS confs for different clients.

For example, if your instance wants to connect two clusters with different JAAS conf we can override on different producer and consumer levels. Just create 2 separate producer factories and set sasl.jaas.conig

clustera.java.security.auth.login.config=com.sun.security.auth.module.Krb5LoginModule required \
    useKeyTab=true \
    storeKey=true \
    keyTab="test.keytab" \
    principal="test@domain.com";

clusterb.java.security.auth.login.config=com.sun.security.auth.module.Krb5LoginModule required \
    useKeyTab=true \
    storeKey=true \
    keyTab="testb.keytab" \
    principal="testb@domain.com"
Related