I have setup strimzi kafka on azure AKS and trying to access it from outside using TLS authentication but facing below error.We are using our own certificates and configured under listener section.
kafka yaml:
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: eesb-cluster
spec:
kafka:
version: 3.1.0
replicas: 3
logging:
type: inline
loggers:
kafka.root.logger.level: "INFO"
resources:
requests:
memory: 2Gi
cpu: "1"
limits:
memory: 4Gi
cpu: "2"
readinessProbe:
initialDelaySeconds: 15
timeoutSeconds: 5
periodSeconds: 10
successThreshold: 1
failureThreshold: 3
livenessProbe:
initialDelaySeconds: 15
timeoutSeconds: 5
periodSeconds: 10
successThreshold: 1
failureThreshold: 3
template:
pod:
imagePullSecrets:
- name: artifactory-kafka
# Kafka JVM
jvmOptions:
gcLoggingEnabled: true
-Xms: 1g
-Xmx: 2g
-XX:
UseG1GC: true
MaxGCPauseMillis: 20
InitiatingHeapOccupancyPercent: 35
ExplicitGCInvokesConcurrent: true
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: external
port: 9093
type: loadbalancer
tls: true
authentication:
type: tls
configuration:
bootstrap:
loadBalancerIP: 20.22.199.99
annotations:
external-dns.alpha.kubernetes.io/hostname: bootstrap.example.com
alternativeNames:
- bootstrap.example.com
brokers:
- broker: 0
loadBalancerIP: 20.22.199.100
annotations:
external-dns.alpha.kubernetes.io/hostname: kafka-0.example.com
advertisedHost: kafka-0.example.com](url)
- broker: 1
loadBalancerIP: 20.22.199.101
annotations:
external-dns.alpha.kubernetes.io/hostname: kafka-1.example.com
advertisedHost: kafka-1.example.com
- broker: 2
loadBalancerIP: 20.22.199.102
annotations:
external-dns.alpha.kubernetes.io/hostname: kafka-2.example.com
advertisedHost: kafka-2.example.com
brokerCertChainAndKey:
secretName: source-kafka-listener-cert
certificate: tls.crt
key: tls.key
config:
num.partitions: 1
offsets.topic.replication.factor: 3
log.retention.hours: 24
log.segment.bytes: 1073741824
log.retention.check.interval.ms: 300000
num.network.threads: 3
num.io.threads: 8
transaction.state.log.replication.factor: 3
transaction.state.log.min.isr: 2
default.replication.factor: 3
min.insync.replicas: 2
num.recovery.threads.per.data.dir: 1
inter.broker.protocol.version: "3.1"
auto.create.topics.enable: false
delete.topic.enable: true
# Rack awareness
# replica.selector.class: org.apache.kafka.common.replica.RackAwareReplicaSelector
# broker.rack:
#rack:
# topologyKey: topology.kubernetes.io/region=
# topologyKey: topology.kubernetes.io/zone=
storage:
type: jbod
volumes:
- deleteClaim: false
id: 0
size: 80Gi
type: persistent-claim
# Kafka Prometheus JMX Exporter
#metricsConfig:
#type: jmxPrometheusExporter
#valueFrom:
#configMapKeyRef:
#key: kafka-metrics-config.yml
#name: kafka-zk-metrics
#metrics:
#lowercaseOutputName: true
#kafkaExporter:
#groupRegex: .*
#topicRegex: .*
# Kafka JMX metrics
#jmxOptions:
#authentication:
#type: "password"
zookeeper:
replicas: 3
storage:
type: persistent-claim
size: 10Gi
deleteClaim: false
# Zookeeper loggers and log level
#logging:
#type: external
#name: kafka-logging-configmap
logging:
type: inline
loggers:
zookeeper.root.logger: "INFO"
resources:
requests:
memory: 1Gi
cpu: "1"
limits:
memory: 2Gi
cpu: "2"
# Zookeeper JVM
jvmOptions:
gcLoggingEnabled: true
-Xms: 1g
-Xmx: 1g
-XX:
UseG1GC: true
MaxGCPauseMillis: 20
InitiatingHeapOccupancyPercent: 35
ExplicitGCInvokesConcurrent: true
I have setup same kafka cert in authentication section of kafka connect and added all the CA certicate in truststore on the client side.
Kafka connect client yaml:
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
name: eesb-connect-cluster
annotations:
strimzi.io/use-connector-resources: "true"
spec:
version: 3.1.0
replicas: 2
image: packages.alm.eurofins.com/docker-eurofins-eesb-prerelease/com/eurofins/eesb/eesb-kafka-connect:0.0.1-SNAPSHOT
bootstrapServers: itaag108-uat2.eesb.eurofins.com:9093
authentication:
type: tls
certificateAndKey:
certificate: tls.crt
key: tls.key
secretName: source-kafka-listener-cert
tls:
trustedCertificates:
- certificate: ca1.crt
secretName: source-kafka-trust-cert
- certificate: ca2.crt
secretName: source-kafka-trust-cert
- certificate: ca3.crt
secretName: source-kafka-trust-cert
config:
key.converter: org.apache.kafka.connect.json.JsonConverter
value.converter: org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable: true
value.converter.schemas.enable: true
offset.flush.timeout.ms: 120000
offset.flush.interval.ms: 10000
group.id: connect-cluster
offset.storage.topic: connect-cluster-offsets
config.storage.topic: connect-cluster-configs
status.storage.topic: connect-cluster-status
# -1 means it will use the default replication factor configured in the broker
config.storage.replication.factor: 2
offset.storage.replication.factor: 2
status.storage.replication.factor: 2
config.providers: file
config.providers.file.class: org.apache.kafka.common.config.provider.FileConfigProvider
resources:
requests:
cpu: "1"
memory: 512m
limits:
cpu: "2"
memory: 2Gi
logging:
type: inline
loggers:
log4j.rootLogger: "INFO"
readinessProbe:
initialDelaySeconds: 60
periodSeconds: 20
timeoutSeconds: 30
livenessProbe:
initialDelaySeconds: 60
periodSeconds: 20
timeoutSeconds: 30
template:
pod:
imagePullSecrets:
- name: artifactory-kafka
externalConfiguration:
volumes:
- name: connector-config
secret:
secretName: target-uat-db-auth
jvmOptions:
gcLoggingEnabled: false
-Xmx: 1024m
-Xms: 512m
-XX:
UseG1GC: true
MaxGCPauseMillis: 20
InitiatingHeapOccupancyPercent: 35
ExplicitGCInvokesConcurrent: true
UseStringDeduplication: true
I have not made any changes to cluster-ca-cert or clients-ca-cert automatically generated by strimzi.
Error:
2022-09-13 09:39:34,227 WARN [AdminClient clientId=adminclient-1] Metadata update failed due to authentication error
org.apache.kafka.clients.admin.internals.AdminMetadataManager) [kafka-admin-client-thread | adminclient-1]
org.apache.kafka.common.errors.SslAuthenticationExceptio
n: Failed to process post-handshake messages
Caused by: javax.net.ssl.SSLException: Tag mismatch!
please help on this issue.
Thanks, Shiva