How connect to MSK cluster from EKS cluster

Viewed 616

I am having difficulties connecting to my MSK cluster from my EKS cluster even though both clusters share the same VPC and the same subnets.

The security group used by the MSK cluster has the following inbound rules

type protocol port range source
all traffic all all custom SG_ID
all traffic all all anywhere ipv4 0.0.0.0/0

Where SG_ID is the EKS' Cluster security group. The one labeled: EKS created security group applied...

In the EKS cluster, I am using the following commands to test connectivity:

kubectl  run kafka-consumer \
-ti \
--image=quay.io/strimzi/kafka:latest-kafka-2.8.1 \
--rm=true \
--restart=Never \
-- bin/kafka-topics.sh --create --topic test  --bootstrap-server b-1.test.z35y0w.c4.kafka.us-east-1.amazonaws.com:9092 --replication-factor 2 --partitions 1 --if-not-exists 

With the following result

Error while executing topic command : Call(callName=createTopics, deadlineMs=1635906680860, tries=1, nextAllowedTryMs=1635906680961) timed out at 1635906680861 after 1 attempt(s) [2021-11-03 02:31:20,865] ERROR org.apache.kafka.common.errors.TimeoutException: Call(callName=createTopics, deadlineMs=1635906680860, tries=1, nextAllowedTryMs=1635906680961) timed out at 1635906680861 after 1 attempt(s) Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: createTopics (kafka.admin.TopicCommand$) pod "kafka-consumer" deleted pod default/kafka-consumer terminated (Error)

Sadly, the second bootstrap server displayed on the MSK Page gives the same result.

nc eventually times out

kubectl  run busybox -ti --image=busybox --rm=true --restart=Never -- nc b-2.test.z35y0w.c4.kafka.us-east-1.amazonaws.com

nslookup fails as well

kubectl  run busybox -ti --image=busybox --rm=true --restart=Never -- nslookup b-2.test.z35y0w.c4.kafka.us-east-1.amazonaws.com
If you don't see a command prompt, try pressing enter.
*** Can't find b-2.test.z35y0w.c4.kafka.us-east-1.amazonaws.com: No answer

Could anyone please give me a hint? Thanks

0 Answers
Related