How would I be able to generate route 53 aliases for MSK bootstrap servers?

Archimedes Trajano 0
Viewed 22

I can create a single bootstrap server alias for Kafka as follows.

resource "aws_msk_cluster" "kafka" {
  count        = var.kafka_number_of_broker_nodes > 0 ? 1 : 0
  ...
}

# This generates a single bootstrap DNS entry which are used by the Kafka client for simplicity
resource "aws_route53_record" "kafka" {
  count   = var.kafka_number_of_broker_nodes > 0 ? 1 : 0
  zone_id = aws_route53_zone.internal.zone_id
  name    = "corekafka"
  type    = "CNAME"
  ttl     = "300"
  records = [split(":", sort(split(",", aws_msk_cluster.kafka[0].bootstrap_brokers))[0])[0]]
}

However, for fault tolerance, I would like to be able to specify all the bootstrap broker DNS names. I had the following code but if the kafka gets busted, usually on a test environment and the disk becomes full, if I won't be able to simply taint the kafka and need to manually remove the DNS entries first using the console or terraform -destroy

# This generates bootstrap DNS entries which are used by the Kafka client for better load balancing
resource "aws_route53_record" "kafka-bootstrap" {
  zone_id  = aws_route53_zone.internal.zone_id
  for_each = { for i, s in sort(split(",", aws_msk_cluster.kafka[0].bootstrap_brokers)) : s => [split(":", s)[0], i + 1] }
  name     = "b${each.value[1]}.corekafka"
  type     = "CNAME"
  ttl      = "300"
  records  = [each.value[0]]
}

Is there anything I can change on the block above to make it handle taints of the kafka server?

amazon-route53 apache-kafka terraform
1 Answers

This removes the for_each and uses the broker node count. So it avoids the dependency.

resource "aws_route53_record" "kafka-bootstrap" {
  zone_id = aws_route53_zone.internal.zone_id
  count   = var.kafka_number_of_broker_nodes
  name    = "b${count.index + 1}.corekafka"
  type    = "CNAME"
  ttl     = "300"
  records = [split(":", sort(split(",", aws_msk_cluster.kafka[0].bootstrap_brokers))[count.index])[0]]
}
Archimedes Trajano0
Related