how to find AKS external ip using terraform data block

user_16560737 0
Viewed 528

I am trying to get ingress EXTERNAL-IP in k8s. Is there any way to get the details from terraform data block. like using data "azurerm_kubernetes_cluster" or something?

azure-aks terraform kubernetes
3 Answers

The solution that pops into my head (which I'm sure is not the ideal one) is to use local-exec, and then use kubectl to interrogate the ingress resource in-cluster to get the IP.

Something like this (caveat: haven't tested it and I don't use AKS, so I don't know for sure if it'll work as expected)

resource "null_resource" "example1" {
  provisioner "local-exec" {
    command = "kubectl get ingress name-of-ingress-controller-lb | jq .status.loadBalancer.ingress[0].ip"
  }
}
user_20175900

you can create the Public IP in advance with terraform and assign this IP to your ingress service:

YAML:

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/azure-load-balancer-resource-group: myResourceGroup # only needed if the LB is in another RG
  name: ingress-nginx-controller
spec:
  loadBalancerIP: <YOUR_STATIC_IP>
  type: LoadBalancer

Same but Terraform code:

resource "kubernetes_service" "ingress_nginx" {
  metadata {
    name = "ingress-nginx-controller"
    
    annotations {
      "service.beta.kubernetes.io/azure-load-balancer-resource-group" = "${azurerm_resource_group.YOUR_RG.name}"
    }

  spec {
    selector = {
      app = <PLACEHOLDER>
    }
    port {
      port        = <PLACEHOLDER>
      target_port = <PLACEHOLDER>
    }

    type = "LoadBalancer"
    load_balancer_ip = "${azurerm_public_ip.YOUR_IP.ip_address}"
  }
}
user_167764510

Another solution would be to use the DNS provider's dns_a_record_set data source to resolve the FQDN at build time. Unlike the IP, the FQDN is exported from the azurerm_kubernetes_cluster resource.

That would allow you to work with the resulting IP address via the dns_a_record_set's addrs attribute, as shown below (an example where the IP was needed for the destination of a firewall rule):

# not shown:
# * declaring dns provider in required_providers block
# * supporting resources eg azurerm_resource_group, etc

resource "azurerm_kubernetes_cluster" "this" {
  ...
}

data "dns_a_record_set" "aks_api_ip" {
  host = azurerm_kubernetes_cluster.this.fqdn
}

resource "azurerm_firewall_network_rule_collection" "firewall_network_rule_collection" {
  name                = "ip_based_network_rules"
  azure_firewall_name = azurerm_firewall.this.name
  resource_group_name = azurerm_resource_group.this.name
  priority            = 200
  action              = "Allow"

  rule {
    name                  = "aks-nodes-to-control-plane"
    description           = "Azure Global required network rules: https://docs.microsoft.com/en-us/azure/aks/limit-egress-traffic"
    source_addresses      = azurerm_subnet.this.address_prefixes
    destination_ports     = [ "443", "1194", "9000" ]
    destination_addresses = data.dns_a_record_set.aks_api_ip.addrs
    protocols             = [
      "UDP",
      "TCP"
    ]
  }

  ...

}

The above worked in my case, and successfully added the correct IP to the rule destination. No depends_on needed, Terraform thankfully is able to suss out build order.

user_148847260
Related