terraform eks node labels

Viewed 595

I’m creating a eks k8s cluster in AWS, all is fine. I’m using worker_groups like so:

module "eks" {
  source          = "terraform-aws-modules/eks/aws"
  cluster_name    = var.cluster_name
  cluster_version = "1.17"
  subnets         = var.private_subnets_ids

  tags = {
    Environment = var.environment
  }

  vpc_id = var.vpc_id

  worker_groups = [
    {
      name                          = "backend"
      instance_type                 = var.eks_workers.backend.instance_type
      asg_desired_capacity          = var.eks_workers.backend.desired_capacity
      key_name                      = var.eks_workers.backend.ssh_key_name
      subnets                       = var.private_subnets_ids
      additional_userdata           = "echo foo bar"
      associate_public_ip_address   = false
      additional_security_group_ids = [
        aws_security_group.sg_ssh_mgmt.id
      ]
      labels = {
        application = "backend"
      }
    }
  ]
}

The ASG is created, but the nodes do not have the label created. I need to label the nodes so that i can then use nodeSelector

Appreciate your help!

David

2 Answers

There doesn't seem to be a way of labeling nodes via terraform, so I ended up using additional_userdata.

  • We need to install kubectl
  • We need to configure the cluster for aws
  • Then we need to wait until the node has added itself to the kuberneter cluster.
  • Then we can add the label using the hostname, which is how they show up in the cluster.
      additional_userdata           = <<EOT
curl -o /root/kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.17.12/2020-11-02/bin/linux/amd64/kubectl
chmod +x /root/kubectl
aws eks --region us-east-1 update-kubeconfig --name dev-cluster
until [ "$(/root/kubectl --kubeconfig=/root/.kube/config get nodes | grep $(hostname) | wc -l)" == "1" ]; do sleep 1; done
/root/kubectl --kubeconfig=/root/.kube/config label nodes $(hostname) application=backend
EOT

And now I can deploy to specific node groups via nodeSelector

Related