I have a Kubernetes EKS cluster on AWS, an my goal is to be able to watch particular config maps in my Spring Boot application. On my local environment everything works correctly, but when I use this setup inside AWS I get forbidden state and my application fails to run. I've created a Service Account but don't understand how to create Terraform script which can assign the needed IAM Role. Any help would be appreciated.
3 Answers
This depends on several things.
An AWS IAM Role can be provided to Pods in different ways, but the recommended way now is to use IAM Roles for Service Accounts, IRSA.
Depending on how you provision the Kubernetes cluster with Terraform, this is also done in different ways. If you use AWS EKS and provision the cluster using the Terraform AWS EKS module, then you should set enable_irsa to true.
You then need to create an IAM Role for you application (Pods), and you need to return the ARN for the IAM Role. This can be done using the aws_iam_role resource.
You need to create a Kubernetes ServiceAccount for your pod, it can be created with Terraform, but many want to use Yaml for Kubernetes resources. The ServiceAccount need to be annotated with the IAM Role ARN, like:
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::14xxx84:role/my-iam-role
See the EKS workshop for IAM Roles for Service Accounts lesson for a guide through this. However, it does not use Terraform.
First I created the necessary role using below code:
data "aws_iam_policy_document" "eks_pods" {
statement {
actions = ["sts:AssumeRoleWithWebIdentity"]
effect = "Allow"
condition {
test = "StringEquals"
variable = "${replace(aws_iam_openid_connect_provider.eks.url, "https://", "")}:sub"
values = ["system:serviceaccount:kube-system:aws-node"]
}
principals {
identifiers = [aws_iam_openid_connect_provider.eks.arn]
type = "Federated"
}
}
}
# create a role that can be attached to pods.
resource "aws_iam_role" "eks_pods" {
assume_role_policy = data.aws_iam_policy_document.eks_pods.json
name = "eks-pods-iam-role01"
depends_on = [aws_iam_openid_connect_provider.eks]
}
resource "aws_iam_role_policy_attachment" "aws_pods" {
role = aws_iam_role.eks_pods.name
policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
depends_on = [aws_iam_role.eks_pods]
}
Then I used the below command to attach the role created to the service account. I have not found any way to do it from within terraform:
kubectl annotate serviceaccount -n kube-system aws-node eks.amazonaws.com/role-arn=arn:aws:iam::<your_account>:role/eks-pods-iam-role01
Then you can verify your service account, it should show the new annotations.
kubectl describe sa aws-node -n kube-system
Name: aws-node
Namespace: kube-system
Labels: <none>
Annotations: eks.amazonaws.com/role-arn: arn:aws:iam::<ur_account>:role/eks-pods-iam-role01
Image pull secrets: <none>
Mountable secrets: aws-node-token-xxxxx
Tokens: aws-node-token-xxxxx
Events: <none>
I think you missed something here..you should add trust relationship between the role and the oidc provider as described here:
Related