How to manage multiple kubernetes namespaces in a terraform project

jdkealy 0
Viewed 28

I've been using terraform for a while now and I have deployed everything into separate clusters. Now due to cost, we'd like to merge the different clusters into one cluster and use kubernetes namespaces.

My desired outcome would be that I could call terraform apply - var="kubernetes_namespace=my-namespace" and it would create the namespace which could live alongside my other namespaces, however, due to how terraform remote state is managed, any new deployment will overwrite the old and I can't have co-existing namespaces.

When I try to redeploy another namespace I get namespace = "deploy-pr-image" -> "test-second-branch-pr" # forces replacement

I can see why because it's writing everything to a single workspace file.

terraform {
  backend "s3" {
    bucket               = "my-tf-state"
    key                  = "terraform-services.tfstate"
    region               = "us-west-1"
    workspace_key_prefix = "workspaces"
    #dynamodb_table       = "terraform-state-lock-dynamo"
  }
}

Is there some way to use the workspace/namespace combination to keep terraform from overwriting my other namespace ?

kubernetes terraform
1 Answers

Since now you'll be merging all your clusters into a single one, it would make sense to only have a backend where you can manage the state of the cluster, therefore rather than having multiple backends per Kubernetes namespaces.

I'd suggest you update your module or root deployment to be flexible enough that it can create X number of Kubernetes namespaces resources rather than a single one using count or for_each.

javierlga0
Related