Terraform Override Provider for use with Localstack

Question

I have a Git Repository with Terraform code that is being deployed into AWS. I am adding Localstack to this repository so that I can do higher-level validation testing before a plan and apply into my real AWS account. In order to use Localstack I have to create a new provider with the custom endpoint:

provider "aws" {
  alias  = "real"
  region = "${local.aws_region}"
}

provider "aws" {
  alias                       = "fake"
  region                      = "${local.aws_region}"
  access_key                  = "fake"
  secret_key                  = "fake"
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true

  endpoints {
    dynamodb = "http://localhost:4566"
    lambda   = "http://localhost:4566"
    kinesis  = "http://localhost:4566"
  }
}

How can I use one provider for Localstack and one provider with AWS without duplicating my code?

user_281848 · Answer

Unfortunately because the configuration structure is quite significantly different between these two cases it's not really possible to make this dynamically switchable without the resulting configuration looking quite complicated, but it is possible to use Terraform language expression operators and dynamic blocks to set all of the provider arguments conditionally, and thus have a single provider configuration with dynamic settings rather than two separate provider configurations.

The first thing to decide would be how you'll decide between the two possibilities. Since your localstack pseudo-infrastructure will unavoidably be distinct from the "real" infrastructure I expect you'll want to use a separate state for it, and so this could be a reasonable situation to use a separate workspace for development/testing, and I'll write this example assuming that the localstack configuration should be active whenever the workspace dev is selected. If that isn't what you want then hopefully this should still be enough to adapt to match your needs.

locals {
  use_localstack = (terraform.workspace == "dev")

  aws_settings = (
    local.use_localstack ?
    {
      region     = local.aws_region
      access_key = "fake"
      secret_key = "fake"

      skip_credentials_validation = true
      skip_metadata_api_check     = true
      skip_requesting_account_id  = true

      override_endpoint = "http://localhost:4566"
    } :
    {
      region     = local.aws_region
      access_key = null
      secret_key = null
      
      skip_credentials_validation = null
      skip_metadata_api_check     = null
      skip_requesting_account_id  = null

      override_endpoint = null
    }
  )
}

provider "aws" {
  region     = local.aws_settings.region
  access_key = local.aws_settings.access_key
  secret_key = local.aws_settings.secret_key

  skip_credentials_validation = local.aws_settings.skip_credentials_validation
  skip_metadata_api_check     = local.aws_settings.skip_metadata_api_check
  skip_requesting_account_id  = local.aws_settings.skip_requesting_account_id

  dynamic "endpoints" {
    for_each = local.aws_settings.override_endpoint[*]
    content {
      dynamodb = endpoints.value
      lambda   = endpoints.value
      kinesis  = endpoints.value
    }
  }
}

The above is relying on two particular behaviors of the Terraform language:

When setting arguments that will be passed to a provider, setting null is always the same as omitting that argument, because Terraform internally handles unset arguments by implicitly setting them to null anyway.
Using [*] with a non-list value automatically converts it into a zero-element list or a one-element list, depending on whether the value is null. That allows us to dynamically declare the endpoints block only if the override_endpoint attribute is non-null, and then write its value into all three of the overridden endpoint arguments.

Terraform Override Provider for use with Localstack

1 Answers