Build AWS IAM policy from JSON array of account numbers

You can load the file using the file function and then extract the list of accounts that you'll need by decoding the JSON with jsondecode and then selecting the accounts key.

As an example:

locals {
  accounts = jsondecode(file("accounts.json")).accounts
}

output example {
  value = local.accounts
}

This will return the following:

example = [
  "1234567890",
  "0987654321",
  "1029384756",
  "6574839201",
  "0192837465",
]

If you then wanted to put that into your policy you'd want to use the formatlist function to pass a list of accounts into a single string format:

output "example" {
  value = formatlist("arn:aws:s3:::bucket_name/AWSLogs/%s/*", local.accounts)
}

This outputs:

example = [
  "arn:aws:s3:::bucket_name/AWSLogs/1234567890/*",
  "arn:aws:s3:::bucket_name/AWSLogs/0987654321/*",
  "arn:aws:s3:::bucket_name/AWSLogs/1029384756/*",
  "arn:aws:s3:::bucket_name/AWSLogs/6574839201/*",
  "arn:aws:s3:::bucket_name/AWSLogs/0192837465/*",
]

If you notice carefully though, Terraform uses trailing commas in lists which is invalid JSON so would create an invalid JSON structure for your IAM policy. To fix that we can then re-encode it into JSON with jsonencode:

output "example" {
  value = jsonencode(formatlist("arn:aws:s3:::bucket_name/AWSLogs/%s/*", local.accounts))
}

Which then outputs:

example = ["arn:aws:s3:::bucket_name/AWSLogs/1234567890/*","arn:aws:s3:::bucket_name/AWSLogs/0987654321/*","arn:aws:s3:::bucket_name/AWSLogs/1029384756/*","arn:aws:s3:::bucket_name/AWSLogs/6574839201/*","arn:aws:s3:::bucket_name/AWSLogs/0192837465/*"]

without the trailing comma.

Putting this altogether you can then create your IAM policy like this:

locals {
  accounts = jsondecode(file("accounts.json")).accounts
}

resource aws_iam_policy policy {
  name        = "example"
  path        = "/"

  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:PutObject"
      ],
      "Effect": "Allow",
      "Resource": ${jsonencode(formatlist("arn:aws:s3:::bucket_name/AWSLogs/%s/*", local.accounts))}
    }
  ]
}
EOF
}

I've managed to get this working with help from ydaetskcoR:

Here is my variables file:

locals {
  accounts = jsondecode(file("../configuration/envs.json")).accounts
  config = jsonencode(formatlist("arn:aws:s3:::${var.config_bucket_name}/AWSLogs/%s/*", local.accounts))
}

I could test the output by using Terraform console:

> jsonencode(formatlist("arn:aws:s3:::${var.config_bucket_name}/AWSLogs/%s/*", local.accounts))
["arn:aws:s3:::bby-central-configlogs-splunk/AWSLogs/123456789/*","arn:aws:s3:::bby-central-configlogs-splunk/AWSLogs/123456789/*"]

As you can see it has [] around them: I put the following into my policy:

            "Principal": {
                "Service": "config.amazonaws.com"
            },
            "Action": "s3:PutObject",
            "Resource": ${local.config},
            "Condition": {
                "StringEquals": {
                    "s3:x-amz-acl": "bucket-owner-full-control"
                }

I thought I would put my answer :)

Getting JSON encoding right with string templates can be frustrating. Unless there's a strong reason to format the JSON a particular way, we can avoid templating JSON by constructing the desired data structure directly and then passing it to jsonencode, like this:

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid    = "AWSConfigAclCheck20150319"
        Effect = "Allow"
        Principal = {
          Service = "config.amazonaws.com"
        }
        Action   = "s3:GetBucketAcl"
        Resource = "arn:aws:s3:::${var.config_bucket_name}"
      },
      # etc, etc
    ]
  })

You can then use Terraform's operators and functions directly to construct parts of that object. For example, to produce the list of ARNs you wanted:

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      # ...
      {
        Sid    = "AWSConfigWrite20150319"
        Effect = "Allow"
        Principal = {
          Service = "config.amazonaws.com"
        }
        Action   = "s3:PutObject"
        Resource = [for acct in local.accounts : "arn:aws:s3:::${var.config_bucket_name}/AWSLogs/${acct}/*"]
      },
      # ...
    ]
  })

If the policy becomes complicated enough that you want to factor it out into a separate template file, you can still use the jsonencode function by making the entire template be a jsonencode call:

${jsonencode({
  # ...
})}

Using jsonencode for the whole structure at once means the result is guaranteed to be valid JSON, without needing to worry about exactly where to place commas and other delimiters as you would need to when constructing JSON as a template.

You could have a dynamic config reload in your code that updates the list of accounts and calls S3 API when the config file changes.

For the purpose of watching on config file (account list) you can use some libraries. I believe "Viper" is a good choice.

And also if you are worried about losing these kind of events on file caused by API call failures or application crash, you can try to make updating the action list idempotent and do it on application start or in an interval.

These links might come in hand:

About idempotency: https://nordicapis.com/understanding-idempotency-and-safety-in-api-design/

Viper: https://github.com/spf13/viper