Terraform aws_s3_bucket_replication_configuration can't generate multiple rules with for_each

user_6629280 0
Viewed 462

I have an S3 bucket with the following "folder" structure:

Bucket1----> /Partner1 ----> /Client1 ----> /User1
        |               |              |--> /User2
        |               |
        |               |--> /Client2 ----> /User1
        |
        |--> /Partner2 ----> /Client1 ----> /User1

and so on.

I'm trying to setup replication from this bucket to another such that a file placed in

Bucket1/Partner1/client1/User1/ should replicate to Bucket2/Partner1/client1/User1/,

Bucket1/Partner2/client1/User2/ should replicate to Bucket2/Partner2/client1/User2/,

and so on.

I'm trying to achieve this with the following terraform code:

locals {
 s3_folders = [
  "Partner1/client1/User1",
  "Partner1/client1/User2",
  "Partner1/client1/User3",
  "Partner1/client1/User4",
  "Partner1/client1/User5",
  "Partner1/client2/User1",
  "Partner1/client3/User1",
  "Partner2/client1/User1",
  "Partner3/client1/User1"
 ]
}

resource "aws_s3_bucket_replication_configuration" "replication" {
  for_each      = local.s3_input_folders

  depends_on    = [aws_s3_bucket_versioning.source_bucket]
  role          = aws_iam_role.s3-replication-prod[0].arn
  bucket        = aws_s3_bucket.source_bucket.id

  rule {
    id = each.value

    filter {
      prefix =  each.value
    }

    status = "Enabled"

    destination {
      bucket        = "arn:aws:s3:::${var.app}-dev"
      storage_class = "ONEZONE_IA"

      access_control_translation {
        owner       = "Destination"
      }
      account = var.dev_account_id
    }

    delete_marker_replication {
        status = "Enabled"
    }
  }
}

This is not looping and creating 10 different rules, rather it overwrites the same rule on every run and I only get one rule as a result.

terraform-provider-aws terraform amazon-web-services amazon-s3
2 Answers

You should use dynamic block:

resource "aws_s3_bucket_replication_configuration" "replication" {
  
  depends_on    = [aws_s3_bucket_versioning.source_bucket]
  role          = aws_iam_role.s3-replication-prod[0].arn
  bucket        = aws_s3_bucket.source_bucket.id

  dynamic "rule" {
    
    for_each      = toset(local.s3_input_folders)
    
    content {
    
        id = rule.value

        filter {
            prefix =  rule.value
        }

        status = "Enabled"

        destination {
        bucket        = "arn:aws:s3:::${var.app}-dev"
        storage_class = "ONEZONE_IA"

        access_control_translation {
            owner       = "Destination"
        }
        account = var.dev_account_id
        }

        delete_marker_replication {
            status = "Enabled"
        }
    }    
  }
}
user_2488230

Thanks, Marcin. The dynamic block construct you mentioned works to create the content blocks but it fails to apply because AWS needs multiple replication rules to be differentiated by priority. So some slight modifications achieve this:


locals {
  s3_input_folders_list_counter = tolist([
    for i in range(length(local.s3_input_folders)) : i
  ])
  s3_input_folders_count_map = zipmap(local.s3_input_folders_list_counter, tolist(local.s3_input_folders))
}

resource "aws_s3_bucket_replication_configuration" "replication" {
  depends_on    = [aws_s3_bucket_versioning.source_bucket]
  role          = aws_iam_role.s3-replication-prod[0].arn
  bucket        = aws_s3_bucket.source_bucket.id

   dynamic "rule" {
    
    for_each      = local.s3_input_folders_count_map
    
    content {
    
        id = rule.key
        priority = rule.key

        filter {
            prefix =  rule.value
        }

        status = "Enabled"

        destination {
        bucket        = "arn:aws:s3:::${var.app}-dev"
        storage_class = "ONEZONE_IA"

        access_control_translation {
            owner       = "Destination"
        }
        account = var.dev_account_id
        }

        delete_marker_replication {
            status = "Enabled"
        }
    }    
  }
}

which creates rules like these:

+ rule {
          + id       = "0"
          + priority = 0
          + status   = "Enabled"

...
}
+ rule {
          + id       = "1"
          + priority = 1
          + status   = "Enabled"

...
}

and so on...

user_66292800
Related