terraform to generate a file, zip and upload to s3

Question

I would like to generate a file dynamically, zip it and upload it to s3 using terraform. Below is what I have so far.

#Setup a local variable with docker container information
locals {
  file_content = jsonencode({
    "AWSEBDockerrunVersion" : "1",
    "Image" : {
      "Name" : "${var.aws_account_id}.dkr.ecr.${var.aws_region}.amazonaws.com/vin-${var.application}-api:${var.environment}",
      "Update" : "true"
    },
    "Ports" : {
      "ContainerPort" : 80,
      "HostPort" : 80
    }
  })
}

#Generate the Dockerrun.aws.json with the container information and environment tag
resource "local_file" "docker_container_info" {
  content  = local.file_content
  filename = "./${var.service_name}-${var.environment}-Dockerrun.aws.json"
}

#Zip the file
data "archive_file" "source" {
  type        = "zip"
  source_dir  = "./${var.service_name}-${var.environment}-Dockerrun.aws.json"
  output_path = "./${var.service_name}-${var.environment}-Dockerrun.aws.json.zip"

  depends_on = [
    local_file.docker_container_info
  ]
}

/*
#Upload the zip file to s3 bucket under DockerRunFiles folder
resource "aws_s3_bucket_object" "file_upload" {
  bucket           = var.run_file_bucket
  key              = "${var.bucket_name}/${var.service_name}-${var.environment}-Dockerrun.aws.json.zip"
  source           = "${data.archive_file.source.output_path}"
}
*/


variable "aws_region" {
  default = "us-east-1"
}

variable "environment" {
  default = "lab"
}

variable "aws_environment" {
  default = "lab"
}

variable "service_name" {
  default = "service_name"
}

variable "application" {
  default = "application_name"
}

variable "aws_account_id" {
  default = "1234567890"
}

variable "bucket_name" {
  default = "my_bucket"
}

I see the service_name-lab-Dockerrun.aws.json file getting generated correctly with json in it. But I am getting the error as Error: error archiving directory: could not archive directory that is a file: ./service_name-lab-Dockerrun.aws.json. Is the terraform data archive_file can only zip a folder and it cannot zip a json file? Appreciate the helps

Version details:

Terraform v0.12.28
+ provider.archive v2.0.0
+ provider.local v2.0.0

user_248823 · Answer

Your "./${var.service_name}-${var.environment}-Dockerrun.aws.json" is a file, not a directory. Thus, you should use source_file instead of source_dir as shown in the terraform docs.

#Zip the file
data "archive_file" "source" {
  type        = "zip"
  source_file  = "./${var.service_name}-${var.environment}-Dockerrun.aws.json"
  output_path = "./${var.service_name}-${var.environment}-Dockerrun.aws.json.zip"

  depends_on = [
    local_file.docker_container_info
  ]
}

user_4641153 · Answer

I'll note here that any sane person running terraform in a pipeline is going to have a plan phase with an approval and then passing the plan artifact to the apply stage after approval.

When you run a plan the data blocks will create the zip archive. When run apply without a plan the data blocks will create the zip archive. When you run a terraform apply plan where a plan aritfact has been passed to a new environment (like in a pipeline) it will fail as the plan expects the archive to be there.

You have to pass the plan and zip archive to the next stage as an artifact if you're separating your stage. Or release your code in a separate pipeline and have the lambda consume it.

terraform to generate a file, zip and upload to s3

2 Answers