Not able to create Zip file for AWS Lambda Fx in Gitlab through Terraform

user_17592420 0
Viewed 851

I am trying to create a lambda function through terraform for files present in gitlab repo however I am getting error in CICD pipeline:

"./lambda_function.zip: no such file or directory"

The folder (src folder) containing the lambda function python file is different than the folder (terraform) containing the terraform file.

My Gitlab Project looks like

ProjectName

-src

  • lambda_function.py

-terraform

  • lambda.tf

And the terraform code in lambda.tf is:

data "archive_file" "lambda" {
type = "zip"
source_file = "../src/lambda_function.py"
output_path = "lambda_function.zip"

}

    resource "aws_lambda_function" "automation-lambda" 
{filename=data.archive_file.lambda.output_path
  description       = "Creating lambda"
  function_name     = "lambda_fx"
  role              = "xxxxxxxxxxxxx"  
  handler           = "lambda_function.lambda_handler"
  memory_size       =  128
  timeout           =  300
  source_code_hash  = data.archive_file.lambda.output_base64sha256
  runtime = "python3.7"
}

Please suggest how the issue can be resolved.

Thanks

terraform-provider-aws aws-lambda gitlab-ci amazon-web-services
1 Answers

The problem in short:

The problem comes down to this: terraform plan creates data.archive_file resources, that are later used by terraform apply. If you do these two terraform commands in separate gitlab pipeline stages. The zip files generated in the plan stage, won't be available to the apply stage. Unless you add the output directory as an artifact

The longer version of the answer

If you are using multiple stages of the gitlab pipeline, one to plan, then one to apply. This is your problem.

The "dist" zip files are created during the plan stage, so you need to add the $PWD/dist as an artifact to your pipeline. Then in your apply stage, tell the pipeline it needs the plan stage, making the artifacts available to the apply command.

So in our pipeline, I had something like this:

plan_lambda:
  stage: plan
  needs:
    - init_lambda
    - validate_lambda
  script:
    - terraform -chdir=${PROJECT} plan -out=planfile ${args[@]}
  artifacts:
    paths:
      - ${PROJECT}/planfile
      # This is important, data.archive_file's are generated during plan stage, not apply, so these artifacts need to be stored
      # https://github.com/hashicorp/terraform-provider-archive/issues/39#issuecomment-1013680518
      - ${PWD}/dist

and then in the apply stage

apply_lambda:
  stage: apply
  when: manual
  needs:
    - init_lambda
    - plan_lambda
    - apply_execution_role
  variables:
    PROJECT: lambda
  script:
    - terraform -chdir=${PROJECT} apply -auto-approve planfile

and here, the generated dist zip files generated in the plan stage will be available to the apply stage and your problem should be fixed.

For extra information and the entire conversation that led me to this solution, here is the ticket in the terraform github repository:

https://github.com/hashicorp/terraform-provider-archive/issues/39#issuecomment-1013680518

What does "dist" or "dist zip" files mean?

Here is some terraform code which runs yarn to install node_modules and then packages up the source code into a zip file so you can deploy on AWS Lambda

resource "null_resource" "run_yarn" {
  triggers  =  {
    always_run = timestamp()
  }

  provisioner "local-exec" {
    command = "yarn --cwd ${local.root_path}/app/src install"
  }
}

data "archive_file" "app_src" {
  depends_on    = [null_resource.run_yarn]
  type          = "zip"
  source_dir    = "${local.root_path}/app/src"
  output_path   = "${local.root_path}/dist/app.zip"
}

So "dist" files, are the distributed zip files generated and uploaded to AWS Lambda

user_2791470
Related