terraform: local_file dependency with null_resource resulting in: no such file or directory

user_10122644 0
Viewed 1022

I have a null_resource executing a docker command which creates a lambda zip file (it is the only way, since i need some compiled libraries that I would normally get with apt-get or yum install) - hence not using other terraform provided modules for creating lambda files:

resource "null_resource" "install_dependencies" {
  provisioner "local-exec" {
    command = "docker run -w ${path.cwd}/lambda_folder/ foot.bar.docker >> ${path.cwd}/lambda_folder/lambda.zip"
  }
}

in order then to upload the lambda file with aws_lambda_function resource, i need to use local_file datasource:

data "local_file" "lambda_file" {
  filename = "${path.module}/lambda_folder/lambda.zip"
  depends_on = [
    null_resource.install_dependencies
  ]
}

uploading lambda resource:

resource "aws_lambda_function" "database_rotation" {
  vpc_config {
    subnet_ids         = data.aws_subnet_ids.subnet_ids.ids
    security_group_ids = [data.aws_security_group.rotation_sg.id]
  }
  filename         = data.local_file.lambda_file.content
  function_name    = "foo_bar_lambda"
  role             = aws_iam_role.iam_for_lambda.arn
  handler          = "lambda_function.lambda_handler"
  runtime          = "python3.8"
  timeout          = 120
  environment { variables = var.env_vars }

  depends_on = [null_resource.install_dependencies]
}

This all works fine on a first run - before i have any state of null_resource and data.local_file.lambda_file - but once my initial deployment was successful, terraform will fail with following error:

Error: open ./lambda_folder/lambda.zip: no such file or directory

  on main.tf line 62, in data "local_file" "lambda_file":
  62: data "local_file" "lambda_file" {

Which makes sense - null_resource did not run and did not create lambda.zip file. My question is, how to get around this ? How to make sure that either data.local_file.lambda_file is not triggered every time or if null_resource is executed every time i run terraform and thus creating/uploading the same lambda.zip file I am a bit lost and i would appreciate any help. Thank you!

aws-lambda terraform
1 Answers

This is a difficult one, unfortunately. You can use the triggers input for the null_resource, combined with the fileexists function and the uuid function, something like this:

resource "null_resource" "install_dependencies" {
  triggers = {
    filefound = fileexists("${path.cwd}/lambda_folder/lambda.zip") ? "1" : uuid()
  }
  provisioner "local-exec" {
    command = "docker run -w ${path.cwd}/lambda_folder/ foot.bar.docker >> ${path.cwd}/lambda_folder/lambda.zip"
  }
}

This guarantees that it will re-run whenever the file is missing, but the problem is that it will run twice each time (once when it discovers the file is missing, and again when the trigger changes from the UUID to "1" after the file is created. Subsequent runs, however, shouldn't re-create the resource.

EDIT: It was really bothering me that there's no way to do this, so I built a module that solves the issue. In your case, you would use it like so:

module "conditional_trigger" {
  source = "Invicton-Labs/conditional-trigger/null"
  // A new output ID will only be generated when this field is `true`
  regenerate = fileexists("${path.cwd}/lambda_folder/lambda.zip")
}

resource "null_resource" "install_dependencies" {
  triggers = {
    id = module.conditional_trigger.uuid
  }
  provisioner "local-exec" {
    command = "docker run -w ${path.cwd}/lambda_folder/ foot.bar.docker >> ${path.cwd}/lambda_folder/lambda.zip"
  }
}

It's some serious black magic, but it works for most cases.

user_6125800
Related