In Terraform for AWS, can I do a retry on a resource creation?

Viewed 4302

Terraform 0.12.x

I'm attaching an EBS volume to my EC2

resource "aws_volume_attachment" "dev_xvdf" {
  device_name  = "/dev/xvdf"
  instance_id  = aws_instance.jenkins_master.id
  skip_destroy = var.skip_att_ebs_destroy
  volume_id    = var.ebs_id
}

But I get

module.ec2_blue.aws_volume_attachment.dev_xvdf: Still creating... [10s elapsed]
module.ec2_blue.aws_volume_attachment.dev_xvdf: Still creating... [20s elapsed]

Error: Error waiting for Volume (vol-123456abcedf) to attach to Instance: i-123456abcdef, error: unexpected state 'detached', wanted target 'attached'. last error: %!s(<nil>)

Ca I do a retry, or increase a timeout?

1 Answers

Retries and timeouts in Terraform are handled (or not) by the provider as an implementation detail, and so you can't control them directly in the Terraform configuration unless the provider developer exposed a way to do so.

Some resource types follow a convention of accepting a timeouts block to optionally shorten or extend the timeout on a particular operation:

 timeouts {
   create = "1m"
 }

The aws_volume_attachment doesn't seem to support this, instead hard-coding a five-minute timeout.

However, we can see in your case that the problem doesn't seem to actually be the timeout, because the operation failed after only 20 seconds and it seems like the problem was that it entered an unexpected status rather than that it took to long to reach the expected status.

I'm not super familiar with the state machine for volume attachments, but from reading the AWS provider code it seems like the provider expects the attachment to initially be in the "attaching" state and then eventually reach the "attached" state. It failed here because it actually entered the "detached" state, which is neither the state the provider considers to mean "pending" nor the one meaning "succeeded".

With all of that said, I think this is a real error rather than a timeout. I don't know what is causing the error, but you may be able to find out by attempting to perform the same operation manually using either the web console or the AWS CLI. If you're able to gather more information doing that, I expect someone will be able to give a more helpful answer to a new question including that additional information.

Related