Getting "Failure sending request: StatusCode=429 -- Original Error: context deadline exceeded" on Terraform

user_14946366 0
Viewed 6361

Terraform is throwing an error "Failure sending request: StatusCode=429 -- Original Error: context deadline exceeded" when creating over 2000 A and CNAME records on Azure.

Tried Terraform v0.13.5 and v0.14, same error

Using version 2.39 of the azurerm plugin

An example of my terraform code:

provider "azurerm" {
#    version="2.39"
    features {}
}

resource "azurerm_resource_group" "example" {
  name     = "large-rg"
  location = "West US"
}

resource "azurerm_dns_zone" "example" {
  name                = "dnsrecords.com"
  resource_group_name = azurerm_resource_group.example.name
}

resource "azurerm_dns_a_record" "arecord" {
        name                = " arecord "
        zone_name           = " dnsrecords.com"
        resource_group_name = "large-rg"
        ttl                 = 300
        records             = ["1.1.1.1"]
}

resource "azurerm_dns_cname_record" "cname" {
        name                = "cname"
        zone_name           = " dnsrecords.com"
        resource_group_name = "large-rg"
        ttl                 = 300
        record              = "testing.trafficmanager.net."
}

Getting this in the logs

2020/12/09 17:11:22 [TRACE] eval: *terraform.EvalWriteState 
2020/12/09 17:11:22 [TRACE] EvalWriteState: recording 1 dependencies for azurerm_dns_zone.example 
2020/12/09 17:11:22 [TRACE] EvalWriteState: writing current state object for azurerm_dns_zone.example 
2020/12/09 17:11:22 [TRACE] [walkRefresh] Exiting eval tree: azurerm_dns_zone.example 
2020/12/09 17:11:22 [TRACE] vertex "azurerm_dns_zone.example": visit complete 
2020/12/09 17:11:22 [TRACE] vertex "azurerm_dns_zone.example": dynamic subgraph completed successfully 
2020/12/09 17:11:22 [TRACE] vertex "azurerm_dns_zone.example": visit complete 
2020/12/09 17:11:22 [TRACE] vertex "azurerm_dns_zone.example (expand)": dynamic subgraph completed successfully 
2020/12/09 17:11:22 [TRACE] vertex "azurerm_dns_zone.example (expand)": visit complete 
2020/12/09 17:11:22 [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/hashicorp/azurerm\"] (close)" errored, so skipping 
2020/12/09 17:11:22 [TRACE] dag/walk: upstream of "root" errored, so skipping 
2020/12/09 17:11:22 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info 
2020/12/09 17:11:22 [TRACE] statemgr.Filesystem: unlocked by closing terraform.tfstate 
2020-12-09T17:11:22.725-0800 [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/hashicorp/azurerm/2.39.0/windows_amd64/terraform-provider-azurerm_v2.39.0_x5.exe pid=18812 
2020-12-09T17:11:22.725-0800 [DEBUG] plugin: plugin exited

Edit: I split the records into batches of around 170 records and created its own terraform.tf and folder. I would run terraform init then import the exact same resource group and dns zone, in this case large-rg and dnsrecords.com. Next step was running terraform plan then terraform apply.

What would happen is that it would first throw the StatusCode=429 error, but rerunning it would solve the error. Adding a new record to it will also work.

Doing it this way isn't ideal just because of the size of the records. The amount of records will only get larger as time passes.

Looks like a limitation of Terraform.

Edit2: Nancy Xiong provided a link where the recommended solution is to use multiple smaller state files. https://github.com/terraform-providers/terraform-provider-azurerm/pull/6866

terraform-provider-azure terraform
1 Answers

If you create all your resources azurerm_dns_zone and azurerm_resource_group in the same .tf file. I suggest instead of hardcode the value, you can change your terraform template like this:

resource "azurerm_resource_group" "example" {
  name     = "large-rg"
  location = "West US"
}

resource "azurerm_dns_zone" "example" {
  name                = "dnsrecords.com"
  resource_group_name = azurerm_resource_group.example.name
}

resource "azurerm_dns_a_record" "arecord" {
        name                = "arecord"
        zone_name           = azurerm_dns_zone.example.name
        resource_group_name = azurerm_resource_group.example.name
        ttl                 = 300
        records             = ["1.1.1.1"]
}

resource "azurerm_dns_cname_record" "cname" {
        name                = "cname"
        zone_name           = azurerm_dns_zone.example.name
        resource_group_name = azurerm_resource_group.example.name
        ttl                 = 300
        record              = "testing.trafficmanager.net."
}

Also, the name value with having spaces is invalid. You can upgrade to

Terraform v0.14.3
+ provider registry.terraform.io/hashicorp/azurerm v2.41.0
user_98333140
Related