I added a json file that contains all the tables i want to create :
tables.json
"tables": {
"table1": {
"dataset_id": "dataset1",
"table_id": "table1",
"schema_path": "folder/table1.json"
},
"table2": {
"dataset_id": "dataset2",
"table_id": "table2",
"schema_path": "folder/table2.json"
}
}
Then with a foreach on a Terraform resource, i want to create these tables dynamically :
local.tf file
locals {
tables = jsondecode(file("${path.module}/resource/tables.json"))["tables"]
}
variables.tf file
variable "project_id" {
description = "Project ID, used to enforce providing a project id."
type = string
}
variable "time_partitioning" {
description = "Configures time-based partitioning for this table."
type = map(string)
default = {
type = "DAY"
field = "my_field"
}
}
main.tf file
resource "google_bigquery_table" "tables" {
for_each = local.tables
project = var.project_id
dataset_id = each.value["dataset_id"]
table_id = each.value["table_id"]
dynamic "time_partitioning" {
for_each = [
var.time_partitioning
]
content {
type = try(each.value["partition_type"], time_partitioning.value["type"])
field = try(each.value["partition_field"], time_partitioning.value["field"])
expiration_ms = try(time_partitioning.value["expiration_ms"], null)
require_partition_filter = try(time_partitioning.value["require_partition_filter"], null)
}
}
schema = file("${path.module}/resource/schema/${each.value["schema_path"]}")
}
The schema files contains classic bigquery schema, for example :
[
{
"name": "field",
"type": "STRING",
"mode": "NULLABLE",
"description": "My field"
}
]
The creation of tables works well, but when i add a new nullable field on a schema, Terraform proposes to "replace table" (destroy and recreate) instead of "update table".
The normal behaviour in this case for native Bigquery and Terraform is to update the table.
When a do the same test with the same Terraform resource but without forEach, Terraform has the expected behaviour and proposes to update the table.
An example of the Terraform log with "forEach" :
# google_bigquery_table.tables["table1"] must be replaced
-/+ resource "google_bigquery_table" "tables" {
~ creation_time = 1616764894477 -> (known after apply)
dataset_id = "dataset1"
deletion_protection = true
~ etag = "G9qwId8jgQS8nN4N61zqcA==" -> (known after apply)
~ expiration_time = 0 -> (known after apply)
~ id = "projects/my-project/datasets/dataset1/tables/table1" -> (known after apply)
- labels = {} -> null
~ last_modified_time = 1617075251337 -> (known after apply)
~ location = "EU" -> (known after apply)
~ num_bytes = 0 -> (known after apply)
~ num_long_term_bytes = 0 -> (known after apply)
~ num_rows = 0 -> (known after apply)
project = "project"
~ schema = jsonencode(
~ [ # forces replacement
{
description = "Field"
mode = "NULLABLE"
name = "field"
type = "STRING"
}
.....
+ {
+ description = "Field"
+ mode = "NULLABLE"
+ name = "newField"
+ type = "STRING"
}
Terraform displays and detects correctly the new column to add for a table, but indicates a replace instead of an edition.
I repeat that, the exact same test with a the same Terraform resource without forEach and on a single Bigquery table, works well (same schema, same change). I create the table and when a add a new nullable column, Terraform proposes an edition (the expected behaviour).
I checked on Terraform doc and web, i didn't saw some examples to manage a list of table with Terraform.
Is it not possible to create and update tables with configured tables and foreach ?
Thanks for your help.