Terraformer merging multiple tfstate files

Question

I'm importing with terraformer (not terraform, check this https://github.com/GoogleCloudPlatform/terraformer) some infrastructure hosted on Google Cloud, please see my command below.

i.e: terraformer import google --resources=networks,subnetworks,firewall,routes,forwardingRules,vpnTunnels --regions=europe-blabla1 --projects=my_project

It works okay, the output of the command is a separate folder with tfstate and tf files per every resource I imported.

2021/05/18 11:03:20 google Connecting....
2021/05/18 11:03:20 google save firewall
2021/05/18 11:03:20 google save tfstate for firewall
2021/05/18 11:03:20 google save routes
2021/05/18 11:03:20 google save tfstate for routes
2021/05/18 11:03:20 google save forwardingRules
2021/05/18 11:03:20 google save tfstate for forwardingRules
2021/05/18 11:03:20 google save vpnTunnels
2021/05/18 11:03:20 google save tfstate for vpnTunnels
2021/05/18 11:03:20 google save networks
2021/05/18 11:03:20 google save tfstate for networks
2021/05/18 11:03:20 google save subnetworks
2021/05/18 11:03:20 google save tfstate for subnetworks

Now what I want to accomplish is to merge those multiples tfstate files into a single one, in order to have stacks for let's say: one for networking, and the same for iam, instances and so for.

I wonder if someone has been able to do this?, or If there is a better approach to accomplish this?.

user_5532207 · Answer

I've been there and use Terraformer then merge state files, here is a script I wrote to do state file migration (merging):

#!/usr/bin/env bash

state_pull() {
    echo ">> state pull"
    cd "$1" || return 1
    terraform init
    terraform state pull >terraform.tfstate
    cd ..
    return 0
}

state_mv() {
    echo ">> state mv"
    cd "$2" || return 1
    terraform init
    for i in $(terraform state list); do
        terraform state mv -state-out="../$1/terraform.tfstate" "$i" "$i"
    done
    cd ..
    return 0
}

state_push() {
    echo ">> state push"
    cd "$1" || return 1
    terraform state push terraform.tfstate
    cd ..
    return 0
}

cleanup() {
    echo ">> cleanup"
    cd "$1" || return 1
    rm terraform.tfstate
    rm terraform.tfstate.*
    cd ..
    return 0
}

# Print help text and exit.
if [ "$1" = "-h" ] || [ "$1" = "--help" ]; then
    echo "Usage: tfmv.sh [options]"
    echo
    echo "Examples:"
    echo
    echo "  - ./tfmv.sh stack1-migrate-to stack2-migrate-from"
    echo "  - ./tfmv.sh app data"
    exit 1
fi

echo "> START"
echo ">> TF =  $(terraform version)..."
state_pull "$1"
state_mv "$1" "$2"
state_push "$1"
# cleanup "$1"
echo "> FINISHED"

If this looks so fancy, you can use https://github.com/minamijoyo/tfmigrate which do a better job using Go.

user_13676217 · Answer

I don't know anything about terraformer. But I don't think there's a native Terraform way to accomplish this easily.

When I am faced with a similar scenario (i.e., merging two terraform.tfstate files into one -- and merging the corresponding terraform source code to match it), I had to manually do terraform import ... for each resource. It is tedious but I fortunately had only a handful of resources to merge.

As an aside (just to preempt some questions), the command terraform state mv ... won't do. It is only for renaming a resource's address inside the same state file rather than moving across 2 state files. I wish that command were better named to something like terraform state rename-resource-address ....

Terraformer merging multiple tfstate files

2 Answers