How to refresh remote state from another module

Question

Looking for an advice and some general guidance with using terraform and in my case completely splitting states by services.

This is an example of directory structure how it's maintained.

global
├── iam
│   ├── .tf files
├── kms
│   ├── .tf files
└── organization
    └── root
        ├── .tf files
        ├── project_1_OU
        |   └── .tf files
        ├── project_2_OU
        |   └── .tf files
        └── project_3_OU
            └── .tf files

Every state is in it's service folder. For example: global/organization/root has the list of all accounts in the organization, controls the Resource Access Manager shares, SCPs on the global level, and etc. One of the outputs is the following one which returns the list of all accounts.

output "root_organization_accounts" {
  value = "${data.template_file.organization_accounts_list.*.rendered}"
}

KMS service on the other hand uses this output variable from the remote state to update it's policy with the list of new accounts

Scenario I'm trying:

create global/organization/root/project_x_OU
- this creates the wanted OU structure under Root with sub-accounts, SCPs and etc.
- accounts list is now different and needs to be refreshed (state in the level above)!
terraform apply/refresh is now needed manually to change the output of the global/organization/root (and nothing reported as to be changed which is normal in this case - it's only output change)
now, we should do the terraform apply in global/kms level so that the policy is updated with new accounts added

In short, global/organization/root state is used as read-only in global/organization/root/project_x_OU and global/kms levels.

A change in global/organization/root/project_x_OU should trigger state refresh in global/organization/root level so that global/kms level can be updated with new data.

My questions:

What would be the best way to trigger refresh on another remote state if we know that it will change and we need it to change so another module can be updated?
Am I doing something wrong and how?
Should I create a fake/unused variable in global/organization/root level and increment it with every new account just to force changes, is it even possible?
Any other advice and how it is done at your place?

I'm using Atlantis as a type of CI/CD solution in combination with plain terraform https://www.runatlantis.io/ and it doesn't like to perform changes if there aren't any (on global/organization/root level in this case)

How to refresh remote state from another module

0 Answers