Use terraform module from another private repo with git submodule

Stan B 0
Viewed 74

I am trying to use module from another private github repository. This another github repository has git submodule from third repository.

module "ecr_repos" {
  for_each = local.ecr_repos 

  source = "git@github.com:gitaccount/repo2.git//terraform/modules/ecr"
  name = each.key
}

git submodule directory in the repo is not used by tf modules.

It works locally (with my ssh keys), but when I run terraform in github workflow it fails.

It is also reproduced locally in docker container.

In both cases I inject GITHUB_TOKEN env variable, that is a github private token (not the temporary token generated by github actions)

I run

git config --global url."https://oauth2:$GITHUB_TOKEN@github.com".insteadOf ssh://git@github.com
terraform init

Initializing modules...
Downloading git::ssh://git@github.com/gitaccount/repo2.git for ecr_repos...
╷
│ Error: Failed to download module
│ 
│ Could not download module "ecr_repos" (ecr.tf:1) source code from "git::ssh://git@github.com/gitaccount/repo2.git": error downloading 'ssh://git@github.com/gitaccount/repo2.git': /usr/bin/git exited with 1: Submodule 'third/repo/submodule/path'
│ (git@github.com:gitaccount/repo3.git) registered for path 'third/repo/submodule/path'
│ Cloning into '/root/repo1/terraform/infra/ecr/.terraform/modules/ecr_repos/third/repo/submodule/path'...
│ git@github.com: Permission denied (publickey).
│ fatal: Could not read from remote repository.
│ 
│ Please make sure you have the correct access rights
│ and the repository exists.
│ fatal: clone of 'git@github.com:gitaccount/repo3.git' into submodule path '/root/repo1/terraform/infra/ecr/.terraform/modules/ecr_repos/third/repo/submodule/path' failed
│ Failed to clone 'third/repo/submodule/path'. Retry scheduled
│ Cloning into '/root/repo1/terraform/infra/ecr/.terraform/modules/ecr_repos/third/repo/submodule/path'...
│ git@github.com: Permission denied (publickey).
│ fatal: Could not read from remote repository.

As a result the repo2 downloaded into .terraform directory, but it fails to download submodule from repo3

If I try to clone repo2 with GITHUB_TOKEN then clone operation succeeds and submodule exists.

What am I missing and what else should I configure to make it work? Adding ssh keys to ~/.ssh is a last resort and I really do not want to use it.

UPDATE:

I solved the issue

As I described in the beging of question, repo2 has .gitmodules file that contains definition of submodule with url = git@github.com:gitaccount/repo3.git

I added additioanal git config command before terraform init to match git submodule definition:

git config --add --global url."https://oauth2:$GITHUB_TOKEN@github.com/".insteadOf "git@github.com:"

The ending / in url and : in git@github.com: are critical

git-submodules git github terraform
1 Answers

The error Permission denied (publickey) means you're trying to authenticate using ssh public / private keys. It sounds like you want to use https instead as you're providing a GITHUB_TOKEN value.

For example git::https://example.com/vpc.git

I'm not too sure about your gitconfig command but personally I'd just change my links. If you're unhappy that the code is using https instead of your locally configured ssh, you'll need to add .ssh keys to your workflow as dynamic module sources aren't supported in Terraform. Perhaps a git config command may work too, but it seems like you've tried that and it didn't have the desired effect.

Leslie Alldridge0
Related