Pulling from a private submodule github repository using a deploy key?

Question

I have a private github repository (Server) with another private repository (Shared) as a submodule. Since they're both set to private, and github doesn't allow sharing deploy keys - when I try to run submodule update I get the following error:

ERROR: Repository not found. fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.

It works fine if I'm using a github-wide SSH key on my desktop, but I obviously don't want my server to have access to all the repositories on my account - so I need to use deploy keys.

How can I update submodules using github deploy keys?

VonC_6309 · Answer

The actions/checkout issue 183 proposes a few options:

For instance (to be tested)

What do you think about being able to specify multiple ssh keys. For example:
ssh-key: |
 ${{ secrets.my_main_repo_deploy_key }}
 ${{ secrets.my_other_repo_deploy_key }}
I think if one doesnt work, it will fallback and try the next. To be clear, this currently won't work - would need to update the action to support it.

Or:

I already use a deploy key to pull in a python dependency from another private repository 'B' into the build of our repository 'A'.

For this to work, I set up a private key via a configured secret in 'A' and the respective public key in 'B' and use the following step:
     - name: Setup access via public/private key.
       # Below command requires the FOOBAR_PRIVATE_KEY to be configured via github repository secrets.
       # Also the key's public part must be added to the foobar repository deploy keys.
       # A private/public key pair without password (required in this case) can be generated with ssh-keygen.
       # This part is used for the git/foobar part in requirements.txt
       run: |
         mkdir ~/.ssh
         echo "${{ secrets.FOOBAR_PRIVATE_KEY }}" > ~/.ssh/id_rsa
         chmod 600 ~/.ssh/id_rsa

Also:

On GitHub the problem is, that we need one key for each submodule. AFAIK the idea in PR #190 will only allow one key for all submodules.
But we need multiple if we have multiple private submodules.

You have a full workaround in "Using private git submodules in GitHub CI" from Maximilian Ehlers.

The OP jon adds in the comments

I got it working: I wasn't putting the "-----BEGIN OPENSSH PRIVATE KEY-----" and the END in the secret; adding those fixed it.

Note: trying to replicate that on your workstation, using multipe SSH keys, will mean using ~/.ssh/config in order to reference those keys under different Host entries.

See "Enable Multiple SSH Key for GitHub on Windows 10" as an example.

Note: the issue 116 "private submodule checkout fails" now (July 2022) includes as an alternatives:

This solution works when you want to keep flexibility of URL repos and still use GitHub Actions with Deploy Keys to access private submodules:

  - name: Checkout
    uses: actions/checkout@v3

  - name: Clone Submodule
    run: |
        mkdir -p $HOME/.ssh
        echo '${{ secrets.SUBMODULE_REPO_DEPLOY_KEY }}' > $HOME/.ssh/ssh.key
        chmod 600 $HOME/.ssh/ssh.key
        export GIT_SSH_COMMAND="ssh -i $HOME/.ssh/ssh.key"
        git submodule set-url  git@github.com:.git
        git submodule update --init --recursive
        git submodule set-url  https://github.com/.git
        unset GIT_SSH_COMMAND

Pulling from a private submodule github repository using a deploy key?

1 Answers