Pull latest changes for all git submodules

Viewed 1478291

We're using git submodules to manage a couple of large projects that have dependencies on many other libraries we've developed. Each library is a separate repo brought into the dependent project as a submodule. During development, we often want to just go grab the latest version of every dependent submodule.

How do I pull the latest changes for all git submodules?

21 Answers

If it's the first time you check-out a repo you need to use --init first:

git submodule update --init --recursive

For git 1.8.2 or above, the option --remote was added to support updating to latest tips of remote branches:

git submodule update --recursive --remote

This has the added benefit of respecting any "non default" branches specified in the .gitmodules or .git/config files (if you happen to have any, default is origin/master, in which case some of the other answers here would work as well).

For git 1.7.3 or above you can use (but the below gotchas around what update does still apply):

git submodule update --recursive

or:

git pull --recurse-submodules

if you want to pull your submodules to latest commits instead of the current commit the repo points to.

See git-submodule(1) for details

Note: This is from 2009 and may have been good then but there are better options now.

We use this. It's called git-pup:

#!/bin/bash
# Exists to fully update the git repo that you are sitting in...

git pull && git submodule init && git submodule update && git submodule status

Just put it in a suitable bin directory (/usr/local/bin). If on Windows, you may need to modify the syntax to get it to work :)

Update:

In response to the comment by the original author about pulling in all of the HEADs of all of the submodules -- that is a good question.

I am pretty sure that git does not have a command for this internally. In order to do so, you would need to identify what HEAD really is for a submodule. That could be as simple as saying master is the most up to date branch, etc...

Following this, create a simple script that does the following:

  1. check git submodule status for "modified" repositories. The first character of the output lines indicates this. If a sub-repo is modified, you may NOT want to proceed.
  2. for each repo listed, cd into it's directory and run git checkout master && git pull. Check for errors.
  3. At the end, I suggest you print a display to the user to indicate the current status of the submodules -- perhaps prompt them to add all and commit?

I'd like to mention that this style is not really what git submodules were designed for. Typically, you want to say "LibraryX" is at version "2.32" and will stay that way until I tell it to "upgrade".

That is, in a sense, what you are doing with the described script, but just more automatically. Care is required!

Update 2:

If you are on a windows platform, you may want to look at using Python to implement the script as it is very capable in these areas. If you are on unix/linux, then I suggest just a bash script.

Need any clarifications? Just post a comment.

Henrik is on the right track. The git submodule foreach command can execute any arbitrary shell script. Two options to pull the very latest might be:

git submodule foreach git pull origin master

and:

git submodule foreach /path/to/some/cool/script.sh

That will iterate through all initialized submodules and run the given commands.

As it may happens that the default branch of your submodules is not master, this is how I automate the full Git submodules upgrades:

git submodule init
git submodule update
git submodule foreach 'git fetch origin; git checkout $(git rev-parse --abbrev-ref HEAD); git reset --hard origin/$(git rev-parse --abbrev-ref HEAD); git submodule update --recursive; git clean -dfx'

For me, git 2.24.03, get updated to latest commit of remote branches defined in .gitmodules.

git submodule update --recursive --init

git submodule update --recursive --remote

git version 2.24.3 (Apple Git-128)

Please Note: Someone said that git pull --recurse-submodulesis the same as git submodule update --recursive --remote. But from my test, git pull --recurse-submodules may not get updated to latest commit of remote branches defined in .gitmodules.

The above answers are good, however we were using git-hooks to make this easier but it turns out that in git 2.14, you can set git config submodule.recurse to true to enable submodules to to updated when you pull to your git repository.

This will have the side effect of pushing all submodules change you have if they are on branches however, but if you have need of that behaviour already this could do the job.

Can be done by using:

git config submodule.recurse true

To clarify a few things based on already available answers of pulling "latest" code of each submodule from remote.

If "latest" means the submodule pointers that were checked in, then by all means use:

git submodule update --recursive
  - or -
git pull --recurse-submodules --jobs=X

If "latest" means the latest of main, then something like this can work:

git submodule foreach "git checkout main && git pull"

Unfortunately, this means there's no "--jobs" option, so we cannot run it in parallel. The closest I've seen to running this in parallel is by using the pfs python code.

I often use this commands, it works so far.

git pull
git submodule foreach --recursive git checkout master
git submodule foreach --recursive git pull

Hope this faster.

From the top level in the repo:

git submodule foreach git checkout develop
git submodule foreach git pull

This will switch all branches to develop and pull latest

All you need to do now is a simple git checkout

Just make sure to enable it via this global config: git config --global submodule.recurse true

I think you'll have to write a script to do this. To be honest, I might install python to do it so that you can use os.walk to cd to each directory and issue the appropriate commands. Using python or some other scripting language, other than batch, would allow you to easily add/remove subprojects with out having to modify the script.

Related