How to quickly find all git repos under a directory

user_553865 0
Viewed 38219

The following bash script is slow when scanning for .git directories because it looks at every directory. If I have a collection of large repositories it takes a long time for find to churn through every directory, looking for .git. It would go much faster if it would prune the directories within repos, once a .git directory is found. Any ideas on how to do that, or is there another way to write a bash script that accomplishes the same thing?

#!/bin/bash

# Update all git directories below current directory or specified directory

HIGHLIGHT="\e[01;34m"
NORMAL='\e[00m'

DIR=.
if [ "$1" != "" ]; then DIR=$1; fi
cd $DIR>/dev/null; echo -e "${HIGHLIGHT}Scanning ${PWD}${NORMAL}"; cd ->/dev/null

for d in `find . -name .git -type d`; do
  cd $d/.. > /dev/null
  echo -e "\n${HIGHLIGHT}Updating `pwd`$NORMAL"
  git pull
  cd - > /dev/null
done

Specifically, how would you use these options? For this problem, you cannot assume that the collection of repos is all in the same directory; they might be within nested directories.

top
  repo1
  dirA

  dirB
     dirC
        repo1
bash
8 Answers

The answers above all rely on finding a ".git" repository. However not all git repos have these (e.g. bare repos). The following command will loop through all directories and ask git if it considers each to be a directory. If so, it prunes sub dirs off the tree and continues. 

find . -type d -exec sh -c 'cd "{}"; git rev-parse --git-dir 2> /dev/null 1>&2' \; -prune -print

It's a lot slower than other solutions because it's executing a command in each directory, but it doesn't rely on a particular repository structure. Could be useful for finding bare git repositories for example.

user_22746680

I list all git repositories anywhere in the current directory using:

find . -type d -execdir test -d {}/.git \\; -prune -print

This is fast since it stops recursing once it finds a git repository. (Although it does not handle bare repositories.) Of course, you can change the . to whatever directory you want. If you need, you can change the -print to -print0 for null-separated values.

To also ignore directories containing a .ignore file:

find . -type d \( -execdir test -e {}/.ignore \; -prune \) -o \( -execdir test -d {}/.git \; -prune -print \)

I've added this alias to my ~/.gitconfig file:

[alias]
  repos =  !"find -type d -execdir test -d {}/.git \\; -prune -print"

Then I just need to execute:

git repos

To get a complete listing of all the git repositories anywhere in my current directory.

user_6333060

This answer combines the partial answer provided @Greg Barrett with my optimized answer above.

#!/bin/bash

# Update all git directories below current directory or specified directory
# Skips directories that contain a file called .ignore

HIGHLIGHT="\e[01;34m"
NORMAL='\e[00m'

export PATH=${PATH/':./:'/:}
export PATH=${PATH/':./bin:'/:}
#echo "$PATH"

DIRS="$( find "$@" -type d \( -execdir test -e {}/.ignore \; -prune \) -o \( -execdir test -d {}/.git \; -prune -print \) )"

echo -e "${HIGHLIGHT}Scanning ${PWD}${NORMAL}"
for d in $DIRS; do
  cd "$d" > /dev/null
  echo -e "\n${HIGHLIGHT}Updating `pwd`$NORMAL"
  git pull 2> >(sed -e 's/X11 forwarding request failed on channel 0//')
  cd - > /dev/null
done
user_5538650
Related