Git - Extracting Last Commit Data and Ahead of Master Info and Exporting to CSV File

Viewed 23

I want to write a script that writes to a csv file the name of all branches that I have in bitbucket which also includes the following fields:

  1. last commit id
  2. last commit developer
  3. last commit date
  4. last commit message
  5. ahead of master (Yes/No)

I found the following script that generates the branch name and the last commit developer and saves it to a csv file:

for i in git branch -a |grep remotes |awk '{print $1}' | cut -d"/" -f 3,4,5 |grep -v ^master$ |grep -v ^HEAD$; do echo "git log -1 --pretty=format:\"%an (%ae)\" origin/$i|$i"; done |sort > /tmp/branches.csv

Any idea how this script should be adjusted in order to get the last commit id, last commit date, last commit message and whether it is ahead of master or not? Any other idea is also mostly welcome.

Thanks, Benny

1 Answers

The last commit in any branch is, by definition, the hash ID stored in the branch name.

Given that you're using refs/remotes/origin/blah to look at origin/blah, rather than branch names, you're looking at your own remote-tracking names. These are copied from some other Git repository's branch names, so they have the same property. But these are not branch names, they're remote-tracking names. (In some other Git repository, they are, or were at the time, branch names. By now, they could be literally seconds out of date. You need to make sure that's OK, for whatever purposes you have. Consider running git fetch --prune or git remote update --prune so that it's only seconds, and not minutes or hours or worse.)

Your existing code has a (minor-ish) flaw, illustrated here:

$ echo remotes/origin/feature/one/two/three/four | cut -d"/" -f 3,4,5 
feature/one/two

Instead of using git branch -a, though, you should use the Git plumbing command, git for-each-ref, here. This enables you to use %(refname:short):

$ git for-each-ref --format='%(refname:short)' refs/remotes
origin/HEAD
origin/main
origin/maint
origin/master
origin/next
origin/seen
origin/todo

for instance. You probably still want to ditch the HEAD symbolic ref and the master one but you can now get the commit hash ID directly:

$  git for-each-ref --format='%(objectname) %(refname:short)' refs/remotes
9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c origin/HEAD
9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c origin/main
ad60dddad72dfb8367bd695028b5b8dc6c33661b origin/maint
9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c origin/master
91fe8e635439f67be8837601cbf4bd61eddc41b4 origin/next
d0cdfd77733ad946e8c60e9b50286778fb813e56 origin/seen
59d992158534c8291f548563a9c949dae4ad7796 origin/todo

Pipe this through a grep -v to ditch /master and /HEAD as before:

$ git for-each-ref --format='%(objectname) %(refname:short)' refs/remotes | grep -E -v '/master$|/HEAD$'
9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c origin/main
ad60dddad72dfb8367bd695028b5b8dc6c33661b origin/maint
91fe8e635439f67be8837601cbf4bd61eddc41b4 origin/next
d0cdfd77733ad946e8c60e9b50286778fb813e56 origin/seen
59d992158534c8291f548563a9c949dae4ad7796 origin/todo

It's possible to reject the unwanted names in the for-each-ref directly, but it's much more complicated, and we have to run git log to get author information anyway, so we might as well leave the for-each-ref part here.

What we should do now, though, is pipe the output to a shell script. This script can:

  • get more information from the commit, such as author and committer name;
  • compute the ahead and/or behind counts with respect to any other commit, including that identified by refs/remotes/origin/master;
  • format the output as desired.

From here it's simple shell programming, with one caveat: there's no plumbing equivalent of git log, so you must use the porcelain command here. See the PRETTY FORMATS section of the git log documentation for all the available %-directives.

To compute ahead and behind values for the commit with hash $h as compared to refs/remotes/origin/master, use git rev-list --count --left-right:

$ git rev-list --count --left-right d0cdfd77733ad946e8c60e9b50286778fb813e56...refs/remotes/origin/master
253     0

which shows that origin/seen here is 253 ahead of origin/master and zero behind origin/master (or equivalently, origin/master is zero ahead of origin/seen, and 253 behind). Note that this requires the three-dot syntax, and does not depend on upstream settings.

While git log -1 --format=%ad d0cdfd77733ad946e8c60e9b50286778fb813e56 works fine:

$ git log -1 --format=%ad d0cdfd77733ad946e8c60e9b50286778fb813e56
Wed Aug 17 15:47:50 2022 -0700

I personally prefer git log --no-walk. It's possible to write multiple commit hash IDs into git log --no-walk (e.g., using --stdin), which is not the case with -1. In this particular case, where you want to control everything one item at a time, there's no overriding reason to prefer --no-walk, though.

Your final script will probably resemble:

git for-each-ref --format='%(objectname) %(refname:short)' refs/remotes |
grep -E -v '/master$|/HEAD$' |
while read hash ref; do
    author=$(git log --no-walk --format="%an (%ae)" $hash);
    authordate=$(git log --no-walk --format="%ad" $hash);
    aheadbehind=$(git rev-list --count --left-right $hash...origin/master);
    ... insert other items here as desired ...
    echo "$ref,$hash,$author,$authordate,$aheadbehind,..."
done > output-csv.csv

or similar.

If you only want an "ahead" count, rather than ahead and behind, note that git rev-list --count refs/remotes/origin/master..$hash (note: two dots, no --left-right) obtains that number.

This can be made more efficient, at the cost of readability. The biggest efficiency gains might be made by using a single git log invocation on the specified hash: you may be able to produce most of the information you want in the correct format. Note, however, that there's a danger that someone's name will contain a comma (or whatever your CSV-file delimiter is). It's always wise to sanitize text. You don't want to be the school administrator in charge of the database when Little Bobby Tables shows up.

Related