Why is my git repository so big?

I recently pulled the wrong remote repository into the local one (git remote add ... and git remote update). After deleting the unwanted remote ref, branches and tags I still had 1.4GB (!) of wasted space in my repository. I was only able to get rid of this by cloning it with git clone file:///path/to/repository. Note that the file:// makes a world of difference when cloning a local repository - only the referenced objects are copied across, not the whole directory structure.

Edit: Here's Ian's one liner for recreating all branches in the new repo:

d1=#original repo
d2=#new repo (must already exist)
cd $d1
for b in $(git branch | cut -c 3-)
do
    git checkout $b
    x=$(git rev-parse HEAD)
    cd $d2
    git checkout -b $b $x
    cd $d1
done

git gc already does a git repack so there is no sense in manually repacking unless you are going to be passing some special options to it.

The first step is to see whether the majority of space is (as would normally be the case) your object database.

git count-objects -v

This should give a report of how many unpacked objects there are in your repository, how much space they take up, how many pack files you have and how much space they take up.

Ideally, after a repack, you would have no unpacked objects and one pack file but it's perfectly normal to have some objects which aren't directly reference by current branches still present and unpacked.

If you have a single large pack and you want to know what is taking up the space then you can list the objects which make up the pack along with how they are stored.

git verify-pack -v .git/objects/pack/pack-*.idx

Note that verify-pack takes an index file and not the pack file itself. This give a report of every object in the pack, its true size and its packed size as well as information about whether it's been 'deltified' and if so the origin of delta chain.

To see if there are any unusally large objects in your repository you can sort the output numerically on the third of fourth columns (e.g. | sort -k3n).

From this output you will be able to see the contents of any object using the git show command, although it is not possible to see exactly where in the commit history of the repository the object is referenced. If you need to do this, try something from this question.

Are you sure you are counting just the .pack files and not the .idx files? They are in the same directory as the .pack files, but do not have any of the repository data (as the extension indicates, they are nothing more than indexes for the corresponding pack — in fact, if you know the correct command, you can easily recreate them from the pack file, and git itself does it when cloning, as only a pack file is transferred using the native git protocol).

As a representative sample, I took a look at my local clone of the linux-2.6 repository:

$ du -c *.pack
505888  total

$ du -c *.idx
34300   total

Which indicates an expansion of around 7% should be common.

There are also the files outside objects/; in my personal experience, of them index and gitk.cache tend to be the biggest ones (totaling 11M in my clone of the linux-2.6 repository).

Other git objects stored in .git include trees, commits, and tags. Commits and tags are small, but trees can get big particularly if you have a very large number of small files in your repository. How many files and how many commits do you have?

Did you try using git repack?

I've created a new implementation of the perl script that was originally provided in this answer (which has since been rewritten in rust). After much investigation of that perl script, I realized that it had multiple bugs:

• Errors with paths with spaces
• --sum didn't work correctly (it wasn't actually adding up all the deltas)
• --directory didn't work correctly (it relies on --sum)
• Without --sum it would report a size of an effectively-random object for the given path, which might not have been the largest one

So I ended up rewriting the script entirely. It uses the same sequence of git commands (git rev-list and git cat-file) but then it processes the data correctly to give accurate results. I preserved the --sum and --directories features.

I also changed it to report the "disk" size (i.e. the compressed size in the git repo) of the files, rather than the original file sizes. That seems more relevant to the problem at hand. (This could be made optional, if someone wants the uncompressed sizes for some reason.)

I also added an option to only report on files that have been deleted, on the assumption that files still in use are probably less interesting. (The way I did that was a bit of a hack; suggestions welcome.)

The latest script is here. I can also copy it here if that's good StackOverflow etiquette? (It's ~180 lines long.)

Create new branch where current commit is the initial commit with all history gone to reduce git objects and history size.

Note: Please read the comment before running the code.

1. git checkout --orphan latest_branch
2. git add -A
3. git commit -a -m “Initial commit message” #Committing the changes
4. git branch -D master #Deleting master branch
5. git branch -m master #renaming branch as master
6. git push -f origin master #pushes to master branch
7. git gc --aggressive --prune=all # remove the old files