How to delete many files of same format from git history

user_3411155 0
Viewed 44

I have situation where I need to delete many documentation files like *.html files from git history

These HTML files over years of development causing git repo to bloat its size. It's difficult now to check out 10s of GBs every time.

I am able to figure out which file formats are the reason for this kind of bloating using scripts found in internet, e.g.

git rev-list --objects --all |
  git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' |
  sed -n 's/^blob //p' |
  sort --numeric-sort --key=2 |
  cut -c 1-12,41- |
  $(command -v gnumfmt || echo numfmt) --field=2 --to=iec-i --suffix=B --padding=7 --round=nearest**

up on segregating documentation file formats using grep. I hit dead end.

git-history git
1 Answers

You could use the thrid-party tool git filter-repo (python needed), with callbacks

Specifcally, a filename callback.

  • Returning None means the file should be removed from all commits,
  • returning the filename unmodified marks the file to be kept, and
  • returning a different name means the file should be renamed.

An example:

git-filter-repo --filename-callback '
  if b"/src/" in filename:
    # Remove all files with a directory named "src" in their path
    # (except when "src" appears at the toplevel).
    return None
  elif filename.startswith(b"tools/"):
    # Rename tools/ -> scripts/misc/
    return b"scripts/misc/" + filename[6:]
  else:
    # Keep the filename and do not rename it
    return filename
  '

If you need to look at those files content (with, for instance, gnumfmt), then a bob callback is needed:

git filter-repo --blob-callback '
  if len(blob.data) > 25:
    # Mark this blob for removal from all commits
    blob.skip()
  else:
    blob.data = blob.data.replace(b"Hello", b"Goodbye")
  '
VonC_63090
Related