What happened to the changes that I made using git filter-branch?

user_62163 0
Viewed 240

For the first time, I tried to rewrite my git history using git filter-branch. I did this by writing a (Python) script (let's call it edit_file) that makes an edit to a file (let's call it target_file). Then, I ran this command:

git filter-branch --tree-filter "path/to/edit_file" HEAD

I got a nice stream of output which seemed to indicate that I'd gotten the effect that I wanted, but when viewed target_file, I did not see any changes. When I run edit_file directly, target_file in my working copy successfully receives the edits that I intended.

It sounds like my changes exist within some deep, dark, and dank recess in git's twisted mind, and I just need a magical incantation to summon forth my changes. I have no idea whether this is correct, nor do I understand where to begin looking, because all the material that I've read (including the official git book) indicates that once git filter-branch finishes, the branch that I am working on should have the changes that edit_file would perform on every version of target_file...

Halp?

Sorry if this is a bit long winded, but I don't know what details are necessary (because that is a core feature of being confused).

More details:

The reasons I say that it looked like filter-branch did what I wanted are:

  1. I could see the output of edit_file running on each commit, and it indicated success on all of them. The output of edit_file changes as it operates on different versions of target_file, and I was able to see the different output from edit_file as git filter-branch moved through history.

  2. At the end, I saw this:

    Ref 'refs/heads/my-branch' was rewritten

PS: Before doing git filter-branch, I ran

git checkout -b my-branch

to create a new branch named my-branch (and check it out), in case git filter-branch went horribly wrong.

After seeing that git filter-branch ... left target_file unchanged, I ran git checkout -b my-branch, but I guess that did nothing. I thought it might do something, because the last line from git filter-branch seems to be saying that the branch my-branch has been changed, but I honestly do not understand what that line means.

git-filter-branch git
2 Answers
VonC_63090

So... It looks like git filter-branch did not do anything except leave weird back up poop in the .git dir. git gc does not clean that up for whatever reason (maybe there should be a git clean-poop command as well as gc?). Not sure what will, other than

cd ..  # Assuming you are at the root of your repo
git clone --no-local original fresh-copy
cd fresh-copy

Yes, even though we are making a local copy, --no-local is needed, because This. Is. GIIIIIIIIIT! This is probably what you should do before attempting filter-branch or filter-repo. Not sure why the documentation does not advise this, but anyway. Don't do what I did, and just skimp out by creating a new branch. Consider git filter-X to be a nuclear weapon. You don't just need bunker to protect you; you need disposable parallel universe.

After much gnashing of teeth, I was finally able to get VonC's suggestion of using git-filter-repo to work. If, like me, your platform does not have a new enough version of git to work with git-filter-repo (requires >= 2.22), you may be able to do something like

sudo add-apt-repository ppa:git-core/ppa
# followed by the usual
sudo apt-get update
# song and dance routine...
sudo apt-get install git --upgrade

as suggested on the git *nix download page to upgrade to the "latest and greatest". That did not work for me (so take care about blindly copying and pasting the above suggestion), but apparently, I have a very deranged system, so you may have better luck than I did. Anyway...

Once you obtain a new enough version of git, the only thing you should need is the git-filter-repo script itself (amazingly, it just consists of the one main file). So just download that straight from github, and stick it anywhere on your PATH. Remember to chmod +x the shit out of it first tho.

You will most likely NOT want to use the --path flag, because that will NOT just target the one file that you want to edit. Instead, --path will nuke all other files.

With that in mind, all you need to do is something like this:

git-filter-repo --blob-callback 'import sys
sys.path.append("dir/where/your/edit_file/py/file/lives")
import my_module

new = my_module.modify(blob.data.decode())
new_bytes = new.encode()
assert isinstance(new_bytes, bytes), ""
blob.data = new_bytes
'

Yes, that is all one command. blob.data contains the contents of whatever file git-filter-repo is examining. Furthermore, notice that it is a bytes object, not str. Let me re-emphasize a very important point: this operation goes over EVERY file (in every commit). So, your my_module.modify function better be very selective if you only intend to modify one file. (What git-filter-repo really needs is a way for your script to detect the path of blob, not just give you the contents of a file. But hopefully, you can recognize your file by its contents, not just its path.) If you mess up, it's not a big deal, because you can just nuke the fresh-copy dir and start over.

Refrain from gouging your eyes out after this ordeal. Your eyes are way too valuable to be destroyed over something as stupid as git. Feel free to have a good cry in the shower tho. Hey, at least you got it working finally, and you weren't eaten by sharks.

Oh, and another thing: the git clone you did in the first step does not copy over any submodules, because that would make sense and be too easy. Therefore, you must also do this in fresh-copy (even though git clone DOES copy over the .gitmodules file):

git submodule init
git submodule update

PS: You may get some therapeutic value from the one of the various git man page generators available on the Interwebs. They really do read like git man pages, even though they are literally randomized gibberish.

user_621630
Related