How do you fix a bad merge, and replay your good commits onto a fixed merge?

Please don't use this recipe if your situation is not the one described in the question. This recipe is for fixing a bad merge, and replaying your good commits onto a fixed merge.

Although filter-branch will do what you want, it is quite a complex command and I would probably choose to do this with git rebase. It's probably a personal preference. filter-branch can do it in a single, slightly more complex command, whereas the rebase solution is performing the equivalent logical operations one step at a time.

Try the following recipe:

# create and check out a temporary branch at the location of the bad merge
git checkout -b tmpfix <sha1-of-merge>

# remove the incorrectly added file
git rm somefile.orig

# commit the amended merge
git commit --amend

# go back to the master branch
git checkout master

# replant the master branch onto the corrected merge
git rebase tmpfix

# delete the temporary branch
git branch -d tmpfix

(Note that you don't actually need a temporary branch, you can do this with a 'detached HEAD', but you need to take a note of the commit id generated by the git commit --amend step to supply to the git rebase command rather than using the temporary branch name.)

If you haven't committed anything since, just git rm the file and git commit --amend.

If you have

git filter-branch \
--index-filter 'git rm --cached --ignore-unmatch path/to/file/filename.orig' merge-point..HEAD

will go through each change from merge-point to HEAD, delete filename.orig and rewrite the change. Using --ignore-unmatch means the command won't fail if for some reason filename.orig is missing from a change. That's the recommended way from the Examples section in the git-filter-branch man page.

Note for Windows users: The file path must use forward slashes

This is the best way:
http://github.com/guides/completely-remove-a-file-from-all-revisions

Just be sure to backup the copies of the files first.

EDIT

The edit by Neon got unfortunately rejected during review.
See Neons post below, it might contain useful information!

E.g. to remove all *.gz files accidentally committed into git repository:

$ du -sh .git ==> e.g. 100M
$ git filter-branch --index-filter 'git rm --cached --ignore-unmatch *.gz' HEAD
$ git push origin master --force
$ rm -rf .git/refs/original/
$ git reflog expire --expire=now --all
$ git gc --prune=now
$ git gc --aggressive --prune=now

That still didn't work for me? (I am currently at git version 1.7.6.1)

$ du -sh .git ==> e.g. 100M

Not sure why, since I only had ONE master branch. Anyways, I finally got my git repo truely cleaned up by pushing into a new empty and bare git repository, e.g.

$ git init --bare /path/to/newcleanrepo.git
$ git push /path/to/newcleanrepo.git master
$ du -sh /path/to/newcleanrepo.git ==> e.g. 5M 

(yes!)

Then I clone that to a new directory and moved over it's .git folder into this one. e.g.

$ mv .git ../large_dot_git
$ git clone /path/to/newcleanrepo.git ../tmpdir
$ mv ../tmpdir/.git .
$ du -sh .git ==> e.g. 5M 

(yeah! finally cleaned up!)

After verifying that all is well, then you can delete the ../large_dot_git and ../tmpdir directories (maybe in a couple weeks or month from now, just in case...)

This is what git filter-branch was designed for.