How to move files during a refactor without causing conflicts?

Viewed 77

I'm working in a repo with the following directory structure:

my-project/
  a/
  b/
  c/

I need to refactor it and move a, b, and c into a subfolder, like this:

my-project/
  subfolder/
    a/
    b/
    c/

However, there are many active branches checked out where people are working on files inside a, b, and c and they're not yet in the subfolder. How can I perform this refactor without causing all of my teammates' branches to have conflicts when they try to merge?

1 Answers

As Evert commented, ideally, you don't have to do anything.

In practice, how well this works is somewhat variable. See the detailed discussion below. Note that any configuration settings, such as diff.renameLimit, are the responsibility of the person doing the diff or merge operation later. If that's you, you can set your settings now, but if that's someone else, they have to set their settings (whenever they like).

Details

It's important to know that Git does not store changes to files. Instead, Git stores snapshots of files. Each commit contains a complete snapshot of every file.

Each commit also contains some metadata—information about the commit itself—that includes the name and email address of whoever made the commit, for instance, but also includes the raw hash ID of its previous commit, which Git calls the commit's parent. This strings commits together, albeit backwards.

Each commit has its own unique hash ID. This hash ID is actually a cryptographic checksum of the contents of the commit, from both its main data—its snapshot—and its metadata. This means that once a commit is made, it can never be changed, not one bit: changing any single bit, or any group of bits, just makes a new commit, with a new, unique hash ID, while the existing commit is still there.

Git in general works backwards. A branch name holds the raw hash ID of the last commit. From there, Git finds the second-to-last commit, which finds the third-to-last, and so on:

... <-F <-G <-H   <--branch

The name branch contains the actual hash ID of some commit we're just calling H. When Git reads H from its big database, using that hash ID, H contains the hash ID of earlier commit G. Git can now read G from its Git-objects-database, which gets it the hash ID of earlier commit F. This lets git log and other commands work through the commits, backwards.

Again, each commit holds a snapshot of all of its files. But Git shows you changes. The way this works is that when you have Git show you some commit, Git looks up the commit's parent—its backwards-pointing link to its previous commit—and extracts into a temporary working area (in memory) both commits. Git can then compare the files in each commit.

If commit G's README.md and commit H's README.md are identical, Git won't even tell you that H has a README.md. If they're different, though, Git will compare the two different README.md files' content and show you what changed. So that's how you see changes.

You can compare any two commits, not just parent-and-child, but comparing parent-and-child is extremely common: many Git commands just do that automatically. Some, such as git diff, let you pick the two commits, and a few—such as git merge—pick a commit on their own, as we'll see in a moment.

Rename detection

If you rename some files, what really happens is that earlier commit G has a file named, say, README.txt, and later commit H has a file named README.md. Git notices that G doesn't have README.md and H doesn't have README.txt, and guesses that maybe, just maybe, you renamed these two files, across these two commits.

If you rename a whole collection of files—in Git's eyes, there are no directories; files just have long names: a/b/c.ext is the file's name, it's not a folder named a holding a folder named b and so on, it's just a long name with slashes in it—Git will try matching up each file-pair that it can. (Quite recently, there have been some attempts to improve the name-matching-up to take typical "folder-izing" into account. It's gone wrong a few times but I think it's back in now.)

This rename detecting is optional in Git's internal difference engine. When running git diff, it is on by default in modern Git, but off by default in very old versions of Git. You can force it on using git diff --find-renames (git diff -M for short) or set diff.renames to true in your configuration.

Merging and rename detection

When you run git checkout somebranch; git merge otherbranch, Git relies on the commit graph to find a merge base. I'm going to omit all the details about that here; see other answers for more.

Consider a commit graph that looks like this:

          I--J   <-- somebranch (HEAD)
         /
...--G--H
         \
          K--L   <-- otherbranch

That is, the names somebranch and otherbranch select commits J and L respectively. Commits through H are on both branches, while commits I-J are only on somebranch and, at the moment, commits K-L are only on otherbranch. You've run git checkout somebranch, as indicated by having HEAD attached to the name somebranch, and git merge otherbranch is now starting.

Git will now find merge base commit H on its own. Having found the merge base, Git now needs to convert the snapshots in H, J, and L into what you changed on somebranch and what they changed on otherbranch respectively.

Since git diff can find renames, merge just runs git diff with the find-renames option turned on for sure:

git diff --find-renames <hash-of-H> <hash-of-J>   # what we changed
git diff --find-renames <hash-of-H> <hash-of-L>   # what they changed

In a merge where there were no renames, the rename detector doesn't find anything, and Git just merges, e.g., the README.md in H based on the changes found by comparing H's README.md against J's README.md and then again based on L's README.md. When there are renames, though, Git has to pair up each file-pair. For instance, if:

  • README.txt in H is README.md in J, and
  • README.txt in H is README.txt in L

then you renamed the file, and they didn't. The combination of these operations is rename the file.

So, in your case, you are going to make a commit in which the files' names are now subfolder/a/file.ext and so on, when they used to just be a/file.ext and so on. With any luck, Git will correctly match up the merge-base a/file.ext with the other guy's a/file.ext, in that diff, and correctly match up a/file.ext with your subfolder/a/file.ext. The diff will show that one side renamed the file and the other side didn't, and the combination of those two changes includes "rename the file".

When does this go wrong?

Git's rename detector, as handled by its diff engine, depends on three things:

  • The file under its old name has to be there-on-the-left and not-there-on-the-right, and under its new name, it has to be there-on-the-right and not-there-on-the-left.

    That is, suppose we had path/to/file.ext and now have new/path/to/file.ext. We'll put the old commit on the left and the new one on the right. But what if we've also created a new and different path/to/file.ext on the right?

    Git won't even try to compare path/to/file.ext on the left to new/path/to/file.ext on the right, because it will have matched up the left-side path/to/file.ext with the new, but unrelated, path/to/file.ext on the right. Git will just call new/path/to/file.ext on the right a new file.

    Hence, the initial left-vs-right comparison has to show some "deleted" files on the left and some "new" files on the right. The rename detector will try to match up the left-side deleted files with the right-side added files, and convert such file-pairs to (detected) renames.

  • Even if you have such a pair, Git won't call the file renamed unless the contents are similar. That is, suppose you not only renamed the file, but also changed something. Git will do a fast test for similarity, expressed as a percentage. If the left and right side files are at least "50% similar", Git will consider this a candidate for a rename.

    Git has to do this for every left-and-right-side unpaired file. That is, for each file pair, Git has to compute a similarity index. Is left/file1.a similar to right/file2.b? How similar is left/file1.a to right/file3.c? Repeat for every file on both sides.

    To make this go faster, Git can easily match up 100%-identical files. You can therefore commit a rename op first, then commit change to the renamed files, and make things better when going commit-by-commit as git log does.

    This is less helpful during merges because git merge never goes commit-by-commit. (I think it should have an option to do so, just to find renames, but it does not.)

    The default similarity threshold, of 50%, is only a default. When running git diff you can raise or lower the minimum required similarity using the -M or --find-renames option with a number, as in -M30 to lower it to 30% or -M75 to raise it to 75%.1 When using git merge, you can set it with -X find-renames=<number> to choose a limit. (In old versions of Git, you need -X rename-threshold=<number> instead: check your particular Git version's documentation, with git help merge for instance.)

  • Last, Git imposes rename limits, because of the difficulty of finding similar files.

    The default limit in modern Git (Git 2.26) is 400, i.e., 400 renames can be detected. If you renamed 402 files, 400 of them will be detected and two won't. You can raise or lower this limit using diff.renameLimit. Setting it to zero tells Git not to artificially limit rename detection.

    The git merge command and has its own separate configuration knobs, but aside from defaulting to have rename detection on even in old versions of Git, it will obey your diff.renameLimit if you do not set a separate merge.renameLimit.

I set my diff.renameLimit to zero, to undo the rename cap. This makes some git diff and git merge commands run very slowly sometimes, but I like not having to worry about it (and I know to turn it back on if needed).


1Watch out, --find-renames=4 means 40%, not 4%. You can add the % character, --find-renames=4%, or just write it as --find-renames=04. It's probably not wise to lower the rename threshold too far, though, as Git will start finding renames that make no sense.

Related