How to fuse commits/changes from two branches/repos into the same commit timeline?

user_10827704 0
Viewed 217

I have a unique requirement for which I am searching for a solution... I did look at a lot of other solutions online and didn't find one that would suit my expectations...

There are two repos - X and Y (illustration below). I want to delete Repo X forever. But there is a folder, say A, that I want to carry to the second repo (Y) with all the Git Commit Histories too… It is okay to tamper with history and I want repo Y's history to claim that the folder A existed in the repo Y from the beginning and all changes to that folder A was made when it was inside repo Y. I do not want Repo Y's history to show that the folder A came after a merge from another branch/repo. I also do not want to rebase as all the commits (~1000) for the folder A will be dumped on top of the existing commits of Repo Y.

***** Original Commits on the timeline for Repo X and Repo Y *****
Repo Y: -------Cx--------Cy--------Cz----------
Repo X: -----------Ca----------Cb------Cc------  (only commits that touch Folder A)

I found a command to filter-branches and rewrite the commits of the repo X such that according to git only the content of folder A existed.

git filter-branch --subdirectory-filter ./A -- --all

And then used the following script to move all the content of the "Folder A" in to a folder named "A", in Repo X

# https://gist.github.com/xkr47/f766f4082112c086af63ef8d378c4304
# placed the script above in PATH and executed the following command at Repo X
git filter-mv 's!^!A/!'

So now I have the repo Y, which is the repo to which I need to move folder A from repo X with all the Git commit histories… I also have the modified repo X, where the only content is folder A

Please note that there is no possibility of a conflicting commits as all the commits on the repo X are done to a folder (A), that does not exist in repo Y at all

Now, I will be able to mark the local Repo X as a remote to my Repo Y and either Merge or rebase

# repo X lives in the same directory as repo Y
# commands run inside repo Y
git remote add repoX ../X

git pull repoX <branch> --allow-unrelated-histories
# or
git pull repoX <branch> --rebase

MERGE

***** MERGE's output *****
Repo Y: -------Cx--------Cy-------Cz-----(MERGE)----
                                         /
                                        /
              -----Ca---------Cb------Cc  (Folder A commits)

The commits’ timeline will be maintained, but will be on different tracks till both are merged, and I will not be able to see the contents from both the repos together at any point before the merge commit as they will be on separate commit lines (I want folder A, with all the changes it has undergone, at repo Y and it’s commit lines) (… so that I can go back to any point in Repo Y and see the evolution of both Repo Y and Folder A together, according to the time of the commit)

REBASE

***** REBASE's output *****
Repo Y: -------Cx--------Cy--------Cz------Ca-----Cb-----Cc----

But I do not want the commits from Repo X to be after the latest commits of Repo Y… In my use case, Repo X will have ~1000 commits and rebasing them all at once will spoil the commit timeline of Repo Y. I want them to be fused within the commits of Repo Y according to the time in which the commit had happened

EXPECTED

***** Expected Output *****
Repo Y: -------Cx--Ca----Cy----Cb--Cz--Cc------  (Repo Y commits fused with commits from repo X - only adds a folder (A) to repo Y)

I know that meddling with Git Histories is not a good thing. But for my use case, this is probably the best thing I can do…

git
2 Answers

Linear histories scenario

If your "Repo Y" is single-branched and linear (no merges; merges can be linearized with simple git rebase however), then one of possible solutions I can think of is replacing the generated rebase sequence with the one containing the fused commits in proper order.

  • At your "Repo Y", make sure you're on the master branch and tag the current master branch commit in case of something goes wrong:
git checkout master
git tag old-master-before-fuse
  • Fetch "Repo X" into the repository so that this repository is aware of objects in two histories:
git remote add X ...path-to-your-Repo-X...
git fetch X
  • Prepare the interactive rebase sequence by sorting commits from both repositories from oldest to newest:
git rev-list --format='%at %ct %H' refs/heads/master refs/remotes/X/master \ # produce a table of all commits from both repositories (fields: author timestamp, committer timestamp, commit hash)
    | grep -P '^\d+' \         # don't know to remove the "commit XXXX" lines from the output above otherway
    | sort -k1 -n \            # sort by the author timestamp
    | cut -d' ' -f3 \          # take hashes only
    | while read LINE; do      # prepare each hash for the interactive rebase sequencing
        echo "pick $LINE"
    done \
> .git-rebase-todo # save it to a temporary file to be used later
  • Start the interactive rebase from the very first commit:
git rebase -i --root

Once your editor shows up, remove all "pick" commands with a single "break", and exit the editor. Git will stop here (git rebase --continue without further steps will make the master branch blank).

  • Replace the current to-do sequence with the previously generated file .git-rebase-todo and proceed the rebase:
cp .git-rebase-todo .git/rebase-merge/git-rebase-todo
git rebase --continue

This will make the master branch to contain the fused commits history.

  • Examine the master branch and if everything went fine, delete the checkpoint tag (use with caution of course: git tag -d old-master-before-fuse) and the remote link (git remote remove X).

Here are the dumps of the repositories I used for the test:

"Repo X"

blob
mark :1
data 0

reset refs/heads/master
commit refs/heads/master
mark :2
author - <-> 1577886449 +0200
committer - <-> 1606398449 +0200
data 3
Cx
M 100644 :1 Cx

commit refs/heads/master
mark :3
author - <-> 1580564850 +0200
committer - <-> 1606398450 +0200
data 3
Cy
from :2
M 100644 :1 Cy

commit refs/heads/master
mark :4
author - <-> 1583070451 +0200
committer - <-> 1606398451 +0200
data 3
Cz
from :3
M 100644 :1 Cz

"Repo Y"

blob
mark :1
data 0

reset refs/heads/master
commit refs/heads/master
mark :2
author - <-> 1577972852 +0200
committer - <-> 1606398452 +0200
data 3
Ca
M 100644 :1 Ca

commit refs/heads/master
mark :3
author - <-> 1580651253 +0200
committer - <-> 1606398453 +0200
data 3
Cb
from :2
M 100644 :1 Cb

commit refs/heads/master
mark :4
author - <-> 1583156854 +0200
committer - <-> 1606398454 +0200
data 3
Cc
from :3
M 100644 :1 Cc

The dumps can be restored by using git fast-import.

With these two sample repositories, the fuse routine would produce the following git log --oneline result:

7efcbbd (HEAD -> master) Cc
6cc905d Cz
a20f00e Cb
6650162 Cy
e8e9bf1 Ca
40678ea Cx
user_122328700

Non-linear histories scenario

If your histories are not linear, and making them linear is not an option (you don't want to do that or it results in many conflicts that might be resolved with mistakes), you can also attempt to fuse the repository by editing the interactive rebase sequence manually

Suppose your repository Y (host) has the following master branch graph:

# Cx
touch Cx && git add Cx && git commit -m Cx Cx
git branch -m y/master
# Cy
git checkout -b y/master-Cy && touch Cy && git add Cy && git commit -m Cy \
    && git checkout - && git merge --no-edit --no-ff y/master-Cy && git branch -d y/master-Cy
# Cz
git checkout -b y/master-Cz && touch Cz && git add Cz && git commit -m Cz \
    && git checkout - && git merge --no-edit --no-ff y/master-Cz && git branch -d y/master-Cz

*   478d271 (HEAD -> y/master) - Merge branch 'y/master-Cz' into y/master 3 seconds ago 
|\  
| * 93fbaf8 - Cz 3 seconds ago 
|/  
*   e80b2ea - Merge branch 'y/master-Cy' into y/master 6 seconds ago 
|\  
| * a1842d1 - Cy 6 seconds ago 
|/  
* 9467260 - Cx 31 seconds ago

For brevity and omitting the repository X origin fetching stage suppose we can build it right in the same repository.

git checkout --orphan x/master
git reset --hard
# Ca
touch Ca && git add Ca && git commit -m Ca Ca
# Cb
git checkout -b x/master-Cb && touch Cb && git add Cb && git commit -m Cb \
    && git checkout - && git merge --no-edit --no-ff x/master-Cb && git branch -d x/master-Cb
# Cc
git checkout -b x/master-Cc && touch Cc && git add Cc && git commit -m Cc \
    && git checkout - && git merge --no-edit --no-ff x/master-Cc && git branch -d x/master-Cc

*   e271022 (HEAD -> x/master) - Merge branch 'x/master-Cc' into x/master 1 second ago 
|\  
| * a403baf - Cc 1 second ago 
|/  
*   c34f424 - Merge branch 'x/master-Cb' into x/master 4 seconds ago 
|\  
| * 6107552 - Cb 4 seconds ago 
|/  
* 70d7b2d - Ca 7 seconds ago

There are two branches at that moment: y/master that stands for your host repository master branch, and x/master that stands for your foreign repository master branch

Now, let's prepare the interactive rebase TODO sequence for both master branches:

git checkout y/master
git rebase -i --rebase-merges --root

l onto

# Branch y-master-Cy
t [new root]
p 9467260 Cx
l branch-point
p a1842d1 Cy
l y-master-Cy

# Branch y-master-Cz
t branch-point # Cx
m -C e80b2ea y-master-Cy # Merge branch 'y/master-Cy' into y/master
l branch-point-2
p 93fbaf8 Cz
l y-master-Cz

t branch-point-2 # Merge branch 'y/master-Cy' into y/master
m -C 478d271 y-master-Cz # Merge branch 'y/master-Cz' into y/master

(Please save it as a text file somewhere, named say y-master.txt)

git checkout x/master # or git checkout to the foreign repository remote ref (that leads to a detached HEAD but it's definitely not an issue)
git rebase -i --rebase-merges --root
git checkout y/master

l onto

# Branch x-master-Cb
t [new root]
p 70d7b2d Ca
l branch-point
p 6107552 Cb
l x-master-Cb

# Branch x-master-Cc
t branch-point # Ca
m -C c34f424 x-master-Cb # Merge branch 'x/master-Cb' into x/master
l branch-point-2
p a403baf Cc
l x-master-Cc

t branch-point-2 # Merge branch 'x/master-Cb' into x/master
m -C e271022 x-master-Cc # Merge branch 'x/master-Cc' into x/master

(Please save this one too, x-master.txt)

Now, assuming you have a snapshot tag for the y/master (in order not to crash the original branch), the "only" thing is now to "program" the fused repository sequence:

git rebase -i --rebase-merges --root

(The tabbed commands indicate the X repository commands. The FOREIGN- prefixes are inserted manually.)

l onto

t [new root]
p 9467260 Cx

    # DON'T or the `9467260 Cx` will get lost: t [new root]
    p 70d7b2d Ca

l branch-point
p a1842d1 Cy
l y-master-Cy
t branch-point # Cx
m -C e80b2ea y-master-Cy # Merge branch 'y/master-Cy' into y/master

    l FOREIGN-branch-point
    p 6107552 Cb
    l FOREIGN-x-master-Cb
    t FOREIGN-branch-point # Ca
    m -C c34f424 FOREIGN-x-master-Cb # Merge branch 'x/master-Cb' into x/master

l branch-point-2
p 93fbaf8 Cz
l y-master-Cz
t branch-point-2 # Merge branch 'y/master-Cy' into y/master
m -C 478d271 y-master-Cz # Merge branch 'y/master-Cz' into y/master

    l FOREIGN-branch-point-2
    p a403baf Cc
    l FOREIGN-x-master-Cc
    t FOREIGN-branch-point-2 # Merge branch 'x/master-Cb' into x/master
    m -C e271022 FOREIGN-x-master-Cc # Merge branch 'x/master-Cc' into x/master

This would result into something like this:

*   cbbf73b (HEAD -> y/master) - Merge branch 'x/master-Cc' into x/master 10 minutes ago 
|\  
| * b143f45 - Cc 10 minutes ago 
|/  
*   bd0c586 - Merge branch 'y/master-Cz' into y/master 3 hours ago 
|\  
| * dfb5e4e - Cz 3 hours ago 
|/  
*   01f078d - Merge branch 'x/master-Cb' into x/master 10 minutes ago 
|\  
| * 26b2f61 - Cb 10 minutes ago 
|/  
*   f1105a4 - Merge branch 'y/master-Cy' into y/master 3 hours ago 
|\  
| * aa67233 - Cy 3 hours ago 
|/  
* 7a23a6e - Ca 10 minutes ago 
* 9467260 - Cx 3 hours ago

This looks pretty much like the commit sequence you've described in the question + it incorporates test merges. It can probably be scripted somehow, but the scripting of such a method is definitely beyond what I'm able to do.

The approach above is not well-tested, and it may contain mistakes or something that may make your fused history wrong.

It is way easier to linearize both histories, even if there are text conflicts (that can resolved more or less easily), and the fuse them together. Diverged commits require your intervention into the rebase sequence.

Here is also a gawk script that might help you to build the timeline timestamps for each commit appearing in the generated rebase sequences as comments:

#!/usr/bin/awk

function print_with_extra(orig_line, object) {
    cmd = "git rev-list -1 --format='%ai%n%ci' " object
    author_date = ""
    committer_date = ""
    i = 0
    while ( (cmd | getline line) > 0 ) {
        switch ( ++i ) {
        case 2: author_date = line; break
        case 3: committer_date = line; break
        }
    }
    close(cmd)
    print orig_line " # (" author_date ") (" committer_date ")"
}

{
    switch ( $0 ) {
    case /^p [0-9a-f]+/: print_with_extra($0, $2); break
    case /^m -C [0-9a-f]+/: print_with_extra($0, $3); break
    default: print; break
    }
}

Example of use: gawk -f timestamps.awk y-master.txt or gawk f- timestamps.awk x-master.txt respectively. And example output:

l onto

# Branch x-master-Cb
t [new root]
p 9979de5 Ca # (2020-11-30 15:03:26 +0200) (2020-11-30 15:03:26 +0200)
l branch-point
p 4dcff42 Cb # (2020-11-30 15:03:26 +0200) (2020-11-30 15:03:26 +0200)
l x-master-Cb

# Branch x-master-Cc
t branch-point # Ca
m -C 4de8360 x-master-Cb # Merge branch 'x/master-Cb' into x/master # (2020-11-30 15:03:26 +0200) (2020-11-30 15:03:26 +0200)
l branch-point-2
p 206f5a8 Cc # (2020-11-30 15:03:26 +0200) (2020-11-30 15:03:26 +0200)
l x-master-Cc

t branch-point-2 # Merge branch 'x/master-Cb' into x/master
m -C c3ac7e5 x-master-Cc # Merge branch 'x/master-Cc' into x/master # (2020-11-30 15:03:26 +0200) (2020-11-30 15:03:26 +0200)
user_122328700
Related