How do I migrate an SVN repository with history to a new Git repository?

Viewed 523595

I read the Git manual, FAQ, Git - SVN crash course, etc. and they all explain this and that, but nowhere can you find a simple instruction like:

SVN repository in: svn://myserver/path/to/svn/repos

Git repository in: git://myserver/path/to/git/repos

git-do-the-magic-svn-import-with-history \
svn://myserver/path/to/svn/repos \
git://myserver/path/to/git/repos

I don't expect it to be that simple, and I don't expect it to be a single command. But I do expect it not to try to explain anything - just to say what steps to take given this example.

34 Answers

Magic:

$ git svn clone http://svn/repo/here/trunk

Git and SVN operate very differently. You need to learn Git, and if you want to track changes from SVN upstream, you need to learn git-svn. The git-svn main page has a good examples section:

$ git svn --help

Cleanly Migrate Your Subversion Repository To a Git Repository. First you have to create a file that maps your Subversion commit author names to Git commiters, say ~/authors.txt:

jmaddox = Jon Maddox <jon@gmail.com>
bigpappa = Brian Biggs <bigpappa@gmail.com>

Then you can download the Subversion data into a Git repository:

mkdir repo && cd repo
git svn init http://subversion/repo --no-metadata
git config svn.authorsfile ~/authors.txt
git svn fetch

If you’re on a Mac, you can get git-svn from MacPorts by installing git-core +svn.

If your subversion repository is on the same machine as your desired git repository, then you can use this syntax for the init step, otherwise all the same:

git svn init file:///home/user/repoName --no-metadata

I suggest getting comfortable with Git before trying to use git-svn constantly, i.e. keeping SVN as the centralized repo and using Git locally.

However, for a simple migration with all the history, here are the few simple steps:

Initialize the local repo:

mkdir project
cd project
git svn init http://svn.url

Mark how far back you want to start importing revisions:

git svn fetch -r42

(or just "git svn fetch" for all revs)

Actually, fetch everything since then:

git svn rebase

You can check the result of the import with Gitk. I'm not sure if this works on Windows, it works on OSX and Linux:

gitk

When you've got your SVN repo cloned locally, you may want to push it to a centralized Git repo for easier collaboration.

First create your empty remote repo (maybe on GitHub?):

git remote add origin git@github.com:user/project-name.git

Then, optionally sync your main branch so the pull operation will automatically merge the remote master with your local master when both contain new stuff:

git config branch.master.remote origin
git config branch.master.merge refs/heads/master

After that, you may be interested in trying out my very own git_remote_branch tool, which helps to deal with remote branches:

First explanatory post: "Git remote branches"

Follow-up for the most recent version: "Time to git collaborating with git_remote_branch"

See the official git-svn manpage. In particular, look under "Basic Examples":

Tracking and contributing to an entire Subversion-managed project (complete with a trunk, tags and branches):

# Clone a repo (like git clone):
    git svn clone http://svn.foo.org/project -T trunk -b branches -t tags

As another aside, the git-stash command is a godsend when trying to git with git-svn dcommits.

A typical process:

  1. set up git repo
  2. do some work on different files
  3. decide to check some of the work in, using git
  4. decide to svn-dcommit
  5. get the dreaded "cannot commit with a dirty index" error.

The solution (requires git 1.5.3+):

git stash; git svn dcommit ; git stash apply

Here is a simple shell script with no dependencies that will convert one or more SVN repositories to git and push them to GitHub.

https://gist.github.com/NathanSweet/7327535

In about 30 lines of script it: clones using git SVN, creates a .gitignore file from SVN::ignore properties, pushes into a bare git repository, renames SVN trunk to master, converts SVN tags to git tags, and pushes it to GitHub while preserving the tags.

I went thru a lot of pain to move a dozen SVN repositories from Google Code to GitHub. It didn't help that I used Windows. Ruby was all kinds of broken on my old Debian box and getting it working on Windows was a joke. Other solutions failed to work with Cygwin paths. Even once I got something working, I couldn't figure out how to get the tags to show up on GitHub (the secret is --follow-tags).

In the end I cobbled together two short and simple scripts, linked above, and it works great. The solution does not need to be any more complicated than that!

I´m on a windows machine and made a small Batch to transfer a SVN repo with history (but without branches) to a GIT repo by just calling

transfer.bat http://svn.my.address/svn/myrepo/trunk https://git.my.address/orga/myrepo

Perhaps anybody can use it. It creates a TMP-folder checks out the SVN repo there with git and adds the new origin and pushes it... and deletes the folder again.

@echo off 
SET FROM=%1 
SET TO=%2 
SET TMP=tmp_%random%

echo from:  %FROM% 
echo to:    %TO% 
echo tmp:   %TMP%

pause

git svn clone  --no-metadata --authors-file=users.txt %FROM% %TMP%  
cd %TMP% 
git remote add origin %TO% 
git push --set-upstream origin master


cd .. 
echo delete %TMP% ... 
pause

rmdir /s /q %TMP%

You still need the users.txt with your user-mappings like

User1 = User One <u.1@xxx.com>

First, credit to the answer from @cmcginty. It was a great starting point for me, and much of what I'll post here borrowed heavily from it. However, the repos that I was moving have years of history which led to a few issues following that answer to the letter (hundreds of branches and tags that would need to be manually moved for one; read more later).

So after hours of searching and trial and error I was able to put together a script which allowed me to easily move several projects from SVN to GIT, and I've decided to share my findings here in case anyone else is in my shoes.

<tl;dr> Let's get started


First, create an 'Authors' file which will translate basic svn users to more complex git users. The easiest way to do this is using a command to extract all users from the svn repo you are going to move.

svn log -q | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u > authors-transform.txt

This will produce a file called authors-transform.txt with a line for each user that has made a change in the svn repo it was ran from.

someuser = someuser <someuser>

Update to include full name and email for git

someuser = Some User <someuser@somewhere.com>

Now start the clone using your authors file

git svn clone --stdlayout --no-metadata -r854:HEAD --authors-file=authors-transform.txt https://somesvnserver/somerepo/ temp
  • --stdlayout indicates that the svn repo follows the standard /trunk /branches /tags layout
  • --no-metadata tells git not to stamp metadata relating to the svn commits on each git commit. If this is not a one-way conversion remove this tag
  • -r854:HEAD only fetches history from revision 854 up. This is where I hit my first snag; the repo I was converting had a 'corrupted' commit at revision 853 so it would not clone. Using this parameter allows you to only clone part of the history.
  • temp is the name of the directory that will be created to initialize the new git repo

This step can take awhile, particularly on a large or old repo (roughly 18 hours for one of ours). You can also use that -r switch to only take a small history to see the clone, and fetch the rest later.

Move to the new directory

cd temp

Fetch any missing history if you only pulled partial in clone

git svn fetch

Tags are created as branches during cloning. If you only have a few you can convert them one at a time.

git 1.0.0 origin/tags/1.0.0

However, this is tedious if you have hundreds of tags, so the following script worked for me.

for brname in `git branch -r | grep tags | awk '{gsub(/^[^\/]+\//,"",$1); print $1}'`; do echo $brname; tname=${brname:5}; echo $tname; git tag $tname origin/tags/$tname; done

You also need to checkout all branches you want to keep

git checkout -b branchname origin/branches/branchname

And if you have a lot of branches as well, this script may help

for brname in `git branch -r | grep -v master | grep -v HEAD | grep -v trunk | grep -v tags | awk '{gsub(/^[^\/]+\//,"",$1); print $1}'`; do echo $brname; git checkout -b $brname origin/$brname; done

This will ignore the trunk branch, as it will already be checked out as master and save a step later deleting the duplicate branch, as well as ignoring the /tags that we already converted.

Now is a good time to take a look at the new repo and make sure you have a local branch or tag for anything you want to keep as remote branches will be dropped in a moment.

Ok, now lets clone everything we've checked out to a clean repo (named temp2 here)

cd ..
git clone temp temp2
cd temp2

Now we'll need to checkout all of the branches one more time before pushing them to their final remote, so follow your favorite method from above.

If you're following gitflow you can rename your working branch to develop.

git checkout -b WORKING
git branch -m develop
git push origin --delete WORKING
git push origin -u develop

Now, if everything looks good, you're ready to push to your git repository

git remote set-url origin https://somebitbucketserver/somerepo.git
git push -u origin --all
git push origin --tags

I did run into one final issue which was that Control Freak initially blocked me from pushing tags that I didn't create, so if your team uses Control Freak you may need to disable or adjust that setting for your initial push.

All in One - shell script for SVN to GIT Migration. Mention the GIT and SVN details with placeholder <>

#!/bin/bash

######## Project name 
PROJECT_NAME="Helloworld"
EMAIL="example mail"

#Credientials Repo
GIT_USER='<git username>'
GIT_PWD='<git password>'
SVN_USER='<svn username>'
SVN_PWD='<svn password>'

######## SVN repository to be migrated # Dont use https - error will be thrown
BASE_SVN="<SVN URL>/Helloworld"

#Organization inside BASE_SVN
BRANCHES="branches"
TAGS="tags"
TRUNK="trunk"

#Credientials
git config --global user.name '<git username>'
git config --global user.password '<git password>'
git config --global credential.helper 'cache --timeout=3600'

######## GIT repository to migrate - Ensure already project created in Git
GIT_URL=https://$GIT_USER:$GIT_PWD@<GIT URL>/Helloworld.git

###########################
#### Don't need to change from here
###########################

#Geral Configuration
ABSOLUTE_PATH=$(pwd)
TMP=$ABSOLUTE_PATH/$PROJECT_NAME

#Branchs Configuration
SVN_BRANCHES=$BASE_SVN/$BRANCHES
SVN_TAGS=$BASE_SVN/$TAGS
SVN_TRUNK=$BASE_SVN/$TRUNK

AUTHORS=$PROJECT_NAME"-authors.txt"

echo '[LOG] Starting migration of '$SVN_TRUNK
echo '[LOG] Using: '$(git --version)
echo '[LOG] Using: '$(svn --version | grep svn,)

mkdir $TMP
echo
echo '[DIR] cd' $TMP
cd $TMP

echo
echo '[LOG] Getting authors'
svn --username $SVN_USER --password $SVN_PWD log -q $BASE_SVN | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2"@"$EMAIL">"}' | sort -u >> $AUTHORS

echo
echo '[RUN] git svn clone --authors-file='$AUTHORS' --trunk='$TRUNK' --branches='$BRANCHES' --tags='$TAGS $BASE_SVN $TMP
git svn clone --authors-file=$AUTHORS --trunk=$TRUNK --branches=$BRANCHES --tags=$TAGS $BASE_SVN $TMP

#Not working so no need to mention it
#--stdlayout $PROJECT_NAME
echo
echo '[RUN] svn ls '$SVN_BRANCHES
svn ls $SVN_BRANCHES

echo 
echo 'git branch -a'
git branch -a

echo
echo '[LOG] Getting first revision'
FIRST_REVISION=$( svn log -r 1:HEAD --limit 1 $BASE_SVN | awk -F '|' '/^r/ {sub("^ ", "", $1); sub(" $", "", $1); print $1}' )

echo
echo '[RUN] git svn fetch -'$FIRST_REVISION':HEAD'
git svn fetch -$FIRST_REVISION:HEAD

#Branches and Tags  
echo
echo '[RUN] svn ls '$SVN_BRANCHES
for BRANCH in $(svn ls $SVN_BRANCHES); do
    echo git branch ${BRANCH%/} remotes/svn/${BRANCH%/}
    git branch ${BRANCH%/} remotes/svn/${BRANCH%/}
done

git for-each-ref --format="%(refname:short) %(objectname)" refs/remotes/origin/tags | grep -v "@" | cut -d / -f 3- |
while read ref
do
  echo git tag -a $ref -m 'import tag from svn'
  git tag -a $ref -m 'import tag from svn'
done

git for-each-ref --format="%(refname:short)" refs/remotes/origin/tags | cut -d / -f 1- |
while read ref
do
  git branch -rd $ref
done
  
echo
echo 'git tag'
git tag

echo
echo 'git show-ref --tags'
git show-ref --tags

echo
echo '[RUN] git remote add origin '$GIT_URL
git remote add origin $GIT_URL

echo
echo '[RUN] git push'
git push origin --all --force
git push origin --tags

#echo git branch -d -r trunk
#git branch -d -r trunk

git config --global credential.helper cache
echo 'Successful.'
  • When you run above script, it will fetch branches and tags details from SVN and put it under .git folder.
  • Crosscheck whether all branches are there in the SVN that should be available under this .git/refs/heads folder.
  • If some branches are missing which was there in SVN then do manually copy branches files from .git/refs/remotes/origin/<branches> to .git/refs/heads
  • Only copy branches (including master) and ignore if any tags or trunk.
  • Now run the script again. You could see all branches and tags in git repositories.
Related