Big Bucket Software you like to use
Relocating Subversion Externals
November 10th, 2007

So I recently bought a shiny new 24″ iMac. One of the first things I tried to do was to move my Subversion repository onto it. In Tiger I never did get around to running it in Apache, but this time around I was determined. As it happens, it was incredibly easy.

It wasn’t long before I hit my first snag – Externals Definitions. From the Subversion book:

An externals definition is a mapping of a local directory to the URL—and possibly a particular revision—of a versioned resource.

Commonly, an externals definition will be used to ensure that your project always pulls the most recent stable version of a third-party library. Less commonly though, and in my case, they are used for dependency management. For example, I have the TV Forecast project and the TV Countdown project. The common code between these two projects is in the TV project. So basically, my Subversion repository looks like this:

/TV Forecast/trunk/
/TV Countdown/trunk/
/TV/trunk/

The svn:externals property is applied to a directory. Given that my Repository is located at file:///Repository, the value of the svn:externals property applied to the /TV Forecast/trunk and /TV Countdown/trunk directories is:

TV file:///Repository/TV/trunk/

Therefore, when the trunk of TV Forecast or TV Countdown is checked out, it will check out the trunk of the TV project into a TV directory. I’ve found externals definitions to be very useful when managing releases as releasing is as simple as:

svn export 'file:///Repository/TV Forecast/trunk' 'TV Forecast'

That is, I don’t need to go around fetching dependencies.

Unfortunately, there is an enormous downside: externals definitions don’t relocate easily. Now that my iMac is hosting my Subversion Repository from Apache, the new Repository location is http://Peanut/svn. If I try to check out the head of TV Forecast, Subversion will bail telling me that the external cannot be found. Sure, I could just modify the property, but this will only affect the repository from the current revision onwards; It won’t help me to check out prior revisions. So what I’m going to need is a way to retrofit the new Repository location into all existing values of the svn:externals property. The solution? Hack the repository!

First things first, the repository needs to be serialized. Subversion allows for an entire repository to be serialized to and from a ‘dump file’ using two commands on svnadmin; dump and load. For example:

svnadmin dump OLD_REPO > dumpfileold.data

Magic

svnadmin load NEW_REPO < dumpfilenew.data

To begin, I found SvnDumpTool. This is a package of Python scripts designed to simplify hacking at dump files. It came very close to doing what I required, but not quite. The transform-prop command allows you specify a regular expression and a replacement string. This would almost do it, except that the svn:externals property can specify an arbitrary number of local directory/URL pairs. What I really needed was a simple ‘Find & Replace’ that operated within property values – so I added it.

This package contains the 0.4.0 release of SvnDumpTool patched to include the ‘replace-prop’ command. So in my case, as I’m trying to migrate my externals definitions from file:///Repository to http://Peanut.local/svn, I use the command:

svndumptool.py
    replace-prop
    svn:externals
    file:///Repository
    http://Peanut.local/svn
    dumpfileold.data
    dumpfilenew.data

You should then verify the changes with:

svndumptool.py diff dumpfileold.data dumpfilenew.data

Good luck.

  • So Perforce isn’t so bad then… Actually looking around for a food VCS that i can use throught the web and a proxy…

  • I love changing the past…

  • Cool… Thanks. If your patch works, I’ll be happy. I’m changing an svn:// to a slightly different http://. Same problem.

  • Looks like this feature is in 0.5.0… thanks for bringing it to my attention.

  • Tyler Mace — 2:19 am on 6.12.2008

    I had no trouble switching svn:externals definitions from “svn+ssh://” to “svn://”

    Maybe this is a unique side effect of not changing it to http. More likely, it’s a side affect of having more current software, SVN 1.5.

    Perforce isn’t so bad. But it’s not free, either.