This article is mirrored with permission from the original location http://blogs.collab.net/subversion/foreign-repository-copies. Inactive links have been removed or updated.

Author: C. Michael Pilato

Posted: 2013-06-28

Some time ago, I wrote in this space about some new functionality that I had managed to work into Subversion — the ability to merge changes into a working copy from a so-called “foreign repository” (that is, a different repository than the one reflected by the working copy).  I touted the feature as a lovely alternative way to manage vendor branches.  I admit that I may have glossed over the details a bit in that post, but the idea seemed simple enough:  import a baseline copy of the third-party codebase into your repository, and thereafter merge the changes made by the third-party directly from their repository into your local baseline copy.  Voilà!  Vendor branches.

Recognizing the potential of the foreign repository merge functionality, I’ve since begun using this approach myself.  I have a number of CollabNet-related and personal projects which make use of third-party code.  CollabNet’s TeamForge and Subversion Edge products both use ViewVC, for example.  In many of those situations, there are small customizations that need to be made to the third-party codebase.  Vendor branches are the natural way to manage those changes.

But as I began to put the theory into practice, I ran into an unexpected problem with that first branch initialization step:  properties.  svn import doesn’t recognize or replicate any Subversion file or directory properties which might be recorded on the import source.  Therefore, when I would use svn import to instantiate my vendor branch with a baseline version of the third-party code, I would get a hybrid mixture of information.  Sure, I had the vendor’s file contents and directory structure perfectly replicated in my working copy.  But the metadata — node properties such as svn:ignore, svn:eol-style, svn:keywords, and so on — was not coming from the vendor’s original data source.  It was coming instead from my own local auto-props configuration.

Notice how in the following (somewhat contrived) example svn import effectively loses the svn:ignore property which was present in the third-party library:

$ svn co http://otherfolks.com/svn/trunk third-party-wc
A    third-party-wc/src
A    third-party-wc/src/source_file.c
A    third-party-wc/src/source_file.h
A    third-party-wc/notes
A    third-party-wc/notes/how_to_do_stuff.txt
A    third-party-wc/README
Checked out revision 61.
$ svn plist -vR third-party-wc/
Properties on 'third-party-wc/src':
  svn:ignore
    a.out
$ svn import -m "Initialize vendor branch." \
      third-party-wc http://localhost/svn/my-project/vendor
Skipped 'third-party-wc/.svn'
Adding         third-party-wc/src
Adding         third-party-wc/src/source_file.c
Adding         third-party-wc/src/source_file.h
Adding         third-party-wc/notes
Adding         third-party-wc/notes/how_to_do_stuff.txt
Adding         third-party-wc/README

Committed revision 13.
$ svn co http://localhost/svn/my-project/vendor vendor-wc
A    vendor-wc/src
A    vendor-wc/src/source_file.c
A    vendor-wc/src/source_file.h
A    vendor-wc/notes
A    vendor-wc/notes/how_to_do_stuff.txt
A    vendor-wc/Makefile
A    vendor-wc/README
Checked out revision 13.
$ svn plist -vR vendor-wc
$

Oops! We’re missing an svn:ignore property from the vendor!

As a result, the initial import of the vendor data would go just fine, but then later — while performing a foreign repository merge which included property changes — I’d end up running into property conflicts.  The merge would say, “Change property NAME‘s value from OLD_VALUE to NEW_VALUE.”  My working copy would frown and reply, “But NAME‘s value isn’t OLD_VALUE at all.  It’s SOMETHING_ELSE.”  Boom.  Conflict.

$ svn merge -r61:62 http://otherfolks.com/svn/trunk vendor-wc
Conflict for property 'svn:ignore' discovered on '/home/cmpilato/vendor-wc/src'.
They want to change the property value to 'program', you want to delete the property.
Select: (p) postpone,
        (mf) mine-full, (tf) theirs-full,
        (s) show all options: p
--- Merging (from foreign repository) r62 into 'vendor-wc':
 C   vendor-wc/src
A    vendor-wc/Makefile
Summary of conflicts:
  Property conflicts: 1
$

Ugh.

In this example, the resolution of the conflict is easy enough.  But that’s not the point.  Clearly, I needed a better way to initialize my vendor branches.  I ended up doing things the hard way for a while.  Rather than use svn import, I would use svn add, a slew of svn propset invocations to manually mimic the source data’s property set, and then svn commit.  It was cumbersome, but it worked.  In the meantime, I filed a feature request for something better in the Apache Subversion project’s issue tracker.

I’m happy to report now that, thanks to the massive rewrite of Subversion’s working copy management library which was the cornerstone of Subversion 1.7.0, and with the specific additional efforts of my CollabNet peer Bert Huijben, Subversion 1.8.0 provides support for foreign repository copies.  Initializing a vendor branch from a foreign repository now is as simple as running svn copy with the foreign repository URL as the source, and a local working copy directory as the target.

$ svn cp http://otherfolks.com/svn/trunk my-project-wc/vendor
--- Copying from foreign repository URL 'http://otherfolks.com/svn/trunk':
A         my-project-wc/vendor
A         my-project-wc/vendor/src
A         my-project-wc/vendor/src/source_file.c
A         my-project-wc/vendor/src/source_file.h
A         my-project-wc/vendor/notes
A         my-project-wc/vendor/notes/how_to_do_stuff.txt
A         my-project-wc/vendor/README
$ svn ci -m "Initialize third-party vendor branch." my-project-wc
Adding         my-project-wc/vendor
Adding         my-project-wc/vendor/README
Adding         my-project-wc/vendor/notes
Adding         my-project-wc/vendor/notes/how_to_do_stuff.txt
Adding         my-project-wc/vendor/src
Adding         my-project-wc/vendor/src/source_file.c
Adding         my-project-wc/vendor/src/source_file.h
Transmitting file data ....
Committed revision 13.
$ svn plist -vR my-project-wc/vendor
Properties on 'my-project-wc/vendor/src':
  svn:ignore
    a.out
$

This time, our vendor’s svn:ignore property was preserved!  This means that later, when it’s time to merge the vendor’s recent changes, the merge should run smoothly (so long as we haven’t introduced conflicting customizations, of course):

$ svn merge -r61:62 http://otherfolks.com/svn/trunk my-project-wc/vendor
--- Merging (from foreign repository) r62 into 'my-project-wc/vendor':
 U   my-project-wc/vendor/src
A    my-project-wc/vendor/Makefile
$

Cha-ching!

For the sake of completeness, I should note that there is one exception to the perfect preservation of the foreign repository’s data that occurs during the copy: any svn:mergeinfo property found therein is filtered out of the replica created in your local working.  Doing so avoids a host of problems that might occur were Subversion to think that the merge tracking information applied to your own repository’s history.  But it all turns out okay.  Since foreign repository merges likewise ignore the svn:mergeinfo property, this exception won’t prevent you from using such merges to maintain your vendor branch!

Also, while the svn copy command syntax allows for the source and target of the copy to each be either a repository URL or a working copy path, it’s only the svn copy REPOS-URL WC-PATH syntax which supports foreign repository copies.  You can’t, for example, copy directly from a URL in one repository to a URL target in another repository.

For more details about using this approach to manage vendor branches, check out my freshly re-written section of the Version Control With Subversion book.  And for more information on what Subversion 1.8 offers, see the official release notes.

(And thanks, Bert!)