Apache Subversion 1.8 Release Notes

What's New in Apache Subversion 1.8

Apache Subversion 1.8 is a superset of all previous Subversion releases, and is as of the time of its release considered the current "best" release. Any feature or bugfix in 1.0.x through 1.7.x is also in 1.8, but 1.8 contains features and bugfixes not present in any earlier release. The new features will eventually be documented in a 1.8 version of the free Subversion book (svnbook.red-bean.com).

This page describes only major changes. For a complete list of changes, see the 1.8 section of the CHANGES file.

Compatibility Concerns

Older clients and servers interoperate transparently with 1.8 servers and clients. However, some of the new 1.8 features may not be available unless both client and server are the latest version. There are also cases where a new feature will work but will run less efficiently if the client is new and the server old.

There is no need to dump and reload your repositories. Subversion 1.8 servers can read and write to repositories created by earlier versions. To upgrade an existing server installation, just install the newest libraries and binaries on top of the older ones.

Subversion 1.8 maintains API/ABI compatibility with earlier releases, by only adding new functions, never removing old ones. A program written to any previous 1.x API can both compile and run using 1.8 libraries. However, a program written for 1.8 cannot necessarily compile or run against older libraries.

There may be limited cases where the behavior of old APIs has been slightly modified from previous releases. These are cases where edge cases of the functionality have been deemed buggy, and therefore improved or removed. Please consult the API errata for more detailed information on what these APIs are and what impact these changes may have.

New Feature Compatibility Table

New Feature Minimum Client1 Minimum Server Minimum Repository Notes
working copy records moves 1.8 any any
Automatic reintegration merge 1.8 1.5 1.5
neon support removed 1.8 any any Server-side configuration changes might be required for optimal performance with 1.8 clients which access the repository via HTTP.
Inheritable properties 1.8 any any A 1.8 server isn't required but will deliver superior performance when asked for inherited properties.
Repos dictated config props 1.8 any any
Password caching with gpg-agent 1.8 any any
FSFS enhancements any 1.8 1.8 Repository size reduction requires dump/load.
Authz file in repository any 1.8 any
1Reminder: when using the file:// repository access method, the Subversion program is both the client and the server.

Upgrading the Working Copy

Subversion 1.8 introduces changes to the working copy format. In previous releases of Subversion (1.6 and earlier), Subversion would automatically upgrade the working copy to the new format when a write operation was performed. Subversion 1.8, however, requires an upgrade for both read and write operations on the working copy, and makes the upgrade a manual step.

Before using Subversion 1.8 with existing working copies, users will be required to run the svn upgrade command to upgrade working copy metadata to the new format. This command may take a while, and for some users, it may be more practical to simply checkout a new working copy.

Note: Subversion 1.8 cannot upgrade working copies that a 1.6 -or older- client would have refused to operate upon before an svn cleanup was run (with a 1.6 -or older- client). In other words, before upgrading to 1.8, an older client must be used to run svn cleanup on all working copies that require cleanup. Likewise, Subversion 1.8 cannot upgrade corrupt some 1.6 working copies. Unfixable problems can arise from missing or corrupt meta-data inside .svn directories. Such damage to the 1.6 working copy is permanent, and cannot be fixed even if svn cleanup is run prior to the upgrade.

If your working copy does not upgrade cleanly, please check out a new one.

Command Line Output Changes

Although we try hard to keep output from the command line programs compatible between releases, new information sometimes has to be added. This can break scripts that rely on the exact format of the output. For this reason, we encourage programs which consume the output of the commandline client to consider using the --xml option, or accessing Subversion through the various bindings interfaces.

Improved output of svn mergeinfo

The default output of svn mergeinfo has been changed. Instead of being equivalent to svn mergeinfo --show-revs=merged, it now shows a diagrammatic summary of some information about merges between the two specified branches:

$ svn mergeinfo ^/subversion/branches/moves-scan-log
    youngest common ancestor
    |         last full merge
    |         |        tip of branch
    |         |        |         repository path

    1186287            1445648 
    |                  |       
       --| |------------         subversion/branches/moves-scan-log
      /         /              
     /         /               
  -------| |------------         subversion/trunk
              |        |       
              1241413  1445640 
$ 

Scripts using svn mergeinfo without a --show-revs= option should be updated to add --show-revs=merged.

New options for svn proplist and svn propget

The default output for these commands remains the same, but both support the new option --show-inherited-props which may produce output changes from 1.7.x. See inheritable properties for more information.

Improved output and new options for svnlook proplist and svnlook propget

The output of svnlook proplist and svnlook proplist --verbose has changed and now mimics the output of svn proplist and svn proplist --verbose respectively. svnlook propget now supports the --verbose option, which produces output similar to svn propget --verbose.

Both svnlook proplist and svnlook propget now support the new option --show-inherited-props which may produce output changes from 1.7.x. See inheritable properties for more information.

svn status and svn info now show local moves

svn status now shows moves in its output. See the section about local moves for more information.

svn status shows an extra line for each item involved in a move:

$ svn move epsilon/zeta zeta-moved
$ svn status
D       epsilon/zeta
        > moved to zeta-moved
A  +    zeta-moved
        > moved from epsilon/zeta
$ 

svn info shows extra lines for locally moved items, too. For example, if the file beta was moved to beta-new, svn info beta will show the following (some unrelated output has been omitted in this example):

$ svn info beta      
Path: beta
[...]
Schedule: delete
Moved To: beta-new
[...]

svn info now shows repository-relative URLs

svn info now includes repository-relative URLs of items in the working copy in its output. The command line client has been accepting this URL notation since Subversion 1.6.
$ svn info README                                       
Path: README
Name: README
Working Copy Root Path: /tmp/svn-trunk
URL: https://svn.apache.org/repos/asf/subversion/trunk/README
Relative URL: ^/subversion/trunk/README
Repository Root: https://svn.apache.org/repos/asf
Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
Revision: 1467597
Node Kind: file
Schedule: normal
Last Changed Author: danielsh
Last Changed Rev: 1242804
Last Changed Date: 2012-02-10 15:58:53 +0100 (Fri, 10 Feb 2012)
Text Last Updated: 2012-09-20 01:33:22 +0200 (Thu, 20 Sep 2012)
Checksum: a27c71319a591c4eebe2bb81129413947336a7c6

HTTP client support based on neon has been removed

HTTP client support based on neon has been removed in favor of HTTP client support based on serf.

serf is a high-performance HTTP client library which has formed the basis of an alternative HTTP repository access method for many years. In an effort to decrease the development burden of maintaining multiple HTTP client interfaces (and sets of bugs!) serf is now the only HTTP client provider for Subversion.

Note that server-side configuration changes might be required to avoid performance regressions for serf clients in some setups.

The MaxKeepAliveRequests option in httpd.conf needs to be increased from 100 (the default) to at least 1000 (there is no reason why it could not be 10000). This will improve performance by allowing serf clients to use fewer TCP connections to the server. Clients using neon will also work fine with this configuration.

Skelta style updates are now the default

The svn 1.8 client with serf defaults to skelta mode for update operations (checkout, update, merge and export) instead of the bulk update mode used by previous versions. Skelta mode was introduced in Subversion 1.6.0 and improved in 1.8.0. It uses one HTTP request and response per resource that needs to be fetched from the server, whereas bulk update mode fetches all resources in one massive response.

The main benefit of skelta mode is that it allows a correctly set up Apache server or intermediate proxy server to cache mod_dav_svn’s responses to handle later requests from the cache. This results in improved performance of svn client operations and reduced CPU usage on the server side. It also allows a more detailed audit of clients accessing resources in a Subversion repository.

Skelta mode has some disadvantages:

  • Apache server access log files will grow more quickly due to the larger number of requests and responses. As of 1.7.3, the httpd error logs may also grow more rapidly with serf clients than with neon clients; see r1239697.
  • Network traffic can increase drastically when Kerberos or NTLM authentication is used, as these add a 2 - 4 KB header per HTTP request and response.

This release introduces two options to control if the svn client will use skelta or bulk update mode.

  • For the server administrator: The SVNAllowBulkUpdates directive for mod_dav_svn now accepts Prefer. This will advise the svn client to always use bulk update mode. All svn client versions with a default configuration (see table) will respect this preference.
  • For the user: Set the new option http-bulk-updates in the servers runtime configuration file to yes to force the use of bulk updates, no to never use bulk updates. The default option auto makes svn use skelta mode with a 1.8 server (unless it has SVNAllowBulkUpdates set to Prefer), and bulk updates mode with older servers and 1.8 servers which prefer bulk updates.

Table explaining the mode used between each combination of svn client and server version and relevant configuration directives:

1.8 Server
with SVNAllowBulkUpdates:
1.7 and older Server
with SVNAllowBulkUpdates:
Subversion Client Off On* Prefer Off On*
1.8, http-bulk-updates: auto* Skelta mode Skelta mode Bulk mode Skelta mode Bulk mode
1.8, http-bulk-updates: yes Skelta mode Bulk mode Bulk mode Skelta mode Bulk mode
1.8, http-bulk-updates: no Skelta mode Skelta mode Skelta mode Skelta mode Skelta mode
1.7 and older with neon* Skelta mode Bulk mode Bulk mode Skelta mode Bulk mode
1.7 and older with serf Skelta mode Skelta mode Skelta mode Skelta mode Skelta mode

*Default configuration

The Berkeley DB-based repository back-end has been deprecated

The Subversion developers have decided to deprecate the repository back-end based on Berkeley DB. We simply do not have the necessary resources to continue developing two distinct repository back-ends. Instead, we will concentrate our efforts on improving FSFS with new features, robustness and performance and architectural enhancements.

What this means:

  • New repository features that may appear in future versions of Subversion will not be implemented for the BDB back-end, nor will we make any effort to improve its performance.
  • We will fix bugs and security issues that may be found.
  • At some point, support for the BDB back-end will be completely removed. We will announce such removal well in advance of its happening.

What this does not mean:

  • Users do not have to immediately migrate their repositories to FSFS. The BDB back-end will continue to work, and will receive as much test coverage as it has until now.

This being a volunteer project, we are of course always happy to accept contributions towards maintaining the Berkeley DB back-end.

User-visible changes

The only visible effect of the deprecation is that the svnadmin utility will print a warning when it creates a new Berkeley DB-based repository.

Miscellaneous Compatibility Notes

There are some additional specific areas where changes made in this release might necessitate further adjustment by administrators or users. We'll cover those in this section.

FS paths syntax in authz access rules

Prior to this release, the section headers in Subversion's authz access files—which contain repository names and repository filesystem paths—accepted section headers that would never be looked up, because the repository filesystem path (such as /trunk/secretdir) embedded in the section header is formatted differently from how Subversion formats those paths when it looks them up in the file. Subversion 1.7 and earlier would silently ignore those sections of the authz file; directives in those sections would never apply.

That's been fixed in this release: Subversion will now error out if a section header contains a repository filesystem path that begins with a character other than slash (/), contains two consecutive slashes, or ends with a slash. The practical fallout, though, is that your existing authz files might be depending (perhaps unintentionally) on the old behavior and the new behavior could result in all access to items in your repositories being unexpectedly denied as a result of upgrading to Subversion 1.8. The svnauthz tool, when linked to Subversion 1.8 libraries, can be used to test an authz file for validity using the validate subcommand. (The tool will error out on a file that the Subversion server will error out on.)

Repository listing now honors authz configuration

When Apache is configured with the SVNParentPath directive, the "Collection of Repositories" list will now be filtered based on read access to the root of each repository. Previously, all repositories were included in the list even if navigating to a repository would be forbidden. The "Collection of Repositories" will now be consistent with the directory lists within repositories.

NOTE: Access to "Collection of Repositories" is not restricted by mod_authz_svn, but is instead managed by mod_dav_svn itself. In order to require authentication on this location, the location should have "Satisfy All" (which is the default value of this directive). See examples in mod_authz_svn's INSTALL document for additional details.

FSFS 'svnadmin verify' finds mergeinfo-count and predecessor-count discrepancies

Up to 1.7.0, Subversion could create erroneous metadata on nodes in a FSFS-backed repository in certain situations involving concurrent commit attempts. (Only metadata was affected; file contents and properties was not.) As of 1.7.1 Subversion prevents new instances of the corruption, but only as of 1.8.0 does 'svnadmin verify' detect instances of that corruption in the history of a repository.

The fix to these issues is simple: perform a dump/load cycle. (As usual, svnsync can be used instead of dump/load.) The cycle can be done with any version of Subversion, and after the cycle the repository should be served exclusively by Subversion 1.7.5 or newer to prevent further instances of the bug from entering the repository.

See this summary of issue #4129 for more information about these problems.

Client prompting for SSL client certificates

Subversion has long supported the use of SSL client certificates for authentication against a server which accepts such. Users typically employ the ssl-client-cert-file option in their 'servers' runtime configuration file to inform the client regarding the location of the relevant certificate file. In interactive scenarios, Subversion can also prompt the user for the location of the certificate file when that runtime-configured value isn't set or otherwise suitable.

Prior to 1.8.0, this prompting was enabled by default. Unfortunately, not every server which accepts client certificates also requires them. Some users were being prompted for client certificate file locations in scenarios where no certificate was required (or perhaps even available). So in Subversion 1.8.0, the Subversion client defaults to not prompting for the location of an SSL client certificate file unless the user has set the new ssl-client-cert-file-prompt runtime configuration option (found in the [auth] section of the 'config' file) to "yes".

See issue #2410 for discussion and details.

Star-imports in the SWIG-Python bindings

The swig-py bindings have been changed to make *-imports of submodules to do the right thing: from svn.client import * will now import only symbols beginning with svn_ or SVN_. (Most of these symbols will be svn_client_* symbols, of course.) Star-imports of from svn have not changed (they import just the bare submodule names: 'fs', 'ra', etc), and star imports of from svn.core currently imports some select symbols (notably 'Pool' and 'SubversionException').

This change is incompatible: code that expected symbols such as 'commit_txn' (in 'fs') and 'apr_initialize' to be pulled by a star-import will have to change.

svnserve --config-file behavior with password and authz dbs

The behavior of the --config-file option to svnserve has changed. The password db and authz db files will be reloaded on each connection. In past versions these files were cached on startup when --config-file was used.

The svnserve.conf file directly passed to --config-file will still be cached. Provided that the locations you wish to use for the authz and password dbs have not changed, you will not need to restart svnserve in order to have the changes you make to these files applied. This makes the behavior of --config-file more consistent with configurations that do not use this option.

If you were depending on the configuration changes not being applied until you restarted svnserve you will need to adjust accordingly.

svnauthz-validate renamed to svnauthz

The svnauthz-validate command has been renamed to svnauthz and now has a 'validate' subcommand. Meaning the equivalent to svnauthz-validate file in 1.8 is svnauthz validate file. To maintain command line compatibility, if the svnauthz command is run with the command name of svnauthz-validate then it emulates the behavior of the svnauthz-validate command from 1.7. make install-tools installs a symlink svnauthz-validate to provide this compatibility functionality.

svnauthz also provides new functionality. See this section for details.

New Features

Working copy records moves as first-class operation

The effect of the svn move command is now different from running svn copy followed by svn delete. Moves are represented within the working copy as a copy and a delete which are linked to one another. These links are shown by svn status and svn info.

Some Subversion operations will now treat locally moved files and directories specially. Behavioural changes include:

  • svn move now refuses to move a mixed-revision working copy.

  • svn commit will only commit a moved file or directory if both sides of the move are part of the same commit. This ensures that moves are an atomic operation upon future updates and merges.

  • svn update, svn switch, and svn resolve can now resolve tree conflicts involving locally moved files or directories. By picking the 'mine-conflict' option at the conflict prompt, the update or switch operation can be applied to the new location of the moved file or directory, which resolves the tree conflict and allows the move to be committed. It is also possible to break a move instead of updating it, in which case the move becomes a copy which is no longer linked to a deletion.

Limitations:

  • Moves are recorded as such only within the working copy. The link between the copy and the delete is established only when a local move operation is performed, and is lost upon commit. The committed revision will show a copy and a delete, instead of a move.

Known issues:

  • Tree conflicts involving replacements are currently not detected when updating a moved file or directory (see issue #4318).

  • Tree conflicts flagged by svn merge cannot be automatically resolved yet. This will be addressed in a future release.

Automatic reintegration merge (--reintegrate option deprecated)

During merges which merge all eligible revisions from another branch, Subversion 1.8 will automatically decide whether or not the merge is reintegrating a branch. Therefore, reintegrating a branch does no longer require the --reintegrate option for correct operation.

The --reintegrate option of svn merge is now deprecated and its use is discouraged. To reintegrate a branch, have a clean working copy of trunk and run the following command in its top-level directory:

$ svn merge ^/branches/my-branch

This merge will still perform similar sanity checks which svn merge --reintegrate performed in earlier releases:

  • The working copy must not be a mixed-revision working copy.
  • The working copy must not have switched subtrees.
  • There must be no gaps in revision ranges merged from the reintegration target (e.g. the trunk) to the reintegration source (i.e. the branch to be reintegrated).

If any of these conditions are detected, the merge is aborted and the necessary steps must be taken to fix the problem before the branch can be reintegrated. In contrast to a --reintegrate merge, an automatic reintegration merge into a working copy with local modifications is allowed.

Merging to-and-fro between two branches in any order is possible using the automatic reintegration merge (the "keep-alive dance" is no longer necessary). For best results, it is recommended to always merge all eligible revisions, i.e. not using the -r or -c options of svn merge. Merging just a subset of eligible revisions increases the likelihood of problems during future merges.

Using --reintegrate in Subversion 1.8 will force a reintegration merge, whether or not that's the right merge to perform in the given situation.

Inherited Properties

Property inheritance provides a mechanism to find the versioned properties set on the parents of a given working copy or URL path. Conversely it can be viewed as a mechanism by which a versioned property set on a path also applies to that path's children.

It is important to note that this change does not introduce any such thing as an "inheritable" property. Any versioned property, explicitly set on a path, can be interpreted as inheritable by that path's children. No changes have been made to the ways in which properties are set or what valid property names and values are permitted. Nor does this change grant access to parent paths which a user doesn't have read access to. If a user has no read access to a parent path, he cannot inherit properties from that parent.

Other than the changes detailed below, the only user visible changes resulting from inheritable properties are to the svn proplist, svn propget, svnlook proplist, and svnlook propget subcommands when used with the new --show-inherited-props option.

This new option finds the explicit properties set on a given path's parent(s) lists them prior to the explicit properties on the target path itself. For example:

   > svn propget -vR --show-inherited-props tsvn:logwidthmarker ^/subversion/trunk/subversion/po
   Inherited properties on 'https://svn.apache.org/repos/asf/subversion/trunk/subversion/po',
   from 'https://svn.apache.org/repos/asf/subversion/trunk':
     tsvn:logwidthmarker
       78
   Properties on 'https://svn.apache.org/repos/asf/subversion/trunk/subversion/po':
     tsvn:logwidthmarker
       80

If a working copy is the target, some properties may be inherited from the working copy and some may be inherited from repository parent paths not present in the working copy. These are shown as inherited from a URL:

   > svn propget -vR --show-inherited-props tsvn:logwidthmarker po-wc\de.po
   Inherited properties on 'po-wc\de.po',
   from 'https://svn.apache.org/repos/asf/subversion/trunk':
     tsvn:logwidthmarker
       78
   Inherited properties on 'po-wc\de.po',
   from 'po-wc':
     tsvn:logwidthmarker
       80

The output with the --xml option also returns inherited properties, wrapping them in inherited_property rather than property tags:

   > svn propget --show-inherited-props --xml tsvn:logwidthmarker ^/subversion/trunk/subversion/po
   <?xml version="1.0" encoding="UTF-8"?>
   <properties>
   <target
      path="https://svn.apache.org/repos/asf/subversion/trunk">
   <inherited_property
      name="tsvn:logwidthmarker">78</inherited_property>
   </target>
   <target
      path="https://svn.apache.org/repos/asf/subversion/trunk/subversion/po">
   <property
      name="tsvn:logwidthmarker">80</property>
   </target>
   </properties>

The svnlook proplist and svnlook propget subcommands also support the new --show-inherited-props option. The output is similar to svn proplist --show-inherited-props and svn propget --show-inherited-props except that the paths are not working copy paths or URLs, but absolute repository paths:

   > svnlook propget asf-repos-mirror tsvn:logwidthmarker /subversion/trunk/subversion/po \
       --show-inherited-props -v
   Inherited properties on '/subversion/trunk/subversion/po',
   from '/subversion/trunk':
     tsvn:logwidthmarker
       78
   Properties on '/subversion/trunk/subversion/po':
     tsvn:logwidthmarker
       80

Subversion working copies maintain a cache of the properties that the root of the working copy inherits from its parent(s) in the repository. This cache is updated each time you checkout, update, or switch a working copy. This cache allows the working copy to access properties inherited from the repository without contacting the repository.

For the full details about inheritable properties see the design wiki. Some of this wiki is intended for Subversion developers and will be of little interest to most users. If you fall into the latter group you can focus on these particular sections:

Repository Dictated Configuration (For auto-props and ignores)

Two new Subversion reserved properties, svn:auto-props and svn:global-ignores make use of the new inherited properties feature to provide additional configuration information that overrides/extends some of the settings found in the user's runtime configuration. The svn:auto-props property overrides/extends the auto-props configuration setting. The svn:global-ignores property extends the global-ignores configuration setting as well as the svn:ignore property.

The format of svn:auto-props property values is the same as for the auto-props runtime configuration. The format of svn:global-ignores property values is the same as for the svn:ignore property.

Both properties work just like their analogs in the runtime configuration with two exceptions:

  • The new properties only effect the subtrees rooted at the path on which the property is set. Thus a given path may be affected by multiple svn:auto-props or svn:global-ignores properties set on different parents of that path.
  • The svn:auto-props property is not tied to the enable-auto-props runtime configuration option. So even if you have enable-auto-props=no set in your configuration or via the --config-option option, svn:auto-props are still applicable. Note however, that exactly like the runtime configuration and svn:ignore property, both the svn:auto-props and svn:global-ignores properties can be disregarded with the --no-ignore and --no-auto-props options respectively.

When multiple svn:global-ignores properties apply to a path, then the different values are appended to any runtime global-ignores in effect and the value of any svn:ignore property that applies to the path.

Patterns defined in svn:auto-props property override any identical patterns in the auto-props runtime config. When multiple svn:auto-props properties apply to a file, the pattern from the nearest inheritable property takes precedence. See this section of design wiki for a full explanation.

The design wiki for the repository dictated configuration feature was originally written for developers, but will prove useful to any repository administrator who wants to use the feature.

In-memory password caching via GnuPG Agent (Unix client)

Subversion 1.8 allows the use of the GnuPG Agent (gpg-agent) daemon to temporarily store Subversion passwords in memory.

This feature does not use PGP to encrypt passwords on disk! Rather, it caches passwords in memory (in plaintext) instead of saving them to disk.

To take advantage of this password cache, you'll need Subversion binaries built with gpg-agent support (which is the default on UNIX-like systems), the gpg-agent itself, and a suitable pinentry program which the gpg-agent will use to ask the user for the password. The gpg-agent must be running, and the Subversion client will need the GPG_AGENT_INFO and GPG_TTY environment variables set up correctly. See this page for more information about running the gpg-agent.

Cached passwords will persist in memory until the agent process is terminated, its configured time-to-live threshold is reached, or a HUP signal is sent to the daemon using the UNIX kill(1) utility.

Communication to the gpg-agent happens over a UNIX socket, which is located in a directory which only the user running Subversion can access. However, any program the user runs could access this socket and get the Subversion password if the program knows the "cache ID" Subversion uses for the password.

The cache ID is very easy to obtain for programs running as the same user. Subversion uses the MD5 of the realmstring as cache ID, and these checksums are also used as filenames within ~/.subversion/auth/svn.simple. Unlike with GNOME Keyring or KDE Wallet, the user is not prompted for permission if another program attempts to access the password.

WARNING: Therefore, while the gpg-agent is running and has the password cached, the password cache is no more secure than a file storing the password in plaintext.

FSFS size and performance enhancements

Subversion 1.8 introduces a number of improvements that can reduce the size of FSFS repositories, the total number of files on disk, the I/O overhead and gives the user fine-grained control over all of that. Some of these changes require a format bump, others are backward-compatible.

FSFS format bump

The default repository format created by svnadmin create is now FSFS version 6 which is not accessible by Subversion 1.7 or older. Older repository formats remain fully supported by Subversion 1.8 but will not support revprop packing. To create FSFS repositories compatible with Subversion 1.6 and 1.7, use the --compatible-version 1.6 parameter.

You may use svnadmin upgrade to upgrade existing repositories. However, to fully benefit from the latest repository size reductions, it is recommended to create a new repository, adjust its settings and then dump/load or svnsync the contents into it.

WARNING: Server restart required! Replacing an existing repository with one rebuilt from a dump or restored from backup requires a server restart. This is true for any repository format. Alternatively, you may store the new repository in a different directory and redirect the Apache configuration to use that instead of the old one; svnserve does not offer that option.

Packing revision properties

Format 6 repositories will not only pack the revision files but also the revision property files. Upgrading existing packed repositories will automatically pack existing revision properties. Using default settings, this will reduce the number of revprop files to less than 10 per 1000 revisions in typical usage scenarios.

The db/fsfs.conf file allows you to fine-tune the revprop packing strategy. See the comments in that file for a detailed description of the available options. Please note that no change in this configuration file will affect any existing data. Moreover, upgrading repositories will not extend the existing configuration file, i.e. the server will use the default settings unless the new options are added explicitly.

It is recommended to also activate revprop caching when you use revprop packing. Otherwise, access to timestamps etc. will suffer significant processing overhead.

Caching revision properties

Many operations will access revision properties to e.g. retrieve the commit time stamp. Therefore, thousands of revision property files may need to be read during a checkout. The UI to enable the revprop cache is similar to that for the other caches: svnserve now accepts the --cache-revprops yes parameter. For mod_dav_svn, the same looks like this in httpd.conf

<IfModule dav_svn_module>
    SVNCacheRevProps on
</IfModule>

With revision property packing enabled, revprop caching allows you to read hundreds of revprops at once from a single file. This can give a significant performance boost to e.g. svn log .

Directory and property storage reduction

For each changed node in a FSFS repository, new versions of all parent directories will be created. Larger repositories tend to have relatively deep directory structures with at least one level (branches, modules or projects) having tens of entries. The total size of that data may well exceed the size of the actual change. Subversion 1.8 now supports directory deltification which eliminates most of that overhead.

In db/fsfs.conf, you may now enable and disable directory deltification at any time and these settings will be applied in new revisions. For completeness, node properties may now be deltified as well although the reductions in repository size will usually be minimal.

By default, directory and property deltification are disabled. You must edit db/fsfs.conf to enable these features. The reason is that deltification may amplify I/O in certain situations. To minimize that effect, enable the txdelta cache.

Also, db/fsfs.conf now allows for fine-grained control over how deltification will be applied. See the comments in that file for a detailed description of the individual options.

Another optimization in 1.8 is that node properties with the same contents will only be stored once and then merely referenced. This not only slightly reduces the size of the repository but greatly improves cache effectiveness and may reduce I/O significantly.

It's backward compatible! Please note that the mechanism to read deltified data is the same for file contents as for directories and properties. Hence, Subversion 1.6 and 1.7 will be able to read them but will always write new revisions without that optimization. It is therefore perfectly legal to create a pre-1.8-compatible repository in 1.8, to enable various deltification options, to dump/load into that new repository using 1.8 tooling and to finally use the repository with a 1.6 or 1.7 server.

Rep-sharing within revisions

When representation sharing has been enabled, Subversion 1.8 will now be able to detect files and properties with identical contents within the same revision and only store them once. This is a common situation when you for instance import a non-incremental dump file or when users apply the same change to multiple branches in a single commit.

In repository authz

Subversion 1.8 allows authz files to be stored inside a Subversion repository. This allows you to employ the versioning features of Subversion for the configuration of the path-based authorization feature. You need not store the authz file in the same repository as the one to which its rules are being applied. However, the server which uses the authz file does require filesystem access to the repository in which that file is stored. Administrators should consider that one benefit of having the authz file stored in the same repository as the one to which its rules are being applied allows the authz file to be replicated by svnsync along with the rest of the data in the repository, simplifying administration of mirrored repositories.

When specifying the location of the authz file to Apache HTTP Server or svnserve, there are now four formats an administrator may use:

  1. Absolute path to a file (outside of a repository): /path/to/file or C:\path\to\file
  2. Relative path to a file (outside of a repository): path/to/file or path\to\file
  3. Absolute URL to file in repository: file:///path/to/repo/file
  4. Relative URL to file in a repository: ^/file

The first two formats are those that were already supported in versions prior to 1.8; the latter two are the new formats.

The new absolute URL format is similar to what you might use with svn cat to list the contents of a file versioned in a local repository. (Note that at this time, support exists for file:// URLs only, not for other Subversion URL flavors such as http://, svn://, and so on.)

The relative URL syntax should also look familiar, as it mimics the relative URL syntax that the command-line client recognizes. When parsing path specifications in this format, Subversion simply ignores the leading ^/ and looks for authz file at the remaining path in the repository which is being accessed.

Apache HTTP Server accepts all four formats for both the AuthzSVNAccessFile and AuthzSVNReposRelativeAccessFile configuration directives. The only difference between these two directives is the root path for the "relative path to a file outside a repository" format.

WARNING: Unlike authz files stored on the server's local disk, authz files stored in the repository are accessible via Subversion clients just like any other file in the repository. If you wish to protect the contents of the authz file you should configure appropriate access restrictions for it in the applicable authz file (which could potentially be the same file!).

WARNING: As with other versioned files, Subversion servers do not validate the internal syntax of a versioned authz file in any way. Administrators may wish to set up a pre-commit hook script to validate that the authz file is well-formed and/or the committing user has not removed all permissions to edit the file. If permissions have been removed to edit it via the network server(s) you can of course always edit it via a local (file://) checkout since ra_local does not observe path based permissions. See the validate-files.py hook script and its related configuration files in Subversion's tools/hook-scripts for examples of how to validate a versioned authz file's syntax and specific permissions.

New tools and utilities

svn-bench benchmarking client

Identifying bottlenecks in your Subversion infrastructure may be difficult because client-side influences (virus scanners, slow disks etc.) tend to mask other limitations. Therefore, Subversion 1.8 introduces a very lightweight client that skips most of that local processing.

svn-bench provides commands similar to the standard command line interface, adding a null- prefix to them. Currently implemented are null-export, null-list and null-log. They support most of the parameters of their standard command counterparts.

$ time svn-bench null-export svn://localhost/tsvn_repos/trunk
            179 directories
          3,661 files
     84,902,674 bytes in files
         20,560 properties
        315,031 bytes in properties

real  0m0.185s
user  0m0.128s
sys   0m0.040s

By running the client close to or directly on the server machine, you will be able to determine the raw server speed and compare that to what the client-side is seeing.

fsfs-stats repository statistics

This server-side tool will read a whole format 4 (Subversion 1.6) or newer repository and print out various statistics on the contents of the rev and pack files in it. It starts of with very general information and then breaks it down into various sub-demographics. The statistics include:

  • Total repository size bytes
  • Number of revisions and changes
  • Sum of original and deltified file sizes
  • Sum of original and deltified directory sizes
  • Structural overhead due to node revisions, change lists etc.
  • Effectiveness of representation sharing
  • Paths and revisions of the 64 largest single changes
  • By file extension: Number of changes, total file size and total internal representation size
  • Histograms on the above

Administrators may use this information to decide whether directory deltification should be enabled (ratio of directory representation vs. file representation sizes) or whether commit policies might to be revised (e.g. large .zip files account for most of the repository). Some of the data, however, is useful for performance analysis by SVN developers only.

$ fsfs-stats /repositories/wesnoth_org 500 | gedit &

fsfs-stats takes a local path to the repository root and an optional cache size parameter. The latter is in MBytes and speeds up the analysis of large repositories. Because the tool's output may be up to a 1000 lines of text, you may want to redirect it to some file or application.

Note that the 32 bit version of this tool may fail on repositories with millions of changes or with revision / pack files larger than 4 GB. Use a 64 bit executable instead for those repositories.

Enhancements and Bugfixes

Improved conflict resolution

Conflicts are now always postponed during updates and merges

The svn update, svn switch, and svn merge commands line client now always postpone conflicts. Once the operation has completed, the interactive conflict resolver is automatically run on paths which received conflicts during the operation, which will either run in interactive mode or perform conflict resolution based on the value of the --accept option, if given.

This approach has the advantage that the connection to the server is not held open while interactive conflict resolution is performed. In Subversion 1.7, the connection could time out if the user spent too much time resolving a conflict (see issue #3846).

'svn resolve' can resolve conflicts interactively

The svn resolve command does not require the --accept option anymore. If invoked without any arguments, svn resolve will now perform interactive conflict resolution for any conflicted paths within the current working directory.

Built-in file merge tool for interactive conflict resolution

The command-line client now has a built-in file merge tool which merges any non-conflicting changes automatically and asks the user what to do about each individual merge conflict.

The built-in file merge tool can be invoked from the interactive conflict resolution prompt by choosing the (m) merge option. It displays conflicting file sections in a side-by-side view and offers various ways of resolving the conflict, as shown below:

$ svn update
Updating '.':
C    subversion/svn/file-merge.c
Updated to revision 1358165.
Summary of conflicts:
  Text conflicts: 1
Conflict discovered in file 'subversion/svn/file-merge.c'.
Select: (p) postpone, (df) diff-full, (e) edit, (m) merge,
        (mc) mine-conflict, (tc) theirs-conflict,
        (s) show all options: m
Merging 'subversion/svn/file-merge.c'.
Conflicting section found during merge.
(1) their version (at line 298)       |(2) your version (at line 391)        
--------------------------------------+--------------------------------------
  if (buf->len >= 2 &&                |  if (buf->len > 1)                   
      buf->data[buf->len - 2] == '\r' |    {                                 
      buf->data[buf->len - 1] == '\n')|      if (buf->data[buf->len - 2] == '
    svn_stringbuf_chop(buf, 2);       |        svn_stringbuf_chop(buf, 2);   
  else if (buf->len >= 1 &&           |    }                                 
           (buf->data[buf->len - 1] ==|  else if (buf->len > 0)              
            buf->data[buf->len - 1] ==|    {                                 
    svn_stringbuf_chop(buf, 1);       |      if (buf->data[buf->len - 1] == '
                                      |        svn_stringbuf_chop(buf, 1);   
                                      |    }                                 
--------------------------------------+--------------------------------------
Select: (1) use their version, (2) use your version,
        (e1) edit their version and use the result,
        (e2) edit your version and use the result,
        (eb) edit both versions and use the result,
        (p) postpone this conflicting section leaving conflict markers,
        (a) abort file merge and return to main menu: 

Checkout and update download pristine file data just once

Subversion keeps a cache of pristine copies of files in the working copy's meta data area inside the .svn directory. Before Subversion 1.8, pristine data was always downloaded from the server even if the same data already existed in the cache.

Subversion 1.8 avoids downloading pristine content that is already present in the cache, based on the content's SHA1 or MD5 checksum. This results in more efficient use of network throughput in cases where a lot of files in the working copy share the same content (for example, if a single working copy contains multiple branches).

Administration command-line tools improvements (server)

'svnadmin hotcopy' can now operate incrementally (FSFS only)

The svnadmin hotcopy command now supports incremental operation, triggered by the new --incremental option.

In prior releases of Subversion, svnadmin hotcopy refused to copy over an existing destination repository, and always copied the entire repository. For large repositories, performing a hotcopy could take several hours, preventing an efficient backup process.

In incremental hotcopy mode, revision data which has already been copied from the source to the destination repository will not be copied again. svnadmin hotcopy --incremental will only copy new revisions, and revisions which have changed in size or had their modification time stamp changed since the previous hotcopy operation.

Up to now, svnsync or svnadmin dump --incremental were the only alternatives for incremental repository backup. However, these commands need to perform additional processing while transforming revision data into an intermediate format before creating revision files in the destination repository. Performance of svnadmin hotcopy is only limited by disk I/O.

Incremental hotcopy is not supported for BDB repositories. See issue 4081 for more information.

Using general-purpose incremental backup tools, such as rsync, with Subversion repositories is dangerous if the repository can receive commits while the backup tool is running. The resulting copy might end up being corrupt because general-purpose backup tools do not know anything about consistency requirements of Subversion repositories. It is recommended to use incremental hotcopy instead. Alternatively, if network support is required for the backup procedure, svnsync should be used. There is also a new svnadmin freeze command which allows third-party tools to access a live repository safely.

'svnadmin load' now supports the --revision option

svnadmin load now supports the --revision (-r) option, which causes it to only load revisions from the dump stream within the specified revision number range.

New 'svnadmin lock' and 'svnadmin unlock' commands

The new svnadmin lock and svnadmin unlock commands can be used to lock and unlock paths in the repository directly, without requiring a working copy for use with svn lock or svn unlock.

New 'svnadmin freeze' command

The new svnadmin freeze command can be used to run an arbitrary program while preventing concurrent commits to the repository. The program is invoked by svnadmin with a provided parameter list. This can be used to safely use third-party backup tools on a live repository:

$ svnadmin freeze /svn/my-repos -- rsync -av /svn/my-repos /backup/my-repos

The above will obtain a write lock on the my-repos repository to prevent concurrent commits, and run rsync to back up the repository. Clients trying to commit concurrently will wait until the write lock becomes available again. If the command takes too much time clients trying to commit might time out. It is recommended to run commands which complete very quickly, such as a command to trigger creation of a filesystem snapshot.

'svndumpfilter' now supports dump files with deltas

svndumpfilter can now filter dump files containing deltas, as generated by svnadmin dump --deltas.

svnauthz can print user's access to a path in a repository

svnauthz (formerly known as svnauthz-validate) has an 'accessof' subcommand that can print or test what the permissions would be in a given circumstance, allowing you to validate that your changes have effected the permissions that you intended.

svnauthz accessof supports --username, --repository, and --path arguments. When these are passed, it prints no, r, or rw depending on what access the given username has: none, read-only, or read-write.

'svnlook diff' improvements

svnlook diff now supports the following new options (analogous to svn diff):

  • --ignore-properties suppresses output of property changes
  • --properties-only shows only property changes
  • --diff-cmd ARG use ARG as diff command

Support for high-speed networks in 'svnserve'

If your server has a 10 Gbit/s or faster network connection, the multi-stage data copying from Subversion's caches to the network stack plus frequent status checks become a bottleneck. Today, most clients won't be able to generate enough aggregated traffic to make this an issue and an 8-core server should be able to send about 40Gb/s at 100% CPU load. Future clients may be able, however, to process 1Gb or even 10Gb per second and to create significant server loads.

When the server is given the --client-speed N parameter, it will assume that most clients are able to process data at rates of N Mbit/s; N being a non-negative integer. Future server releases may apply different strategies for different ranges of client bandwidth to optimize the network performance. In 1.8 and if N>=1000, the server will take various shortcuts to reduce internal processing overhead. On the downside, a very slow or hanging client may degrade the server performance for other clients as well. Setting N to values above 100.000 is legal but will often result in a net throughput loss.

Note: This option will have little effect in 1.8 without configuring large fulltext caches.

Command-line client improvements (client)

There are far too many enhancements and new options to the command-line client to list them all here. Aside from all the ones mentioned already in these release notes, below are a few more that we consider important, but please see the 1.8.0 section in the CHANGES file for a complete list.

'svn commit' can now recurse into externals

The svn commit command supports a new --include-externals option, which causes it to commit changes within externals in the current working copy, in addition to the changes in the current working copy. This works by implicitly adding all externals within the commit target to the list of commit targets.

'svn diff' improvements

The svn diff command can compare arbitrary files with one another, using the invocation:

$ svn diff --old foo.c --new bar.c 
Index: bar.c
===================================================================
--- bar.c       (.../foo.c)     (working copy)
+++ bar.c       (.../bar.c)     (working copy)
@@ -1,6 +1,6 @@
 #include <stdio.h>
 int main()
 {
-       printf("Hello world!\n");
+       printf("Good bye world!\n");
        return 0;
 }

This works with any combination of versioned and unversioned files. Comparing arbitrary directories is also supported, in which case the directories are walked recursively and files within them are compared.

svn diff also supports the following new options:

  • --ignore-properties suppresses output of property changes
  • --properties-only shows only property changes
  • --patch-compatible produces output compatible with third party patch programs

svn diff --summarize is extended to match all scenarios that svn diff supports, including the new arbitrary file diff.

Custom keyword definitions

Subversion now supports the definition of custom keywords, which can be defined using a special syntax within the svn:keywords property.

For example, MyKeyword=%r%_%a%_%P defines a keyword called "MyKeyword" which contains the number of the last-changed revision, the author of that revision, and the file's path in the repository, each separated by one space character. Once a custom keyword has been defined it can be used within the file like any other keyword: $MyKeyword$

The svn help propset command lists the known format codes which can be used in custom keyword definitions, which are also shown in the table below. Any characters in the keyword definition which are not format codes are expanded literally, i.e. a keyword such as MyAuthor=Author:%_%a will expand to Author: joe if the last-changed revision's author is "joe".

%aThe author of the revision given by %r.
%bThe basename of the URL of the file.
%dShort format of the date of the revision given by %r.
%DLong format of the date of the revision given by %r.
%PThe file's path, relative to the repository root.
%rThe number of the revision which last changed the file.
%RThe URL to the root of the repository.
%uThe URL of the file.
%_A space (keyword definitions cannot contain a literal space).
%%A literal '%'.
%HEquivalent to %P%_%r%_%d%_%a.
%IEquivalent to %b%_%r%_%d%_%a.

Note that keyword expansions which exceed 255 bytes in length are truncated, and keywords with names that exceed 255 bytes are not expanded.

API changes, improvements and language bindings (client and server)

There are too many new and revised APIs in Subversion 1.8.0 to list them all here. See the Subversion API Documentation page for general API information. If you develop a 3rd-party client application that uses Subversion APIs, you should probably look at the header files for the interfaces you use and see what's changed.

A small number of APIs have slightly changed functionality from their historical roots. Each of these cases has been documented as an errata, detailing the impact of these changes. All of the errata were the result of behavior which was deemed buggy, and the API changes are an attempt to fix these bugs. The circumstances under which each is triggered are relatively rare, and we anticipate the actual impact to end users will be minimal.

Language bindings have mostly been updated for the new APIs, though some may lag more than others.

Bug fixes (client and server)

A great many bugs have been fixed. See the 1.8.0 section in the CHANGES file for details.

Merge-Tracking Enhancements

While there are scores of bug fixes, performance improvements, and other changes to merge-tracking, the following are the major changes. See the 1.8.0 section in the CHANGES file for a complete list.

'svn mergeinfo' now honors the -r option

The svn mergeinfo command now honors the -r option. While the option has always been accepted and documented, it was silently ignored, such that svn mergeinfo always operated on the entire history of the repository.

In Subversion 1.8, the -r option restricts the output of svn mergeinfo to the specified revision range, which is useful when determining whether or not a given revision, or range of revisions, is eligible for merging, or whether it has already been merged. In the following example, r682012 has not been merged from trunk yet, while r26112011 has already been merged:

  $ svn mergeinfo --show-revs=eligible -r 682012 ^/trunk
  r682012
  $ svn mergeinfo --show-revs=merged -r 26112011 ^/trunk
  r26112011
  $

Hook scripts improvements (server)

Hook script environment variables are now configurable

In Subversion 1.7 and earlier, hook scripts always run in an empty environment. If environment variables are needed hook scripts had to configure them manually. Subversion 1.8 supports configuration of hook script environment variables on a per-server or per-repository basis.

The hook script environment is configured via a new configuration file located either at conf/hooks-env in a repository, or at a server-wide location specified in svnserve.conf (using the new hooks-env option) or httpd.conf (using the new SVNHooksEnv option).

The hook script environment configuration file uses a format similar to that of other Subversion configuration files:

### This file is an example hook script environment configuration file.
### Hook scripts run in an empty environment by default.
### As shown below each section defines environment variables for a
### particular hook script. The [default] section defines environment
### variables for all hook scripts, unless overridden by a hook-specific
### section.

### This example configures a UTF-8 locale for all hook scripts, so that 
### special characters, such as umlauts, may be printed to stderr.
### If UTF-8 is used with a mod_dav_svn server, the SVNUseUTF8 option must
### also be set to 'yes' in httpd.conf.
### With svnserve, the LANG environment variable of the svnserve process
### must be set to the same value as given here.
[default]
# LANG = en_US.UTF-8

### This sets the PATH environment variable for the pre-commit hook.
# [pre-commit]
# PATH = /usr/local/bin:/usr/bin:/usr/sbin

mod_dav_svn now supports hook script UTF-8 input/output

As documented in issue #2487, hook scripts run on a mod_dav_svn-based Subversion server previously had problems with non-ASCII input and output. Because Apache HTTPD modules always run in the "C" locale, which is restricted to ASCII, mod_dav_svn usually failed trying to process non-ASCII characters.

Among other problems, this prevented pre-lock and post-unlock hook scripts from operating on paths containing non-ASCII characters, and prevented non-ASCII error output from start-commit and pre-commit hooks from reaching SVN clients.

Subversion 1.8 addresses these problems with a new SVNUseUTF8 option for mod_dav_svn. If this option is set, mod_dav_svn assumes that all hook script input/output is encoded in UTF-8. This is always the case for paths passed to hook scripts, since Subversion uses UTF-8 internally for all paths. If this option is set, hook scripts must write error messages to stderr in UTF-8. Because ASCII is a subset of UTF-8 existing hook scripts will continue to work unmodified.

For best results, hook scripts should use a UTF-8 locale if the SVNUseUTF8 option is active. It is recommended to configure a UTF-8 locale using the hook script environment configuration file.

start-commit hook invocation point changed

Prior to this release, the start-commit hook script was invoked by Subversion just prior to the creation of the commit transaction. An errorful return from the hook would be marshaled back to the client without the commit transaction having ever been created. A successful return would permit the creation of the commit transaction, and allow the commit process to proceed as expected. Administrators generally employed the start-commit hook to immediately deny all commits to a repository, as the amount of information available to the hook by which to do so conditionally was pretty slim.

Beginning with Subversion 1.8, the start-commit hook is invoked immediately after Subversion creates the commit transaction and sets its initial properties. This allows start-commit implementations to examine the commit transaction's properties (such as the revision log message and the new ephemeral transaction properties) when deciding whether or not to allow the commit to proceed. Subversion will abort the created transaction upon detecting an errorful return from the hook and will, of course, still marshal that error condition back to the client.

This change empowers administrators to disallow certain commits via the start-commit hook rather than doing so much later — after the client's changes are transmitted across the network — in the pre-commit hook. Imagine the frustration of a user who has endured the lengthy upload of 2 Gb of commit data only to have his or her commit bounced because of an insufficiently formatted log message! These changes to start-commit allow the go/no-go decision on such a commit to be made before that large commit payload is transmitted across the wire.

Note: Subversion does not require that commit transaction properties (such as the revision log message) be attached to the transaction as part of its initialization. As such, some clients will still not provide that information to the server until after the start-commit hook has been invoked. Here is a list of such clients we are aware of:

  • Pre-1.8 clients communicating via HTTP
  • Clients communicating via HTTP when mod_dav_svn's "SVNAdvertiseV2Protocol" option has been set to "off"

Administrators should consider modularizing the tests that their hooks perform on transaction properties, invoke those tests from both the start-commit and pre-commit hook scripts.

post-commit hook grows txn-name argument

The post-commit hook has grown a txn-name argument as the third positional argument (argv[3]). This argument is the name of the transaction that has become the revision triggering the post-commit hook; it is the same as the second positional argument to the pre-commit, and thus enables the pre-commit and post-commit hooks to easily transfer state between them.

Generically, if the pre-commit hook makes an expensive computation against the would-be-revision, it can store the result of that computation in a persistent hash keyed by the transaction name, which the post-commit hook will use to avoid re-doing the computation. One concrete usage of this would be to implement a pre-commit hook that tests compilability only, while a post-commit hook or a continuous integration tool runs more thorough tests --- on the binaries built at pre-commit, without having to rebuild them itself.

Commit transactions now carry client information

Subversion 1.8 clients communicating to 1.8 servers (via http[s]: or svn:) will now attach "ephemeral transaction properties" to commit transactions which carry metadata considered useful to server-side hook scripts. These special properties are set on the commit transaction as early as possible (generally prior to the invocation of the start-commit hook, but certainly prior to the invocation of the pre-commit hook), and then automatically removed by Subversion just prior to the commit transaction being promoted to a new revision.

Currently, the following ephemeral transaction properties are available to hook scripts for examination:

  • svn:txn-user-agent - carries the "user agent" string which describes the committing client program. (Developers of third-party clients which link against the Subversion libraries may modify the customizable portion of this string by implementing the get_client_string() RA callback API which was introduced in Subversion 1.5.)
  • svn:txn-client-compat-version - carries the Subversion library version string with which the connecting client claims compatibility. (For Subversion clients which use the Apache Subversion C API and libraries, this version is precisely the version of those libraries.)

Note: Administrators who deem the information carried in these properties to be of long-term value may of course log the information as they see fit. For example, such an administrator may wish to use a pre-commit hook script to copy the values of those properties to new, custom transaction properties (which do not have names within the reserved svn: namespace, of course!) for long-term storage associated with the commit. An example hook script that renames the revision properties from svn:txn-* to svn:revision-* is distributed with Subversion itself.

New svnpubsub framework for pushing post-commit notifications (server)

svnpubsub is a daemon that pushes post-commit notifications to clients in streaming JSON over HTTP. The notifications are typically pushed to the daemon by a post-commit hook. The code lives in the tools/server-side/svnpubsub directory of the source tree.

A typical notification looks like this (newlines added for readability):

% curl -i http://svn.foo.org:2069/commits
{"commit": {
   "type": "svn",
   "format": 1,
   "id": 1334845,
   "repository": "13f79535-47bb-0310-9956-ffa450edef68",
   "changed": {"httpd/site/trunk/content/dev/": {"flags": " U "}},
   "committer": "joes",
   "log": "spacing",
   "date": "2012-05-07 00:30:25 +0000 (Mon, 07 May 2012)"}}
^@
...

svnpubsub was written by Paul Querna for the ASF Infrastructure Team.

New SVNMasterVersion Apache HTTP Server configuration directive

Subversion's HTTP protocol continues to mature and expand over time to support new features or better ways to accomplish things. Fortunately, Subversion servers and clients happily negotiate which bits of that protocol they both understand. Unfortunately, WebDAV proxy configurations can throw this negotiation for a loop, as the slave server does the negotiating but not necessarily all the operating. Thus, a Subversion 1.7 WebDAV proxy slave will by default tell the client that it's okay to use the HTTPv2 protocol, and a Subversion 1.7 or better client will do so, even if the master server behind the proxy is running Subversion 1.6 and unable to understand the new protocol.

When Subversion 1.7 was released, this specific problem was addressed by providing an Apache HTTP Server configuration directive — SVNAdvertiseV2Protocol — which allowed administrators in such a scenario to "dumb down" the communications of the Subversion 1.7 slave server so that connecting clients would use the older protocol and, in doing so, not confuse the Subversion 1.6 master server when requests were proxied to it. As development continued toward 1.8, though, there where several new features and protocol changes made which would likewise require a toggle switch to avoid confusing pre-1.8 WebDAV proxy master servers.

Rather than exposing a suite of such toggles — some of which were for behaviors that are entirely specific to the protocol itself and not tied to any user-visible feature — Subversion 1.8 allows administrators to simply declare to the slave servers the version of Subversion which is powering the master server, allowing the slave servers to enable and disable features as makes sense for that scenario. Administrators of Subversion deployments which use the WebDAV write-through proxy feature are strongly encouraged to use the new SVNMasterVersion httpd.conf directive in all slave server configurations running Subversion 1.8 or better.

For example, if your master server is running Subversion 1.7.7, your Subversion 1.8 slave server's Apache HTTP Server configuration might contain something like this:

<Location /svn>
DAV svn
…
SVNMasterURI http://master.svn.company.com/svn
SVNMasterVersion 1.7.7
…
</Location>

In the general case, the new SVNMasterVersion directive supercedes the need for SVNAdvertiseV2Protocol. By declaring the version of the master server, the slave will assume the default level of feature support for that server. However, if you have a master server which can support the HTTPv2 protocol but for which you've disabled this intentionally, you will need to also explicitly disable the feature (again, via the SVNAdvertiseV2Protocol off directive) in your slave servers as well.

Keyword expansion via DAV repository browser

One of the earliest features of the mod_dav_svn Subversion server was the ability for users to browse their repositories directly with a web browser. GET requests issued by browser and aimed at Subversion directory URLs return a directory listing; those aimed at file URLs return the file's current contents.

In Subversion 1.6, we expanded that feature to allow browsing of historical artifacts in your repository via new p=PEGREV and r=REV CGI parameters.

Now in Subversion 1.8, we've added still another CGI parameter: kw=1.

http://host/repos/path?kw=1

Users who include kw=1 in the query string portion of GET requests aimed at a versioned file will now be delivered that file's contents with any Subversion keywords expanded therein. Because keyword substitution is typically performed by the Subversion client as part of its working copy administration and management, this is handy way to get the server to deliver a keyword-expanded form of your versioned file without the use of a working copy. Omitting the kw parameter or using something other than "1" for its value will cause Subversion to use its default behavior, which is to deliver the file's contents without any keywords expanded. (NOTE: We reserve the right to recognize new meaningful values for the kw flag in the future, so please avoid specifying any value other than "1" when using the parameter at all.)

As with client-side keyword expansion, only those keywords which are explicitly designated for expansion via the svn:keywords property set on the file itself will be expanded.

Exclusive SQLite locking of working copies

In the root of every Subversion working copy there is an SQLite database in the .svn directory and Subversion clients use this database when accessing the working copy. By default Subversion uses shared SQLite locking and this allows simultaneous access to the working copy by multiple clients with exclusive locks only being held for short periods. The 1.8 command line client can be configured to use exclusive SQLite locking by setting the following in the .subversion/config file:

[working-copy]
exclusive-locking-clients = svn

or by using the command line option --config-option config:working-copy:exclusive-locking-clients=svn.

This reduces the locking overhead but does mean that only one Subversion client will be able to access the working copy at a time. A second client attempting to access a locked working copy will block for 10 seconds and then get an error. In most cases shared locking is preferred but if the working copy is on a network disk rather than a local disk the locking overhead is more significant. When dealing with large working copies on network disks exclusive locking may give a significant performance gain, two or three times faster in some cases.

Known issues in the release

There are some known issues in the Subversion 1.8 releases. These may be fixed in later 1.8.x releases.

1.8.0 and chunked Transfer-Encoding

Subversion 1.8.0 has switched from neon to serf for HTTP access. The Serf-based HTTP access library would use chunked transfer encoding for most requests. When the mod_dav_svn Subversion server is fronted by a proxy or reverse proxy (such as Nginx prior to 1.3.9) which responds with a 411 Length Required, Subversion would sometimes either treat this as a fatal error, such as:

% svn commit <arguments>
POST of '/svn/[project name]/!svn/me': 411 Length Required

or

svn: E175009: XML parsing failed: (411 Length Required)

and sometimes would fail to notice the failure of the request and issue an unrelated error, such as:

svn: E175004: The PROPFIND response did not include the requested properties

Subversion 1.8.1 fixes this issue by detecting automatically whether chunked requests are supported at the set up of a session. This is done by issuing one additional request at the start of the session.

Technical details: A session is often equivalent to calling one svn subcommand (or one svn_client_* function, for API users). However several subcommands currently open multiple sessions. The extra request cost is incurred for each session. A single session may make numerous TCP connections (usually 2 and never more than 8) and the extra request will only be made once for all the connections in a session.

The net effect is that the cost of the additional request may be incurred multiple times, and thus be more noticeable. For example svn log --diff ends up opening 2 sessions for every revision in order to request the diff.

We expect that the default configuration of Subversion 1.8.1+ will work well out of the box for most users. A http-chunked-requests option has been added to the ~/.subversion/servers (Windows: %APPDATA%\Subversion\servers file. Users who wish to avoid the additional request may set that option to yes or no in order to short-circuit the additional request and avoid making it. We recommend not to set that option (or to set it to its default value, auto) unless you have special circumstances which require it (such as an unusually high latency), and even then to set it only in the config sections of specific groups and not in the [global] section. See this dev@ list post for an example.

mod_dav_svn Maps Requests to Local Filesystem

Prior to Subversion 1.7.12 and 1.8.2 all previous versions of mod_dav_svn would allow Apache httpd to map requests handled by mod_dav_svn to the local file system. When this happened httpd would believe that it was serving a file or directory located at the path that comprises the URI path appended to the DocumentRoot of the server but ending at the first path component that does not actually exist on the local disk. For example if your DocumentRoot was /var/www and you were accessing /svn/repo/README and /var/www/svn did not exist then httpd would believe it was serving /var/www/svn, but if /var/www/svn existed while /var/www/svn/repo did not then it would believe it was serving /var/www/svn/repo. This had several undesired side effects.

  • <Directory> blocks could match requests being served by mod_dav_svn. This would allow directives applied in these configuration blocks (most notably access control directives) to be applied.
  • When using Apache httpd 2.4.x and mod_dir, if a file from the DirectoryIndex directive existed at the path Apache httpd believed it was serving then the request would be rewritten to access that file, which may or may not exist. If the path was the SVNParentPath then SVN would try to access a repository with the name of the DirectoryIndex file that was found.
  • When using the %f variable in your log directives you would log the path constructed as described above. This was confusing since it was not the content that was actually served.

Starting with Subversion 1.7.12 and 1.8.2 the requests being served by mod_dav_svn will not be mapped to the local file system and when logging with the %f variable you will see a "-" logged. <Directory> blocks will no longer apply to such requests. Subversion 1.7.14 and 1.8.5 changed this further and the filename for the request will now be set as dav_svn:/path/to/repo/path/in/repo (e.g. if your repo is at /var/svn/repo and the request is accessing /trunk/myfile in the repo then the filename for the request will be set to dav_svn:/var/svn/repo/trunk/myfile. Note that the dav_svn: prefix will prevent the filename from matching an actual path on disk since it isn't a valid path and as such <Directory> blocks will still not apply.

WARNING:Therefore, if you were using <Directory> blocks to restrict access to paths served by mod_dav_svn you will need to adjust your configuration. Since the local filesystem path was not complete this never would have been very useful for path based authorization, but it could be misused for simple things like blocking an IP. These directives must be moved inside the appropriate <Location> blocks.

WARNING:This change may cause checksum errors with misconfigured httpd servers. Since 1.7.14 and 1.8.5 the filename in the request record now includes the filename in the repository, there have been cases of mod_mime interfering with Subversion. Users will see a svn: E200014: Checksum mismatch error when working with files that end in .gz if a configuration setting like AddEncoding x-gzip .gz is in their configuration file and applies to the locations that Subversion is being served from. This happens because the setting now matches on the Subversion traffic and sets the Content-Encoding header to specify that the file has been gziped for transport without changing the file content. Since the Subversion client supports gzip content-encoding it decompresses the file and ends up with the decompressed file which does not match the checksum of the file it expected. It is possible that this change may cause other filters to match Subversion requests that would not have in the past and trigger other similar errors with other httpd configurations.

Subversion is unable to store SHA1 collisions

Subversion up to and including 1.8.17 can incorrectly store files with different content but the same SHA1 checksum. We recommend that all servers update to 1.8.18 and enable representation sharing. See our SHA1 advisory for details.

Subversion 1.6.x series no longer supported

The Subversion 1.6.x line is no longer supported. This doesn't mean that your 1.6 installation is doomed; if it works well and is all you need, that's fine. "No longer supported" just means we've stopped accepting bug reports against 1.6.x versions, and will not make any more 1.6.x bugfix releases.