Tag Archives: source-control

Practical Subversion – Second Edition

Posted on by
Reply

I received a free copy of Practical Subversion, Second Edition by Daniel Berlin and Garrett Rooney on Friday from their publishers, Apress.

I had reviewed the first edition before it was released and had found it to be an excellent companion to “Version Control with Subversion” (C. Michael Pilato, Ben Collins-Sussman, Brian W. Fitzpatrick), mostly due to its coverage of the Subversion API’s – which I had not seen covered in any real depth in any other book.

I have to say, the authors have outdone themselves with the Second Edition. The book is extremely well written for varying levels of Subversion experience. The beginner will find a very easy to understand introduction to using Subversion in the first two chapters, giving a really great tutorial on how to use the tool along with explanations of many of the concepts embodied in the implementation of the tool, such as locking vs. non-locking systems, properties (from file to revision properties), the basic workflow involved in using version control, and how to use the various commands, from checking out, to using svn blame (or ‘praise’ as I learned from the book is an alias for the command) to find the origin of a change in the system.

Thats just the first two chapters. As the book goes on the reader will learn about repository administration, the differences between the BDB and FSFS file systems, using Apache and Apache modules to squeeze additional functionality into the system, migrating from other version control systems such as CVS and Perforce and third party tools that work with Subversion (such as ViewVC, emacs, etc). The book also covers maintaining vendor branches, and contains a very good chapter on Version Control Best Practices. You then have, from my memory anyway, a greatly expanded chapter on using the Subversion API.

Practical Subversion, Second Edition does a really good job of covering information at many skill levels in an extremely accessible way. This book will definitely be one of those that I would put on the shelf at work as we continue to move people into more advanced roles in the management of our repositories – as there’s really nothing the book doesn’t cover.

I’ve been a user of Subversion for a very long time (I think I started around version 0.19 or so) and as I perused the book last night I walked away with some new distinctions about how we were using the tool and changes I could make to make administration and maintenance easier. That says a lot.

Congratulations to Garrett and Daniel on a fine piece of work. Hopefully the next edition will cover some of the newer features of 1.4, specifically the svnsync tool.

Upgrading Subversion to 1.4

Posted on by
Subversion Logo

This weekend I finally bit the bullet and upgraded our production Subversion server to 1.4. The upgrade was painless, after the usual running of autogen.sh on the Neon libraries that for some reason are necessary when building on the Solaris 9 environment. I also had some weird test failures that wound up being caused by hook scripts being called on the test repositories. For some reason, Python couldn’t find one of the standard libraries, even though I had set my path and library path in my environment. A quick Google search later and all tests passed.

We have been running Subversion with the Berkeley 2.2 back-end since May of 2004, when the Subversion team first released 1.0. We’ve done a few upgrades since then, when I found the time, but wound up getting as far as 1.2.3 and my time to actually do the upgrades never really materialized. Part of that had to do with some pretty large projects that were going on that I didn’t want to interrupt the teams with an upgrade.

So yesterday, finally, I did the upgrade to 1.4. As part of the upgrade, I decided to dump and load the repositories (11 of them) to take advantage of the new SVN Diff format that 1.4 brings to the table. As part of that, I also decided to abandon the Berkeley back-end in favor of the FSFS file system.

The dumps and loads took quite a long time, about sixteen hours. However, the results were worth it. After all was said and done, we saw a reduction in server disk space usage of about 46.8%.

Not bad guys!

As I had mentioned in a previous post, one of the motivators for this upgrade were the working copy improvements, which according to the release notes, should cut down on inode usage on machines running the client. In our environment, we have about 20 people all working on the same machine (in their home directories) and we have had problems with inode max-outs in the past. Again, I’ll have to reserve judgement on these changes until we see the results, but this is one of the things that motivated me to bite the bullet and take a Saturday to do the upgrades.

The other big motivation for me, especially after yesterday was this statement from the FSFS document referenced above:

An FSFS repository can be backed up with standard backup software.
Since old revision files don’t change, incremental backups with
standard backup software are efficient. (See “Note: Backups” for
caveats.)

(BDB repositories can be backed up using “svnadmin hotcopy” and can be
backed up incrementally using “svnadmin dump”. FSFS just makes it
easier.)

Aside from normal system backups done nightly by the admin team, our nightly backups have also consisted of doing nightly dumps of all of the repositories so that, in the event that the repository server goes down, we can just load the repository on another machine (you can never have too many backups – in my opinion). The wake up call from yesterday is that it takes much longer to reload a repository on Solaris 9 than it does on the Linux work-group server I have sitting here that I did all of my original testing on a few years ago. What I thought would be a trivial load (3 hours was the estimate that I had in my head) wound up being 5-6 times that on the 2×2 Solaris box at work. It will be nice to be able to rely on the nightly system backups for these now — and perhaps just a quick tarring of the existing repository structure for quick backups (using hotcopy), rather than the dump / load process we have planned to use up until now. I’ve just never been comfortable for some reason with the Berkeley backend and backups outside of repository dumps.

Finally, I’m really looking forward to playing around with svnsync to set up a shadow fail-over repository in the event of an outage. This was something I threw on a wish list back in 2004, and I’m excited to see a tool to synchronize a “stand by repository” included in this release. There apparently is not much documentation on this new feature yet, but this article should be a good starting point.

Thanks to the Subversion team for another great release of their software. As anyone who reads this blog with any regularity knows, this is one of those pieces of software that I continuously receive value from and cannot say enough about.

Now I’m off to search for more svnsync documentation …

Subversion 1.4 Released

Posted on by

The Subversion team has released version 1.4 of its popular version control software. You can check out the release notes over at the official site get the the details, but here’s a summary of the changes, pulled directly from the aforementioned release notes:

  • svnsync, a new repository mirroring tool
  • Huge working-copy performance improvements
  • Support for BerkeleyDB 4.4 and its “auto recovery” feature
  • Size improvements to the binary delta algorithm
  • A handful of new command switches
  • Many improved APIs
  • More than 40 new bugfixes

I was going to post about this yesterday, but I wanted to make sure I had the software built and installed here at the Labs before throwing the link out there. The upgrade went relatively well. Since I still use Berkeley DB for some of the repositories here, I build my own software to minimize the amount of dumping / loading I have to do. Currently I am still running Berkely DB 4.2. Referencing this during the build allowed me to avoid some problems people have reported using pre-packaged distributions that upgrade Berkeley DB for you, rendering your repository useless. Building my own also allows me to get the software without waiting for the binary distributions to become available.

One note on the upgrade. I’ve been a little lax in upgrading my Apache server (also custom built) and was running version 2.0.48 or so. The new release requires an up to date version of the apr libraries, so this also forced me to upgrade Apache to 2.0.59. Overall, the upgrade was painless.

As mentioned above, this release also includes the svnsync tool, which is a repository mirroring tool. From what I’ve read so far, the destination mirror must remain read only – there is no synchronization between two duplicate repositories (at least from the limited reading material I’ve found around so far), so this release by no means invalidates the SVK tool. Nevertheless, the working copy improvements and the mirroring capability shows that the team is still on the right track.

Also noted in the release notes:

… the new working copy format allows the client to more quickly search a working copy, detect file modifications, manage property metadata, and deal with large files. The overall disk footprint is smaller as well, with fewer inodes being used. Additionally, a number of long standing bugs related to merging and copying have been fixed.

I’m going to reserve judgement on these improvements until I get the Solaris boxes at work upgraded. The working copies are really an Achilles heel on Solaris environments, where 20 or so developers use one machine to do all development. We’ve had a number of inode-maxouts over the last year or so. When I get these machines upgraded, I’ll post a follow up on the performance on Solaris.

One other enhancement I’m glad to see, the diff and merge commands now support a -c option which you can use to merge one revision between branches. This allows you to avoid using a revision range for a simple one revision merge. This should simplify things a bit …

Subversion is still, overall, the best version control tool I’ve worked with thus far (and I’ve worked with quite a few of them). Kudos to the team on the new release. I like what I see so far …

SVK Aquired by Best Practical

Posted on by

I’m a little behind on the mailing lists, but I found that on June 5 it was announced that Best Practical Solutions, LLC has aquired SVK. SVK is a distributed version control system written on top of Subversion and provides the ability to mirror Subversion and CVS repositories to your local machine in order to work in a distributed fashion. It also provides advanced branching and merging capabilities far and above what the core Subversion product provides. Read the official announcement here.

Best Practical, LLC produces Open Source incident response, request tracking, and FAQ management software. Chia-liang Kao, the author of SVK, has joined Best Practical as a partner.

Will The Build Bottleneck Put The Brakes On Agile?

Posted on by

There is a great article in the SD Times called Will The Build Bottleneck Put The Brakes On Agile? which talks about the use of both version control and an automated build process in which to receive feedback on the “health of the software” more frequently.

Our group has used Open Source products for this kind of thing for a few years (I think we started using CruiseControl at version 1.0, as Keith and I tend to be those “high risk”, early adopter types) with excellent results. The ability to have the software you are developing automatically build from source to deployment packages (we have both the traditional tarballs and ISO images for a Windows product) is an excellent way to remove a ton of unproductive manual labor from your processes and get builds to QA.

One of the great features of this kind of system that isn’t really publicized in the article is the ability, upon build failure, to notify the committers to the current build that a build failure has occurred. This allows a proactive response to these kinds of failures rather than the typical “one guy watches the build and hunts down the guy who broke it” process that most companies normally go through when one person handles build duties.

The other nice thing about this? It makes it really easy to rotate “buildmaster” duties throughout the department. In our case, we built a system around the open source tools in order to help people manage the build, and utilized functionality in CruiseControl like the JMX interface in order to allow people to kick builds without having physical access to the server. This allows us to control what is going on with the server, but still allow anyone with sufficient role based permissions to access the system in order to manage the day to day activities around a build.

We have also implemented CodeStriker, an online code review tool in order to try and keep people out of meeting rooms and give them the ability to comment on source code at the line level, per project, from wherever they happen to be at the time. This has been another really great addition to our process and has reduced the amount of paper each person had to fill out during a code review. We now have documentation of all items raised during the reviews, who raised them, and what the status is of each. The tool also provides full integration with CVS and Subversion.

I’m glad an article like this has come out. These sorts of automated processes need to become the norm in order for IT to be able to get out of the rut it has been in for the last few years in which the manual effort required to produce a software release prevents it from delivering on the things the business needs. Read it. Think about it. Then give it a try.

Here’s some software resources to get you started:

Keith, did I leave anything out?

Subversion 1.2.1 released.

Posted on by

The Subversion team has released version 1.2.1 of their version control system.

The following changes are included in this release:

User-visible-changes:
– Client:
* fixed: ‘svn lock’ on switched file locks wrong thing (issue #2307)
* fixed: ‘svn (un)lock’ errors on multiple targets (r14736, 14775)
* fixed: ‘svn (un)lock’ problems with URI-unsafe names (issue #2314)
* fixed: ‘svn (un)lock’ not caching authentication (r15088)
* fixed: ‘svn unlock’ loses executable bit (r14859, r14923, r14939)
* fixed: ‘svn unlock URL’ segfault (r14893)
* fixed: ‘svn commit’ failure on XML-unsafe locked paths (issue #2335)
* fixed: recursive directory copy bug (issue #2343)
* fixed: don’t initialize RA library in ‘svnversion’ (r14755)
* fixed: svn-push segfault (r14732)
* various translation updates for localized client messages

– Server:
* fixed: ‘svn log’ performance regression, general (r14116, 14772, 14759)
* fixed: ‘svn log -v’ performance regression, FSFS-specific (r15016)
* fixed: mod_dav_svn bug sets content-type incorrectly (r15046)

Developer-visible-changes:
* fixed: win32 innosetup’s add/repair/remove features (r14830)
* fixed: OBOE with ‘limit’ parameter of svn_repos_get_logs3(). (r15119)
* redhat RPM fixes (r15050)
* perl bindings:
– accessors for svn_lock_t (r15082)
– call utf_initialize, adjust global pool usage (r15076, r15080,
r15081, r15117)

You can download the software at one of the following links:

http://subversion.tigris.org/downloads/subversion-1.2.1.tar.gz
http://subversion.tigris.org/downloads/subversion-1.2.1.tar.bz2
http://subversion.tigris.org/downloads/subversion-1.2.1.zip

SVK 1.0 Beta 2 released.

Posted on by

Yesterday, the release of SVK 1.0 Beta 2 was announced .

The announcements list the following changes since Beta 1:

  • Fix keyword translation for undesired characters. [matthewd]
  • Fix svk switch from a removed branch.
  • Don’t trust LML would return a valid encoding that Encode knows. [Eric Gillespie ]
  • Recognize merge-conflict error, so no stacktrace for it.
  • svk info now skips files not under version control. [#8220] [gugod]
  • Fix locked mirror messages.
  • Various pool usage fixes.

The software can be downloaded from the SVK download area.