Piloting Git in a Subversion Environment

We’re looking to implement a pilot of Git to familiarize the team with the tool and introduce them to the more flexible workflow that Git supports over SVN. Because we support concurrent lines of development, we would like to forego the usual way of using git with SVN (each developer clones a branch of the repository and commits there) and try to implement a pure git experience using an intermediate “project repository”. The idea behind this repository is as follows:

  • We clone the main subversion repository using git svn
  • An empty project repository is created on the server and the Subversion clone is pushed to this repository (its an additional remote).
  • The development team clones the project repository and proceeds to do development in their own clones. Periodic pushes to and pulls from the project repository keeps each developers master branch up to date, from which they can rebase their local branches.
  • Periodically (probably daily), the originating SVN clone is ‘svn rebased’ and the new changes pushed to the project repository. These changes will be pulled by the developers on the next pull from the project repository.

In picture form, the workflow looks something like this:

Envisioned Pilot Process

Nevermind – StackOverflow ruined it for me.

Feeling Bad

I’m kind of feeling bad for not blogging lately. The site is not getting much of my attention over the last few months. I’m playing a lot with git, a version control tool and figuring out how it plays with SVN, the SCM we use at work. I’m most interested in the advantages over SVK for distributed development while still having a central repository. Expect some posts on that soon.

Aside from that it is camping season, so I’m trying to spend most of my weekends away from the computer.

Hopefully, the bookmark trail is giving some good data on what I am looking at lately. If you really want to know whats going on, follow me on Twitter. It’s where I spend most of my blogging time lately.

Subversion, MediaWiki, WordPress, and LDAP

One of the biggest arguments you’ll get in deploying open source software in a corporate environment perception that they are extra, standalone applications. If your corporation uses an LDAP server, you can get some big wins by ensuring that your open source applications can authenticate with your corporate LDAP store, showing integration with the main systems.

I recently went through this exercise with a number of applications in our environment, including:

  • Subversion
  • MediaWiki
  • WordPress

I thought I’d throw up an entry here outlining the tools I used to make this integration possible.

Subversion was a no-brainer, since we host our repositories using mod_dav_svn. Configuring the mod_auth_ldap module in the Apache server and converting all access to SSL made this integration painless, once I figured out how to build Apache to use OpenLDAP and Secure LDAP. For MediaWiki, the Mediawiki LDAP Extension worked flawlessly. The key problem with Mediawiki is that there is no mechanism built in to ensure that logins are performed via SSL. A quick rewrite rule in the Apache server took care of this for me. A complete explanation of this process can be found at Library Web Chic.

For WordPress, I found a great plugin from Kane IT Consulting that was extremely easy to configure. I had the plugin installed and configured in minutes. I highly recommend this one. The Admin-SSL plugin, gave us the security around the login that we needed.

What has been interesting to me is seeing the subtle shift in perception of these applications as we integrated them into the authentication system. They almost seem like legitimate pieces of the system now … even to me.

Subversion and SSL Troubles

I decided to upgrade my home Subversion repository to version 1.4.3 as soon as it was released. Since then, my ViewVC application has ceased to work, getting a Python exception every time I try to execute it. Creating a small Python program that just imports the library (from svn import fs) gave me the following error:

ImportError: /usr/local/lib/libsvn_ra_dav-1.so.0: undefined symbol: SSL_load_error_strings

Thinking it was an SSL library problem, I upgraded SSL – a few times. I kept mucking with the options, rebuilding Subversion, only to get everything installed and get that same error:

ImportError: /usr/local/lib/libsvn_ra_dav-1.so.0: undefined symbol: SSL_load_error_strings

Over, and over and over again I repeated the process and got the same result. The absolute definition of insanity. This has been going on for a couple of months and I’ve been trying to address it in my spare time, as I’ve been pretty busy lately during the week and gone to the Relaxation Unit the last few weekends.

I googled my ass off to find the error, but to no avail. Finally today I ran across this thread that explained the problem. After going through my distribution directory for 1.4.4 (which I upgraded at the beginning of the month only to receive the same error) I realized that I hadn’t pulled down the Subversion dependencies tarball and rebuilt neon. So, basically I was using an old version of the neon libraries.

I finally settled on the configure statement listed here, after downloading and untarring the deps file:

./configure --with-ssl --with-apxs=/usr/local/apache2/bin/apxs 
            --with-apr=/usr/local/apache2 --with-apr-util=/usr/local/apache2 
            --enable-shared --with-libs=/usr/local/ssl

This uses the already installed apr libraries that I build with my Apache server, and ensures that the neon shared libraries are built. A quick configure/make/make install/make swig-py/make install-swig-py sequence later and my Python libraries were working fine.

I made it a point this time to document this on the Labs internal wiki, but thought I should throw this out here in public so that others can find it. Hope it helps save the weeks of frustration that I have been suffering for someone out there.

Happy building …

Video: How Open Source Projects Survive Poisonous People (And You Can Too)

Since getting a 80G iPod about a month ago two weeks ago, I’ve been really getting into watching the Google Tech Talks on Google Video. I recently watched How Open Source Projects Survive Poisonous People (And You Can Too), a lecture given by Brian Fitzpatrick and Ben Collins-Sussman from the Subversion team (now both Google employees) that summarizes a lot of information in Karl Fogels book Producing Open Source Software: How to Run a Successful Free Software Project.

If you haven’t had time to pick up and read Karls book, this video would be a good primer to some of the concepts in it and could very well motivate you to pick it up. Its an excellent book and one that I thoroughly enjoyed reading.

Practical Subversion – Second Edition

I received a free copy of Practical Subversion, Second Edition by Daniel Berlin and Garrett Rooney on Friday from their publishers, Apress.

I had reviewed the first edition before it was released and had found it to be an excellent companion to “Version Control with Subversion” (C. Michael Pilato, Ben Collins-Sussman, Brian W. Fitzpatrick), mostly due to its coverage of the Subversion API’s – which I had not seen covered in any real depth in any other book.

I have to say, the authors have outdone themselves with the Second Edition. The book is extremely well written for varying levels of Subversion experience. The beginner will find a very easy to understand introduction to using Subversion in the first two chapters, giving a really great tutorial on how to use the tool along with explanations of many of the concepts embodied in the implementation of the tool, such as locking vs. non-locking systems, properties (from file to revision properties), the basic workflow involved in using version control, and how to use the various commands, from checking out, to using svn blame (or ‘praise’ as I learned from the book is an alias for the command) to find the origin of a change in the system.

Thats just the first two chapters. As the book goes on the reader will learn about repository administration, the differences between the BDB and FSFS file systems, using Apache and Apache modules to squeeze additional functionality into the system, migrating from other version control systems such as CVS and Perforce and third party tools that work with Subversion (such as ViewVC, emacs, etc). The book also covers maintaining vendor branches, and contains a very good chapter on Version Control Best Practices. You then have, from my memory anyway, a greatly expanded chapter on using the Subversion API.

Practical Subversion, Second Edition does a really good job of covering information at many skill levels in an extremely accessible way. This book will definitely be one of those that I would put on the shelf at work as we continue to move people into more advanced roles in the management of our repositories – as there’s really nothing the book doesn’t cover.

I’ve been a user of Subversion for a very long time (I think I started around version 0.19 or so) and as I perused the book last night I walked away with some new distinctions about how we were using the tool and changes I could make to make administration and maintenance easier. That says a lot.

Congratulations to Garrett and Daniel on a fine piece of work. Hopefully the next edition will cover some of the newer features of 1.4, specifically the svnsync tool.

Building Subversion on The Mac and using Ecto for Blogging

I finally upgraded my Subversion installation on my MacIntosh to the 1.4 version. I was waiting for the “official” packages to come out so that I could just install it, but in looking at the different places recommended by the downloads page, these distributions haven’t been updated since early 1.3 releases.

I’ve had a goal to keep my Mac somewhat pristine. I decided thats not really practical. There are a lot of things that I use that I just like having built from scratch, so that I’m on the most current software and not dependent on someone else’s schedule. Subversion is one of those tools.

One thing I was shocked at was how quickly and seamlessly the build happened on the Mac. These MacBooks are pretty fast machines. I think it took a total of roughly 20 minutes (if that) to build, run tests, and install. The build on the Mac is definitely the fastest configure/check/install cycle I’ve gone through in the many installations of Subversion that I have performed over the years.

I tell you, the more I’m on the Mac, the more I like it. I haven’t run into anything that I’ve found irritating.

Its all good.

This is also the first post I am writing using a trial version of Ecto. I have to say, this application is pretty impressive too. They have both a Mac and a Windows version. I like it much better than blogging in WordPress directly – and at $17.95 a copy, its practically a no brainer to purchase.

Don’t get me wrong, I’m going to milk the 21 day trial, but it feels like this application is a pretty good fit for me.