Subversion and SSL Troubles

I decided to upgrade my home Subversion repository to version 1.4.3 as soon as it was released. Since then, my ViewVC application has ceased to work, getting a Python exception every time I try to execute it. Creating a small Python program that just imports the library (from svn import fs) gave me the following error:

ImportError: /usr/local/lib/libsvn_ra_dav-1.so.0: undefined symbol: SSL_load_error_strings

Thinking it was an SSL library problem, I upgraded SSL – a few times. I kept mucking with the options, rebuilding Subversion, only to get everything installed and get that same error:

ImportError: /usr/local/lib/libsvn_ra_dav-1.so.0: undefined symbol: SSL_load_error_strings

Over, and over and over again I repeated the process and got the same result. The absolute definition of insanity. This has been going on for a couple of months and I’ve been trying to address it in my spare time, as I’ve been pretty busy lately during the week and gone to the Relaxation Unit the last few weekends.

I googled my ass off to find the error, but to no avail. Finally today I ran across this thread that explained the problem. After going through my distribution directory for 1.4.4 (which I upgraded at the beginning of the month only to receive the same error) I realized that I hadn’t pulled down the Subversion dependencies tarball and rebuilt neon. So, basically I was using an old version of the neon libraries.

I finally settled on the configure statement listed here, after downloading and untarring the deps file:

./configure --with-ssl --with-apxs=/usr/local/apache2/bin/apxs 
            --with-apr=/usr/local/apache2 --with-apr-util=/usr/local/apache2 
            --enable-shared --with-libs=/usr/local/ssl

This uses the already installed apr libraries that I build with my Apache server, and ensures that the neon shared libraries are built. A quick configure/make/make install/make swig-py/make install-swig-py sequence later and my Python libraries were working fine.

I made it a point this time to document this on the Labs internal wiki, but thought I should throw this out here in public so that others can find it. Hope it helps save the weeks of frustration that I have been suffering for someone out there.

Happy building …

Metrics As A Side Effect

Yesterday morning I found an article by a former employee who has recently introduced Scrum to his organization called Beef Up Your Scrum-Master Toolbox up on the Devx site. Since Doug started his blog and then subsequently left the company, I’ve enjoyed keeping up with his new adventures and I went right to the DevX site to find out what he had been doing lately.

The article is great, outlining the way that his team is accruing metrics for their Scrum teams using an Excel spreadsheet. However, one thing kept nagging at me the whole time I was reading it. The little voice in my head kept saying:

God, thats way too much work!

We recently moved most of our metrics, such as cycle time of items, to an automated process, eliminating about an hour worth of work from one of my managers that was doing it manually. As it stands right now, all of our metrics except one is generated automatically and posted to our internal intranet site. However, the system still isn’t perfect, as it requires development and QA to navigate items through a pretty tedious workflow to get accurate information, something that in the heat of a release is often easy to forget to do – just because its “extra stuff”. I know, because when I’m involved I often forget to update them myself. It seems to me that the real problem getting metrics is how much work is involved in making sure you get the data.

Doug’s article really got me thinking. How can we get the fine grained metrics he’s talking about, in a way where someone isn’t sitting in an Excel spreadsheet for an extended amount of time inputting data, and where all data is updated real time?

As I was thinking about this dilemma, I hit F12 on my Mac to look at the weather and I noticed this little box staring at me:

Twitter Dashboard Widget

.. and then I started thinking. What about my experiences with Twitter could help with this problem?

Twitter First Impressions

I signed up for Twitter a few weeks ago after hearing Coté and Jason Calacanis mention it quite a few times on their respective podcasts and blogs. I wasn’t really impressed with it at all, only because I’m not willing to set up my phone to use it (I hate typing on a phone) and I don’t like having a separate web page up all of the time. Then I found the Twidget dashboard plugin for OS X and found that hitting F12 and typing a message was much more conducive to my working style than any of the above methods – it was less of an interruption.

Most of the twitter messages I post now are at home, where I have my F12 key handy and it takes little effort to actually post an answer to the question “What are you doing?”. Its effortless.

Trying To Go Lean

One of the adjustments we’ve made to our development process over the last few months is a logical reduction in the “types” of work we track. Rather than deal with “projects”, “change requests”, and “defects”, we are attempting to reduced these queues into something we call “open items”. For each change we want to implement, one or more “open items” are entered to address them. These items are then tracked for cycle time and completion status. Along with cycle time, we also keep track of the total number of “open items” we have that have not yet been deployed to production. These are reported on using your standard burndown chart that would look something like this:

Example Open Item Burndown

For larger projects, the overall project is tracked on a Wiki, where we can both keep track of current tasks, and create technical documentation for the work we are doing as we do it. We’ve found that the documentation we create as we are doing the work is much more complete and accurate if we write it during the development effort than it is when we “go back later” to document it. A particular initiative is mapped to several smaller “open items” and scheduled for subsequent, incremental deployments (sprints, increments, whatever). In an ideal world, only the items that are scheduled for the next deployment would be worked on for the next deployment, but we aren’t there yet.

Tracking “open items” as a general unit of work gives us a start in creating a “pull system”, where the development team can work on open items as they come into the queue. The prioritization process, ideally, would be extremely light: pull things from the head of the queue, in order of criticality (Critical, High, Medium, Low), oldest first. This gives us a very simple ruleset to pull from (no prioritization meetings required!), and allows our business users to set a priority that can expedite changes if it is necessary. The defect tracking system, at this point, is a real time reflection of the work we have to do and the priority that the business thinks the work should be done in. This also allows “demand” to come in in a uniform way. The rules are simple and there is one entry point for all work.

On a weekly basis, data is pulled from the defect tracking system and loaded to our intranet database (a small data mart) where we calculate cycle times, and can graph the current rate of work.

Measuring without The Context Switches

The problem is that reporting is only a part of the problem. You can report on the data, but the data will only be accurate if you have an effortless way to update the items you are working on.

Its like Twitter – if I have a separate window that I have to keep going to in order to actually post something I’m less likely to make the effort – because I want to get done with what I’m doing.

Tracking work within the software development process is even worse if, in order to do it, I have to to look up an item over and over again to update it, and there is some complex workflow that it has to go through. Chances are, Ron is going to look like he is behind all the time, no matter how much work he’s getting done.

So what would a “perfect” work tracking system look like for me?

Ron’s Ideal Work Tracking System

I’ve already asserted that I am more likely to Twitter if I can just hotkey to something simple, type something in and press my update button. I think the accuracy of measurements could be increased dramatically if we could have a system that took these types of things into account and reduced the context switches that the team has to go through in order to update status so that they can update real time and it feels like its part of the work that they are doing, rather than something separate. Its an added bonus if this feels like a social, fun activity rather than an extra set of tasks that need to be performed.

The way I see it, work can be entered into the system with the normal interface to the tracking system. This would give full visibility to everything needed to schedule work.

The new developer/QA interface to this system could look a lot like the Twitter widget above. Just a simple “What are you working on” type of interface that could talk to a network based API that, when a message is sent over, could search for a ticket number in the message and add a note to the item. A Subversion post-commit hook could use the same API to update the “code complete” status of the item (pulling the ticket number from the commit message), while a separate QA “widget” could have a pass / fail status pulldown that the QA team could use to signify the testing state.

As an added bonus, the deployment system could also use the API to let the QA team know the deployment status of the given item and whether it is actually there to test. This functionality, intersected with a “Pass” testing designation from the QA team in the production environment, could actually close the item automagically, which would reflect in your measurements.

The key to a system like this for me, is that if the updates feel like more of a social activity, and some of the obvious things are automated out of the process (like having to close an item once it is deployed to production and passed QA testing) the chances are high that you will actually get accurate real time status of what people are working on and the issues they may be running into. This is definitely some of what I am seeing as we use wiki software to track larger development efforts and as a manager the smaller grain information that I would get from a more “socially oriented” system would be extremely valuable to me just to know what is going on at any point in the process.

Now, this is just one view of the problem, from someone who is trying to lighten the large corporate process of getting work done. It might not work in every scenario or every environment – but I think it would be an interesting experiment.

I’d definitely be interested to hear other peoples opinions on this train of thought. Am I nuts or does this actually make some sense?

Video: How Open Source Projects Survive Poisonous People (And You Can Too)

Since getting a 80G iPod about a month ago two weeks ago, I’ve been really getting into watching the Google Tech Talks on Google Video. I recently watched How Open Source Projects Survive Poisonous People (And You Can Too), a lecture given by Brian Fitzpatrick and Ben Collins-Sussman from the Subversion team (now both Google employees) that summarizes a lot of information in Karl Fogels book Producing Open Source Software: How to Run a Successful Free Software Project.

If you haven’t had time to pick up and read Karls book, this video would be a good primer to some of the concepts in it and could very well motivate you to pick it up. Its an excellent book and one that I thoroughly enjoyed reading.

WordPress 2.1.3 released.

Version 2.1.3 of the WordPress blogging platform has been released and is available for download. According to the WordPress blog, this is a security release that “includes fixes for several publicly known minor XSS issues, one major XML-RPC issue, and a proactive full sweep of the WordPress codebase to protect against future problems”.

I’ve upgraded, and so should you. Take a couple of minutes to do this upgrade, as the possible consequences aren’t worth the humiliation. 😉

WordPress 2.1 Released

The WordPress team has announced the release of version 2.1 of their WordPress blogging software on Monday. A list of the changes can be found in the release announcement.

Sure, this is old news (by internet time standards) but I just got around to getting caught up. I’ve upgraded the site and the process was absolutely painless. All of my current plugins worked as far as I can tell.

For those who have a bit of trepidation over upgrades and the compatibility of plugins, I can tell you that my installation, with over 17 plugins active, seems to be working fine after the upgrade this morning after a total upgrade time of around 15 minutes including backups of my installation directory and database. As usual, the upgrade instructions are available on the WordPress site.

The WordPress team is planning its next release, Version 2.2, around the April 23 timeframe.

I haven’t had a great deal of time this morning to play around, but it does seem that the site runs just a tad bit faster then it did under the previous version.