Tag Archives: agile

Development Methodologies and Jeet Kune Do

Posted on by

[ This post was written in June of 2006 and never finished.    After 2,000 words I didn’t know how to end it.    It still seems relevant today though with all this DevOps stuff going on – I found it while browsing old drafts ]

I was reading an article over on ittoolbox.com called Universal Development Process and it got me thinking about how ineffective most software development methodologies are and why they limit the creativity and productivity of teams.

Over the last year and a half or so, I’ve spent a lot of time in a fairly large company talking about innovation. The dictionary defines innovation as “the act of introducing something new”. I can’t remember where I read it, but I found a better definition of innovation somewhere that more accurately expresses what it is. This definition states that innovation is rarely something new, but is actually a “new application of existing ideas from one industry or problem area to a new industry or problem area”.

In this article, I am attempting to apply existing ideas from the philosophy of Bruce Lee’s Jeet-Kune-Do to the problem area of software development. Hopefully, you find some value in the exercise. I find a lot of value in thinking about it.

Introduction

Development methodologies have been around for as long as I can remember. Each new “fad” promises better results and more productivity for teams using them. Personally, I have never been a part of a team that has implemented “a methodology” that did not run into problems that they could solve within the context of the methodology.

Unfortunately, managers and business users within a software organization are always looking for a “silver bullet” that will take what is ultimately a creative process and turn it into production-like processes that are repeatable and predictable. The basic assumption from these groups is that software development should be like making cars. The problem with this assumption is that each software project is different — and more importantly, each development team is different and unique in and of itself.

Fortunately for these groups, there are other groups of people that create and market “methodologies” that purport to create this productionized process for software development. The idea in most of these implementations are that there are specific things one can do to control the creative process of software development and make it predictable and repeatable.

These complete tool sets are marketed and bought by big businesses and forced on the development teams in order to help them be more productive. More times than not, teams feel limited by the tools, and options that are outside the process cannot be considered within the context of a software process.

Through much of my adult life, I have always enjoyed the writing of Bruce Lee. Over the years I have compared his ideas around “classical forms” of martial arts to the idea of software development and “methodologies”. This article is meant to draw some correlations and spur some thought among the IT managers in the industry to perhaps look at software development a little differently, as a creative, fluid process rather than something completely structured and rigid.

I’d like to take a few topics and talk about how Bruce approached them in the formation of Jeet-Kune-Do, and how these ideas could possibly translate into the “art” of software development and why individual software development methodologies, or styles, may not, in and of themselves, deliver the expected results that were marketed.

The Limited Vocabulary of Methodologies

When there is freedom from mechanical conditioning, there is simplicity. The classical man is just a bundle of routine, ideas and tradition. If you follow the classical pattern, you are understanding the routine, the tradition, the shadow — you are not understanding yourself.

— Bruce Lee

Bruce was constantly frustrated with the “static” nature of martial arts styles. His argument, in a nutshell, was that a specific style taught you a specific way of doing things and that with the limited vocabulary dictated by a particular style, you only had a set number of responses to what is normally a dynamic, ever changing, and most likely random chain of events based on the moment.

Perhaps Bruce could say it better:

Too much horsing around with unrealistic stances and classic forms and rituals is just too artificial and mechanical, and doesn’t really prepare the student for actual combat. A guy could get clobbered while getting into this classical mess. Classical methods like these, which I consider a form of paralysis, only solidify and constrain what was once fluid. Their practitioners are merely blindly rehearsing routines and stunts that will lead nowhere.

I believe that the only way to teach anyone proper self-defence is to approach each individual personally. Each one of us is different and each one of us should be taught the correct form. By correct form I mean the most useful techniques the person is inclined toward. Find his ability and then develop these techniques. I don’t think it is important whether a side kick is performed with the heel higher than the toes, as long as the fundamental principle is not violated. Most classical martial arts training is a mere imitative repetition – a product – and individuality is lost.

When one has reached maturity in the art, one will have a formless form. It is like ice dissolving in water. When one has no form, one can be all forms; when one has no style, he can fit in with any style.

— Bruce Lee

“Classical” methodologies have the same effect as Bruce felt “classicial styles” had. Talk to any team (better yet, ask the management team) and you will find that they develop software by “using RUP”, “being an XP team”, or “using SCRUM” rather than using a set of tools to develop software. The assumption by those above the team in implementing the methodology is that if you aren’t doing whats prescribed, your doing it wrong.

Unfortunately, when using a particular set of tools described as “the way”, one often winds up in situations in which they did not expect and have to find within the methodology the right solution to use, rather than doing what works to get the problem out of the way and get the project moving again. There is also a much greater chance that the team is put into a situation in which some “expert” will tell them that they “did it wrong” because they didn’t do it in accordance with the selected methodology. I’ve seen so much time wasted to figure out the “right” way to do something when everyone already knew what needed to be done, they just had to find a way that it would “fit” into the “official” structure of the team or organization.

The overarching structure of a particular methodology limits the possibilities a team has to act in a way that isn’t in accordance with the structure.

Notice that the stiffest tree is most easily cracked, while the bamboo or willow survives by bending with the wind.

— Bruce Lee

I believe that each development “methodology” is a collection of tools that could be valuable within a particular context. I do not believe that they are all or nothing. It is essential that you, as Bruce would say, “absorb what is useful, discard the rest, and add what is uniquely your own”.

Learning the Basic Tools Enough to Improvise

Within software development, we have a set of tools that we use in order to get our work done. Be it source control, languages, refactoring, test first development, etc, there are intricacies of the tools that we need to know in order to be able to improvise when situations arise that we are not expecting. Rarely, if ever, can we stick to one way of doing things on a project, so unencumbered knowledge of the basic tools are essential.

Again, Bruce addresses this within the philosophy of Jeet-Kune-Do:

I refer to my hands, feet and body as the tools of the trade. The hands and feet must be sharpened and improved daily to be efficient.

It is true that the mental aspect of kung-fu is the desired end; however, to achieve this end, technical skill must come first.

The techniques, though they play an important role in the early stage, should not be too restrictive, complex or mechanical. If we cling to them, we will become bound by their limitation. Remember, you are expressing the technique, and not doing Technique number two, Stance three, Section four?

Practice all movements slow and fast, soft and hard; the effectiveness of Jeet Kune-Do depends on split-second timing and reflexive action, which can be achieved only through repetitious practice.

When performing the movements, always use your imagination. Picture your adversary attacking, and use Jeet Kune-Do techniques in response to this imagined attack. As these techniques become more innate, new meaning will begin to emerge and better techniques can be formulated.

— Bruce Lee

Most of the time when implementing a “methodology”, more focus is put on the structure of the project than learning the tools around the methodology enough to be able to improvise. I like to focus my teams on becoming intimate with the basic tools that they need in order to work effectively.

Perhaps a practical example would help. We implemented Subversion as our version control tool in May of 2004, the day it went to version 1.0 – I would have had a hard time justifying a beta tool to the “enterprise architects“. During the transition to Subversion, I made the rule for the group that no GUI’s would be used with source control. Thats right, no Tortoise SVN, no Rapid SVN, nothing. The team was only allowed to use the command line tool. There was some “resistance” to this idea and to this day I’m not sure that I’ve ever really been able to communicate effectively why I made this rule. It all boils down to the ability to improvise.

I believe that in order to improvise, you have to have intimate knowledge of the tools at your disposal. This intimate knowledge of the capabilities of the core tool would enable each individual team member to aggregate a body of knowledge, internalize it, and be able to improvise when the unexpected occurred. This is one of the reasons why, as I’ve pointed out to my team members a number of times, that I have spent countless hours over my career retyping commands and / or code fragments rather than cutting and pasting – and why I still use emacs rather than the latest cool IDE. When it all comes down to it, repetition is the mother of skill. I believe that this is what Bruce was talking about in the above quote.

For our team, a GUI tool would have tied them to a fixed set of capabilities limited by someone elses interpretation of how the tool worked, rather than the true form of the tool and its capabilities.

Over time, I felt that this intimate knowledge of how the tool worked would enable the team members to generalize the concepts enough that they would be able to find new ways of doing things – that “new meaning will begin to emerge and better techniques can be formulated”. How the team used the tool was up to them, but I wanted them to use the tool in the purest form in order to learn it rather than a filtered version of it.

All or Nothing

When people talk about fighting schools they say that Kung Fu, or Karate, or this other style is the best. That is silly, and the problem becomes that the fighting style then becomes set in stone with no growth, and no adaptation, because what works well with me might not work for you.

— Bruce Lee

I was in a training class recently in which the instructor said “If you are not doing Pair Programming, you are not agile”. His assertion was that you have to use all the tools in the XP “methodology” in order to call yourself an agile team — that you cannot pick and choose the tools that work for the individual teams, but have to take the set as it is and use all parts.

I don’t necessarily agree. Pair programming works in many instances, but I have a hard time believing that this is the only way to do knowledge transfer and to create solid software. If it were, there wouldn’t be such successful distributed teams such as the Subversion Project or many of the different Linux teams (or the kernel team itself). Communication can be heightened without having people sitting next to each other watching or being watched by someone else. Bottom line is, depending on the individual, some might not be comfortable with this tool, and I think thats OK.

Its sad. Management teams overall want production line type processes to create software. They want to be able to predict schedules and everything that could possibly go wrong up front – and they want their predictions to be accurate. While most methodologies do not promise “precognition”, the interpretation by management teams most of the time is that precognition comes with the methodology they are buying. After all, why would they be doing something with a name if they are, more than likely, not going to better off in these areas by doing so?

[ This is where I ended.   After 2000 words I guess I was tired ]

Business Value vs. Customer Value

Posted on by
1

I read an article on the Accurev blog yesterday entitled Three Surprises in Software Development in 2012 that got me thinking. Specifically, it was this quote from the article:

“For every organization that is killing it with Agile, there are five (my agilesta friends say ten) organizations that are limping along, delivering buggy code to their customers, late, and missing committed functionality.  And often all three.”

One of the large challenges I see when moving people to agile are the ideas and concepts around “business value” and “customer value”. The term “business value” is usually used synonymously with “revenue” – how much money can we make if we do this thing that we are planning to do. Usually, this number is what gets a project resourced and approved for further development.

In an agile shop, we want to focus on “Customer Value”. “Customer Value” is considered something your customer is willing to pay for. Something they will find valuable, not something you find valuable.

During the adoption of agile practices, one of the first things that needs to be done is the reframe of how people think about the work they are doing from “Business value” (revenue) to “Customer Value”. Building something your customer is willing to pay for. Thinking of it this way, “revenue” becomes a measurement of success, a “side effect” of providing value. This mindset changes the way you think about your product development and the things you do to service customers.

Lean defines three types of work that usually occur in a process:

  • Work that provides value that the customer would pay directly for (Value Add Work)
  • Work that does not provide direct value to the customer (Non-Value Add Work)
  • Work that does not provide direct value to the customer, but is necessary in order to provide value they would pay for (Non-value Add, But Necessary)

Value add work, as mentioned above, is usually characterized as “something your customer is willing to pay for”, while non-value add is the direct opposite. A recent article I read uses blood tests as an example. Customers are willing to pay for the results, not the processes that they have to be subjected to in order to get the results (registration, paperwork, etc).

Once you identify the value add work the objective is to eliminate as much of the non-value add work as possible. You will not be able to eliminate all of it – which is why there is a bucket for “non-value add, but necessary”. Some things, like paperwork and medical history collection are necessary – your customer just doesn’t want to be exposed to it – or would not directly pay you to do it.

One of the trickiest parts of this classification is figuring out the difference between “non-value add, but necessary” and the other two. Many organizations feel that the things they do as part of their process are all “value-add” (why else would they be doing them?), while others try to eliminate anything that the customer is not willing to directly pay for (like testing or quality procedures).

Once we understand the different types of work, we have to go back to the difference between business and customer value.

There is a huge difference between something “your customer is willing to pay for” and revenue. The former is a statement of value from the customer perspective, the latter is a result of doing those things that the customer finds value in.

If you are viewing “business value” as revenue as an organization, you will make shortcuts in quality in order to hit the market or customers as quickly as possible – in most cases exposing quality defects that will effect the customer experience in a negative way. You will “ramp up resources” because the more people you have on the project the quicker you can get it done and usually you will short cycle things like testing, or code reviews because they aren’t serving the purpose of getting the product out so that people will use it.

Even worse, you will give up creative control of your product and add things based on requests of your highest paying customers – rather than think through the experience and do the things that provide your customers real value that they would pay for.

Revenue as the goal of doing something is the classic “Innovators Dilemma”. Companies continue trying to move up market by addressing their larger paying customers while diluting their product – as small upstarts develop products or processes that actually provide value – even though its not as much – at a smaller price tag. I think this “Innovators Dilemma” concept is what Steve Jobs was talking about in his biography by Walter Isaacson, when he said this:

“I have my own theory about why decline happens at companies like IBM or Microsoft. The company does a great job, innovates and becomes a monopoly or close to it in some field, and then the quality of the product becomes less important. The company starts valuing the great salesmen, because they’re the ones who can move the needle on revenues, not the product engineers and designers. So the salespeople end up running the company. John Akers at IBM was a smart, eloquent, fantastic salesperson, but he didn’t know anything about product. The same thing happened at Xerox. When the sales guys run the company, the product guys don’t matter so much, and a lot of them just turn off. It happened at Apple when Sculley came in, which was my fault, and it happened when Ballmer took over at Microsoft. Apple was lucky and it rebounded, but I don’t think anything will change at Microsoft as long as Ballmer is running it.”

Isaacson, Walter (2011-10-24). Steve Jobs (pp. 568-569). Simon & Schuster, Inc.

The goal of a product or service should be providing value to your customers, something they are willing to pay for. Revenue then becomes a side-effect – a measure of success. It becomes confirmation that you are doing the right things.

I don’t believe an organization can implement Agile successfully until this change in thinking occurs. The wrong motivators absolutely effect behavior – and most of the behaviors Agile and / or Lean try to implement is focusing on value first, providing quality software in small increments, and being able to respond to change quicker. The more you are focused on an end for you and not your customers, the more you are moving away from these values.

This doesn’t mean that revenue isn’t important – every company needs it. It means that if you are doing the right things, in the right order, it will follow and keep you going as a company.

Recipes are for Learning, not for Real Life

Posted on by
1

Recipe BookPeople like recipes. We’re always looking for the next ten step way to improve what we do and how we do it. We see it in leadership trainings, “methodologies”, and pretty much any self improvement or team productivity books that we read. Each of the above give frameworks along with classifications of things that you can use to solve a particular problem.

What gets lost a lot of the time is that the classification of things and the frameworks are written to teach, not to use in every day life. While you might have to consciously think about the way to classify something at first, while you’re learning, the key point of these things is to give you a way to classify and deal with information while you are learning. At some point, the idea is that the framework will fade away and you will innately have the tools at your disposal, without having to think of the distinct classifications or step by step instructions to deal with them.

Its like learning to ride a bike. When you are learning to ride a bike, you learn balance, steering, and the fact that you have to push one foot at a time on the pedals to achieve the motion you need in order to propel the bike forward. At first, its very awkward. Your steering is shaky, you don’t pedal fast enough to get the inertia needed to move the bike forward, because you are thinking of all of the things you need to do at the same time to achieve your goal. Pretty soon though, you internalize all of the independent skills you need that make up this thing called “riding a bike” and it becomes one thing rather than many concurrent things you have to think about. Pretty soon, you are modifying or adding to what you know and doing things like riding no-handed, wheelies, or whatever other things you can think of that augment your experience of this thing called “riding a bike” that makes it a little more “your own”.

Methodologies and religious arguments around software development have always put me off. When I’m asked to participate in a “methodology” discussion I usually cringe, because I know that 9 out of 10 people feel that the methodology as it is written is the goal, rather than taking the methodologies as a set of tools that are used in the learning stage to develop competence and later modify for the environment in which you work. This tends to get lost most of the time and you find teams and people doing things as they were specified in the recipe, whether they are useful or not, rather than adapting the tools – or even throwing some of them out if they do not apply.

“Agile” methodologies are a perfect example of this. You have specific formats for standups that are interpreted literally, specific tools like automated builds, unit tests, user stories, and burn down charts. Some methodologies include specific artifact definitions like activity diagrams, object diagrams, 4 + 1 architectural diagrams. In most cases you find people using all of them as they were specified in the book because they haven’t moved to the point where they understand the purpose of these tools, what they accomplish and how they do or do not fit together for a particular situation.

Agile software development is not the format of the standup or the fact that you have automated builds. As far as I can tell, Agile software development achieves the following:

  • Get the customer involved and the team communicating.
  • Break the functionality required into small, incremental, implementable pieces that are prioritized around both business and technical dependencies – with the goal of creating production ready code. Small iterations also give you the ability to adapt to changing requirements. What seems important at the beginning of one iteration might not be as important as something in the next iteration – so you plan incrementally.
  • Create production ready code at the end of each iteration, with rapid feedback cycles – this is the purpose of unit tests, and automated builds – to be able to “stop the line” when something breaks without a lot of manual intervention – and to deliver “just enough value” as soon as possible.

I think the problem is that most of the time people do not look at the philosophy around a methodology. Most of the tools that appear in these methodologies have a purpose to get to a certain philosophical goal, like the bullets listed above. Unless you understand the goal, you can have perfect stand ups and not ever really “implement agile” in the way that it was intended.

Related Articles

Implementing Open Source Defect Tracking as a Corporate Tool – An Update

Posted on by
2

Note: This article was started around March or April of this year – and I just got the motivation to finish it – because I thought it was important. I hope I did the original idea justice all these months later.

Back in October of 2007 I had written about starting to implement the Eventum Defect Tracking System in place of the defect tracking system that we had been using for about 8-9 years. I thought I’d throw up an update on how we’re doing, and what advantages we are seeing from taking the Open Source route over the proprietary route.

Let me start off by saying that the more we use Eventum, the more impressed I am with the thought that went into the development of the application. There were only a few minor tweaks that we had to do in order to get it working in a way that we could be much more productive. These tweaks were things like enabling basic LDAP authentication on the application and most recently enabling email integration and source control integration, which are actually already implemented in the system. Some little tweaks for our environment to alleviate the change required were necessary, but as you will see, these tweaks were small.

Basic Install and Feedback Process

I had the basic install of Eventum up in about thirty minutes. I then sent the link to the application out to small pieces of the team in order to receive targeted feedback. Most of the feedback revolved around things that were missing that people were used to in the current application – but these things were largely why I wanted to remove it in the first place. They caused a lot of extra work. I made some tweaks to add custom fields, and sent it out to another small group. Once everyone had a chance to play with it a bit, I assembled the team to talk about what we needed to do to cut over. Much of the conversation was around how the new process would work and some were around tweaks that I had made that people didn’t feel we needed (which came directly from feedback from the team). Some of the tweaks were undone and we decided to move ahead with cutover on the next build cycle.

A conscious decision was made to just get the system up and people to start using it without all of the bells and whistles like email integration, which I would add incrementally as I had the time to do so. The next section will talk about some of the things we did in order to expedite the enabling of email integration.

Some of the initial feedback that we received on the tool was that the “screen was too cluttered” or people didn’t understand the states. Much of this, I knew, was simply because the tool was unfamiliar. People do not like change. I spent a lot of time reassuring people and asking them to “hang in there” as we familiarized ourselves with the tool and the new processes that would evolve through its use. To their credit, they did.

Enabling Email Integration

One of the reasons I chose Eventum from the many defect tracking systems I looked at was the capability to integrate email conversations at the ticket level. In April of 2007, I wrote an article called Metrics as a Side Effect in which I talked about how I would love to see metrics be able to be updated without having to make a concerted effort to switch contexts in order to do so. I viewed conversations around a particular ticket the same way – and wanted a system that would record all email conversations with the ticket so that we had something to refer to as things went live, were closed, or were killed. The thought behind this is that if you are required to go into the system in order to document a conversation, chances are you won’t. The conversation needs to be recorded as a side-effect of normal work activities.

One of the killer features Eventum offered was this capability. Unfortunately, the documented ways to enable it were a little unclear and required changes to the mail server in order to enable it. Eventum uses custom email addresses like issue_123@example.com or note_123@example.com to do email association. Common solutions require setting up blanket aliases (issue-*) or using ‘+’ addresses in your mail server to be able to intercept these issues. Also, POP3 or IMAP mailboxes are required. Neither of these were options for us, as we do not control the email servers. We had to find a work around – since we only had one email address for our “build system” and I didn’t want to have the myriad of conversations that I knew I would have to have to create these blanket aliases on the corporate server.

As it turns out, tweaking Eventum only took commenting out a few lines in the notification class and a small procmail recipe to allow the integration without touching the existing mail server configuration.

The code changes look like this, starting at line 159 in class.notification.php and consisted of commenting out 5 lines:

  // RCB:  Don't use custom from addresses
	
  //      if (@$setup[$routing]['status'] != 'enabled') {
            // let's use the custom outgoing sender address
            $project_info = Project::getOutgoingSenderAddress($project_id);
            if (empty($project_info['email'])) {
                /// no project email, use main email address
                $from_email = $setup['smtp']['from'];
            } else {
                $from_email = $project_info['email'];
            }
   //     } else {
   //         $from_email = $setup[$routing]['address_prefix'] . $issue_id . 
   //                               "@" . $setup[$routing]['address_host'];
   //     }
        if (empty($info['sender_name'])) {
            // no sender name, check if this email address belongs to a user and if so use that
            $usr_id = User::getUserIDByEmail($info['email']);
            if (!empty($usr_id)) {
                $info['sender_name'] = User::getFullName($usr_id);
            } else {
                // no name exists, use email address for name as well
                $info['sender_name'] = $info['email'];
            }
        }

These changes force the application to, instead of setting the outgoing sender address to issue-1234@example.com, to send it using the default project email address.

The next problem to face was intercepting these emails and processing them as Eventum would when pulling them from a POP3 or IMAP inbox. Luckily, our system uses procmail quite extensively to process emails for software deployments, and other update notifications for things like branches that are created, etc. We just had to add two procmail receipes and a perl script that looks like the following:

:0
* ^Subject:.*[#[0-9]+] Note: .*$
	| $HOME/preprocessEventumMessages.pl | /usr/local/bin/php /path/to/eventum/misc/route_notes.php

:0
* ^Subject:.*[#[0-9]+].*$
	| $HOME/preprocessEventumMessages.pl | /usr/local/bin/php /path/to/eventum/misc/route_emails.php

These recipes look at the subject and check to see if there is an issue number in it – as well as whether it has the word ‘Note:’ in it. If they do, they pipe the message to the following perl script, which acts as an email filter, translating the from address from our blanket address to the issue specific address and in turn piping it to the Eventum script to process the email:

#!/usr/bin/perl
#--------------------------------------------------------------------------------------------
# preprocessEventumMessages.pl
#
# This script is called by procmail when Eventum recipes are fired.  It pulls the 
# issue id from the subject line and manufactures an issue or note email address, basically
# faking Eventum into thinking that it is routing an issue specific email, without having to 
# change the mail server.
#
# This is basically a filter.   It reads from stdin and writes to stdout, so that procmail
# can process the message and send it to the appropriate routing script.
#
# The procmail rules look like this:
#
# :0
# * ^Subject:.*[#[0-9]+] Note: .*$
#   | $HOME/preprocessEventumMessages.pl | /usr/local/bin/php /path/to/eventum/misc/route_notes.php
#
# :0
# * ^Subject:.*[#[0-9]+].*$
#   | $HOME/preprocessEventumMessages.pl | /usr/local/bin/php //path/to/eventum/misc/route_emails.php
#
#--------------------------------------------------------------------------------------------
@contents = ;

# grab the full email into one big buffer ...
$mailContents = join('', @contents);

# grab the issue number and note designation (if present)
if ($mailContents =~ m/Subject:.*[#(d+)] (Note:)*.*$/mi) {
    # strip the issue number of leading and trailing spaces
    $issueNumber = $1;
    $issueNumber =~ s/^s+//sig;
    $issueNumber =~ s/s+$//sig;
    
    # if the note designation was in the subject, create an email address of 
    # note_xxx@example.com.   If not, its an email, so use issue_xxx@example.com
    if ($2 eq "Note:") {
        $newEmail = "note_" . $issueNumber . "@example.com";
    } else {
        $newEmail = "issue_" . $issueNumber . "@example.com";
    }
 
} 
        
# Find the To: header and replace the umbrella email address with the issue specific
# one and write the email back out to stdout line by line.   This will then be fed
# by procmail to the appropriate routing script (see module comments above)
foreach (@contents) {
    if (m/^To:/sig) {
        s/blanket-email@example.com/$newEmail/sig;
    }       
    
    print $_;
}

The Eventum settings are set just like you are using address based routing, and these scripts do the rest to fake it.

Now, once again, modifying the code directly – probably not the best way to do it. But it was the fastest and allowed us to make changes independently, without having to make changes to the corporate mail server.

Now we did find another issue once email association was working. There were some emails that were coming in that had pieces of the issue id missing. So we had one more tweak to make, again in class.notification.php in the block starting at line 952:

if ($type == 'notes') {
$extra_subject = $data['note']['not_title'];
// don't add the "[#3333] Note: "
// prefix to messages that already have that in the subject line
if (strstr($extra_subject, "[#$issue_id] $subject: ")) {
// RCB: 12-13-2008: This code was lopping off the issue id, which breaks email
// association.
//$pos = strpos($extra_subject, "[#$issue_id] $subject: ");
//$full_subject = substr($extra_subject, 4);
// keep the full subject instead
$full_subject = $extra_subject;
} else {
$full_subject = "[#$issue_id] $subject: $extra_subject";
}
} elseif (($type == 'new_issue') && ($is_assigned)) {
$full_subject = "[#$issue_id] New Issue Assigned: " . $data['iss_summary'];
} else {
$extra_subject = $data['iss_summary'];
$full_subject = "[#$issue_id] $subject: $extra_subject";
}

I’m not entirely sure what this was doing and why the hardcoded ‘4’ is in line 959, but its definitely a bug, which I submitted to the Eventum project as a fix. I’m not sure if it was ever corrected.

The benefit that these changes give us, however, is that now when an internal note or email is sent from the Eventum tool and an email is sent out, the default behavior of people to respond to the email rather than go into the tool still results in an updated conversation about the item in question. No longer are decisions made without documentation – the documentation is created as a side effect.

Enabling Source Control Integration

The final piece to the puzzle was source control integration. We use Subversion for our source control system and ViewVC as a means to browse the source code (both using mod_ldap and mod_svn_authz to authenticate with the corporate LDAP store). Eventum includes a script that you can use in your Subversion post-commit hook to log each commit targeted to a particular issue (specified in the commit comments) that automatically logs commits to a particular issue. You can then set up patterns to build URL’s to your repository so that from each issue a user can view the particular commits through this URL. We set this up to link to our ViewVC application so that each commit will link to the diff of the commit. This allows us to streamline our code review process by being able to view each commit to an issue in context. This has proven to be a huge win – at least for me when I want to see what is going on for a particular issue.

Getting The Business Users Involved in the Process

The biggest benefit I believe we have received from switching to Eventum is that, while it looks complex at the beginning, once you start using it it is quite easy to learn. So much so, that for the first time that I can remember, we have our business users engaged in the defect tracking system to approve work that they feel are a priority. Some of this is still a manual process, but being able to distribute the entry and maintenance of items has been a big win for us, removing a bottleneck that we had of one person on the team designated as the “issue generator” for other teams. We now have our business users engaged in the process – everywhere from entering tickets themselves, to responding to issue emails and having the discussions automatically logged for historical purposes. This is something we had tried to do with our previous tool, but was unsuccessful due to the complexity of the tool and licensing costs involved in deploying the tool (or licenses) to people outside of our group – since ultimately these costs hit our (IT’s) budget. We can now add as many users to the system as we want and let them begin engaging in the actual work being done – rather than being passive participants.

Huge win.

Upgrading

We started with Eventum 2.0.1 and had multiple “projects” set up. The main project was the work queue for the group, with a few specialized projects that might have unique items, or may have to be re-entered into the main project for the development team to have visibility to (each group wanted to work from one specific queue – not check multiple projects for work to be done). The 2.1 release of Eventum included the functionality to be able to move items between projects – which for us was a big deal – as we were re-entering items when they were applicable to the main project and not specific to the sub-project they were entered into.

With this one feature alone, its given us the ability to set up multiple work queues, in which we can begin to segment “nice to haves” from musts – and move these “nice to haves” into the main work flow when they become things our users want to address. This will, in the future, also be a big win for us, as we can segment the demand and begin to have the development teams focus on the things that are important now, while enabling our business users to continue to queue work for us without creating additional noise in the main work queue.

Conclusion

For us, moving from proprietary software with licensing limitations to open source software has been a big win in changing the way we work. The absence of licensing costs has enabled us to engage our users in ways we couldn’t before, while the ability to integrate email and source control has enabled us to collect information as a side effect of our teams normal work process rather than adding additional record keeping work to their duties. The fact that we have the source code has enabled us to customize the software in small ways to our environment, avoiding cross functional work that could take longer to implement due to the additional hand offs between groups – and there is always the possibility that a change to the email server could be viewed as something that shouldn’t be done. Access to source code allowed us to work around that.

Overall, in my view, the implementation of Eventum for work tracking has had a huge positive effect on the way we work. Hopefully, this article summarizes the benefits that we’ve seen in a way that encourages people to give open source a shot in their corporate environments.

I would encourage members of my team to respond to this post as well to keep me honest. What I see may be very different than how it works “in real life” – but I think what I’ve documented here is pretty accurate.

Metrics as a Side Effect – Its REAL.

Aside

Its been over a year since I wrote Metrics As A Side Effect, explaining how my perfect time tracking system would be something based on a Twitter like application. Imagine my surprise when I found that Harvest Time Tracking Software enables Twitter updates. Its not exactly how I described it, but its a step in the right direction. Totally cool.

Agile / Lean or Common Sense and Permission To Change?

Posted on by
6

As anyone who reads this blog regularly knows, I’ve spent a lot of time over the last 3-4 years studying agile methodologies and most recently lean concepts and principles. I have most recently been reading a couple of books by Ricardo Semler, who runs his company in a completely democratic way – doing away with all top down authoritarian management principles and allowing the employees to make decisions on dress, salaries, where they work, when they work, and most importantly, how they work.

I remember when I had first read the book Agile Software Development with SCRUM by Ken Schwaber and Mike Beedle and I had sent an email to my manager with a link to the book and the small sentence fragment “common sense codified”. We began experimenting with Scrum within my group, but it was very difficult to get other groups on the same page. We wound up with a lot of sprints that ended when things left development and entered the “normal corporate process” to finish things up and get them to production. We also had a lot of conversations around whether what we were doing was “standard process”.

As I started reading books on TPS and Lean, the same thing occurred. It struck me how most of the things that are characterized by “lean” are just common sense principles explained in such a way that they sound like a “process” that manager types can “buy into”. But really, they work because they make sense – and people have the permission to standardize and then change their work rather than having things written down and subsequently treating these processes like they are set in stone. You can’t change them unless you go through an agonizing approval process up the management chain.

Over the years, I’ve read and listened to many podcasts talking about the same things going on in other companies. People struggle to be able to change the way they work because they have to get “management buy-in” to take action. So much wasted effort just to try something new.

Interestingly, once the “buy-in” occurs, over and over again people try to “implement Scrum” or “do XP”, mostly by the book, and do not throw out things that do not work for the group. For some reason, we all think we have to be a part of some “methodology” in order to be effective.

One of the most fascinating things about Semlers company is the explicit trust and ability to control ones own destiny that the employees of Semco seem to receive. I ran across a particular section of the book where he was talking about the process improvements that “just started happening” due to this culture change and I found some interesting similarities to lean that I thought would be interesting to highlight. He starts off with:

The factory committee spun off groups that studied the plants products and how the workers made them, looking for ways to save time and make improvements. These teams weren’t created by Semco; they formed spontaneously, as the bracing winds of democracy swept through the food service equipment unit, and often met after hours or during lunch.

Interesting, employees actually wanted to do a better job and self organized because they could.

He goes on, talking about some of the changes:

One group restructured the dishwasher assembly line, changing it from a sequential assembly process to a batch concept in which dishwashers are assembled in twos and threes by teams of workers that do many different tasks and spend the time between batches prefabricating the components they will soon need. They also came up with a system in which all the parts for the dishwashers were stocked in open racks in the middle of the factory. Metal tags, green on one side and red on the other, hung on each rack, and the workers would flip the tags when they saw it was time to reorder, ensuring a steady supply. This was a big improvement on the traditional assembly line, in which dehumanized workers have no role in decisions regarding the production process.

What we see here are people electing to move from an assembly line to, basically, work cells implementing small batches of inventory with workers that are skilled in multiple areas. They even set up a “Kanban” system, though I doubt they knew it, where they had visual cues of when parts needed to be supplied.

Note that not once in these paragraphs does he mention the word “lean”. There was no implementation of “lean”, no “lean” or “continuous improvement” initiatives. This just seemed to make sense to the people doing the work and since they were allowed to do it – and knew enough about the overall process rather than just their small piece of the overall process (i.e. they were multi-skilled), they were able to see the obvious and execute it without all of the red tape – and get great results for the company.

He goes on:

The strength of these groups was their diversity. They included factory workers, engineers, office clerks, sales reps and executives. They didn’t have a formal head; whoever showed the greatest capacity to lead got the job, calling meetings and moderating discussions. In more than one group, a shop-floor worker guided professionals. Instead of a seniority system, or boxes on an organizational chart that guaranteed power, the groups were held together by a natural system of collegial respect.

Again, you hear this a lot in TPS. People are trained from the bottom up in the company and have skills in multiple areas. While there are “leads” that are responsible for a product line, everyone has the ability to lead when they are the most skilled for the job at hand.

The only vague reference to lean that Semler makes in this passage is the following paragraph, where he draws similarities and differences between what Semco did and TPS (though Toyota is not explicitly stated):

There are similarities between this system and the Japanese approach to organizing manufacturing operations, but also important differences. In our groups, younger members didn’t automatically submit to their elders. Moreover, once a team decided an issue, it stayed decided. There was no approval needed to make a change. Then again, there were no special rewards for new ideas. It was a spontaneous process; people participated only if they wanted to.

As I read this it really got me thinking. In most of the process agile / lean related books that I’ve read there seem to be a few common themes:

  • Trust people to do the right thing for the company
  • Give them freedom and authority to work the way they want to
  • Push decisions down the chain as far as possible
  • Work in small batches and change things that aren’t working
  • Allow those who are capable of leading to lead, no matter what their title or position is
  • Put quality checks in place – whether it be test-driven development, or quality checks at each step in an assembly
  • Fix problems at the core and stop the line as quickly as possible – in development this would be TDD and automated builds. Once a problem is found, find the root cause and put a test or quality check in place to ensure it doesn’t happen again
  • and finally, Trust people to do the right thing for the company

One more principle that I would add would be “tolerate mistakes”. Many of the issues that I’ve come across with other groups is that if they make a mistake they feel they will be punished. I’ve had great success with my team in articulating that I know mistakes will be made, but I want them to be made once, a lesson learned, and things put in place (usually automated) to ensure they won’t happen again. I’ve found that if people know it is expected that mistakes will be made, and everything doesn’t have to be perfect, they are more receptive to trying something new.

But I digress.

What Semler’s story shows me is that if people are given the freedom to work the way that is most effective, they will. More than that, if you invest in them with trust, they will want to do these things as their commitment to the company will obviously go up based on how they feel they are treated.

Semler uses a key phrase throughout his books that is repeated over and over. “Treat people like adults”. Semco, Toyota, Amazon and Google seem to do a really good job at this, as I’m sure most high functioning companies do. Read this article called The Google Way: Give Engineers Room and you will see the same concepts outlined in the excerpts on Semco that I have just written about. It seems to be a common theme.

So my real question. Is methodology and process really the answer, or is it deeper than that? Is it the way we treat employees that cause inefficiencies? If it is, if we took this base principal of trust and actually implemented it, would our employees come to the same conclusions as companies like Semco, Toyota and Google?

I think they would, because the principles and processes implemented by these companies are really just common sense without all of the complications of “process” and authoritarian management. They encourage workers to work outside their “box” and learn what they need to learn to be more effective. I would guess these employees feel valued, because they can constantly improve themselves rather than just “be the guy that puts the screw in the hole”. When you are allowed to improve yourself, your commitment rises to those who “allow” you to do so. What you wind up with is a highly efficient company that can change on a dime because people are allowed (and encouraged) to change and improve.

Most interestingly, the processes wind up looking “agile” or “lean”, without all the cruft of trying to follow a cook book.

Am I officially becoming a hippie, or does this line of thinking make sense? Let me know if I should go join a commune.

LDAP Enabling The Eventum Defect Tracking System

Posted on by

Due to a recent reorg, I have the opportunity to replace our defect tracking system, which has quite a bit of really wasteful process baked into the tool, with a new one. I’ve been looking at defect tracking software for a while, and chose Eventum, an open source project by MySQL AB for a number of reasons, some of them including:

  • Its open source
  • Its written in PHP, so I don’t have to worry about messing with fastcgi, mod_perl, or mod_python
  • It is extensible (you can add custom fields, etc)
  • It uses MySQL, rather than SQLLite or something like that, so we can integrate it into the rest of our home-grown build software
  • It supports email integration. While we won’t be using this right away, we’ll be implementing it in a later iteration
  • Its simple to use, with a very simple interface, once you get use to it. Everything is essentially on one screen.
  • It has time tracking, along with some basic reporting built in

One thing it doesn’t have built in is LDAP authentication. I wrote a previous article about all of the work we’ve done to integrate both our home grown applications and a few open source applications in with our LDAP store, to minimize the management of multiple passwords across systems, so this was very important to me. I started with many, many Google searches to see if someone else has done this, only to hit one dead end after another. At first I was being lazy and decided to just forget about it. One system not tied to the LDAP tree isn’t that big of a deal, but then my perfectionism set in. Why would I settle for that when LDAP authentication should be really easy to integrate into an Open Source package?

So I decided to spend a few hours to get it working. Since I had no success finding an implementation, I figured I could do my part and post what I have. There are a couple of caveats that I want to throw out before we actually get to the code though:

  1. It isn’t done “right”. This is all extra work for me, so I got enough done so that it would work. The right way to do this would to refactor the auth stuff out into a workflow like hierarchy that could be pluggable (see this post in the eventum mailing list). I’ll get to that someday, but right now this solution hacks the auth module to get authentication working.
  2. LDAP Settings are not configurable through the interface. I don’t have time for that, so a set of defines at the top of the LDAPAuthenticator class contains all of the configuration information for the LDAP server. Bummer, but like I said, I’m on a schedule.
  3. Users still have to be added to the Eventum database – they are not added automagically when they log in. I want control of who is in the system, so I’ve elected to leave this functionality out and just do authentication.

With these three caveats in place though, given my experience looking around for this stuff, at least this code works and will be able to be used by others. Its a starting point – which is more than is out there today. Anyone is free to use this and take the time to do it right. With that said, I’d love to receive updates if someone actually takes this up. For now though, this works for me.

So, now to the code. I wrote a small PHP class called “class.LDAPAuthenticator.php. There are two functions in it. Because Eventum uses email address as the login, we need a way to get the full user DN from the email address. This is what the email_to_dn function does. Given an email address, it returns the full distinguished name of the user. This is called by the main class function, ldap_authenticate. The ldap_authenticate function takes the same arguments as the class.auth.php function isCorrectPassword, which consist of the email address and the password. It binds to the LDAP authentication tree using the full DN of the user and the password supplied. If authentication is successful, it returns TRUE, otherwise it returns FALSE, just like the isCorrectPassword function used to validate the password from the Eventum database.

The code looks like this:

# Change these values to access another LDAP server.
define("LDAP_PORT", 636);
define("LDAP_HOST", 'ldaps://ldapserver.example.com:' . LDAP_PORT);
define("LDAP_BIND_DN", 'PUT THE BIND DN HERE');
define("LDAP_BIND_PASSWORD", 'PUT THE BIND PASSWORD HERE');
define("LDAP_SEARCH_DN", "PATH OF THE TREE TO SEARCH FOR USERS");

class LDAPAuthenticator {

# Look up a users full distinguised name from
# their email address, since Eventum uses
# email address as the login name.
function email_to_dn($emailAddress) {
$returnDN = "";

$server = ldap_connect(LDAP_HOST);

if ($server == FALSE) {
return($returnDN);
}

ldap_set_option($server, LDAP_OPT_PROTOCOL_VERSION, 3) ;

$ldapbind = ldap_bind($server, LDAP_BIND_DN, LDAP_BIND_PASSWORD);

# verify binding
if ($ldapbind) {
# find the user based on the entered email address.
$result = ldap_search($server,
LDAP_SEARCH_DN,
"(&(mail=$emailAddress))",
array("dn"));

$info = ldap_get_entries($server, $result);

# if we actually got a value back, return the users DN
if ($info["count"] > 0) {
$returnDN = $info[0]["dn"];
}

ldap_unbind($server);
}

return($returnDN);
}

# Authenticate with the LDAP server. Function returns true
# if authentication was successful, false otherwise.
function ldap_authenticate($email, $password) {
$returnValue = FALSE;
$userDN = LDAPAuthenticator::email_to_dn($email);

if ($userDN == "") {
return($returnValue);
}

$server = ldap_connect(LDAP_HOST);

if ($server == FALSE) {
return($returnValue);
}

ldap_set_option($server, LDAP_OPT_PROTOCOL_VERSION, 3) ;

$ldapbind = ldap_bind($server,
LDAPAuthenticator::email_to_dn($email),
$password);

if ($ldapbind) {
$returnValue = TRUE;
ldap_unbind($server);
}

return($returnValue);
}
}

Save this file as class.LDAPAuthenticator.php and put it in your Eventum includes directory. Modify the define statements at the top to contain your LDAP server information.

Now, to use it. Go to your Eventum includes directory and add the following line to the top of the class.auth.php file:

require_once(APP_INC_PATH . "class.LDAPAuthenticator.php");

I have this at the end of all of the rest of the require statements.

Now, replace the isCorrectPassword function in class.auth.php with the following function:

 /**
* Checks whether the provided password match against the email
* address provided.
*
* @access public
* @param string $email The email address to check for
* @param string $password The password of the user to check for
* @return boolean
*/
function isCorrectPassword($email, $password) {
return(LDAPAuthenticator::ldap_authenticate($email, $password));
}

… and VOILA. You can now authenticate off of your LDAP tree.

Now, I know it isn’t pretty, hacking the code directly – but it works, and its more of a starting point than I can find anywhere else. I hope its useful to others. Again, if anyone takes this further and does it “right”, I would be really happy to get a copy of the modifications.

One more thing – don’t forget to require SSL on the URL to your Eventum installation by using the SSLRequireSSL directive in your Apache server. You don’t want these passwords floating around in the clear across the network.

Download the Eventum LDAP hack here and happy authenticating.

Organizational Features of a Lean Plant

Posted on by
Reply

I’m reading The Machine That Changed the World : The Story of Lean Production by James P. Womack, Daniel T. Jones, and Daniel Roos. It is an extremely interesting book.

I ran into this small paragraph yesterday that for some reason stuck in my head as something important:

The truly lean plant has two key organizational features: It transfers the maximum number of tasks and responsibilities to those workers actually adding value to the car on the line, and it has in place a system for detecting defects that quickly traces every problem, once discovered, to its ultimate cause.

I’m telling you, the Poppendeick books are great, but there is nothing like going right to the source for an explanation of lean. I’m about 100 pages into the current book and I am absolutely fascinated at how much of todays current corporate structure (multi-level, many people with very specific task sets or responsibilities) is based on things that Ford and Sloan did with their companies.

In IT, this management style is manifested through all the different groups one hears about all the time from people in the field: Development, Infrastructure, Business Analysts, Quality Assurance. Each its own little silo, with its own responsibilities – and never should one group know how to do, or be privy to, the information in one of the other groups. Handoffs occur between the groups via very large documents.

Sometimes it goes further than that. I was talking to a friend once (who worked at another company, BTW) who told me about how their DBA’s were responsible for uptime and performance of the database and had decided that developers were not allowed to use ORDER BY clauses in their SQL because it effected the performance of the database. These developers were actually forced to sort their results within the application, rather than use the capabilities of the database, adding additional complexity to an already complex application. Worse, management seemed to buy into the decision, as I don’t think I would have been hearing about the situation if it was overruled. Ridiculous.

Another quote from the book, same page:

In old fashioned mass production plants, managers jealously guard information about conditions in the plant, thinking this knowledge is the key to their power.

Again, shocking how much of this mentality you read about in corporations not even connected to automobiles. This sounds like just about every company I’ve talked to people about (or worked at) over the years.

I’ve come to the decision over the years that ultimate transparency is the key to breaking down silos. It only breaks down your silo, but hey – thats a start, and at least you are setting an example.

Its definitely very beneficial, I’m finding, to read about things that are completely outside your profession to give you some distance from what is being taught. The lessons flow in easily this way, because you don’t have the predisposition that you “already know how things work”.

I recommend to anyone in IT to pick this book up. Its absolutely fascinating.