Development Methodologies and Jeet Kune Do

[ This post was written in June of 2006 and never finished.    After 2,000 words I didn’t know how to end it.    It still seems relevant today though with all this DevOps stuff going on – I found it while browsing old drafts ]

I was reading an article over on ittoolbox.com called Universal Development Process and it got me thinking about how ineffective most software development methodologies are and why they limit the creativity and productivity of teams.

Over the last year and a half or so, I’ve spent a lot of time in a fairly large company talking about innovation. The dictionary defines innovation as “the act of introducing something new”. I can’t remember where I read it, but I found a better definition of innovation somewhere that more accurately expresses what it is. This definition states that innovation is rarely something new, but is actually a “new application of existing ideas from one industry or problem area to a new industry or problem area”.

In this article, I am attempting to apply existing ideas from the philosophy of Bruce Lee’s Jeet-Kune-Do to the problem area of software development. Hopefully, you find some value in the exercise. I find a lot of value in thinking about it.

Introduction

Development methodologies have been around for as long as I can remember. Each new “fad” promises better results and more productivity for teams using them. Personally, I have never been a part of a team that has implemented “a methodology” that did not run into problems that they could solve within the context of the methodology.

Unfortunately, managers and business users within a software organization are always looking for a “silver bullet” that will take what is ultimately a creative process and turn it into production-like processes that are repeatable and predictable. The basic assumption from these groups is that software development should be like making cars. The problem with this assumption is that each software project is different — and more importantly, each development team is different and unique in and of itself.

Fortunately for these groups, there are other groups of people that create and market “methodologies” that purport to create this productionized process for software development. The idea in most of these implementations are that there are specific things one can do to control the creative process of software development and make it predictable and repeatable.

These complete tool sets are marketed and bought by big businesses and forced on the development teams in order to help them be more productive. More times than not, teams feel limited by the tools, and options that are outside the process cannot be considered within the context of a software process.

Through much of my adult life, I have always enjoyed the writing of Bruce Lee. Over the years I have compared his ideas around “classical forms” of martial arts to the idea of software development and “methodologies”. This article is meant to draw some correlations and spur some thought among the IT managers in the industry to perhaps look at software development a little differently, as a creative, fluid process rather than something completely structured and rigid.

I’d like to take a few topics and talk about how Bruce approached them in the formation of Jeet-Kune-Do, and how these ideas could possibly translate into the “art” of software development and why individual software development methodologies, or styles, may not, in and of themselves, deliver the expected results that were marketed.

The Limited Vocabulary of Methodologies

When there is freedom from mechanical conditioning, there is simplicity. The classical man is just a bundle of routine, ideas and tradition. If you follow the classical pattern, you are understanding the routine, the tradition, the shadow — you are not understanding yourself.

— Bruce Lee

Bruce was constantly frustrated with the “static” nature of martial arts styles. His argument, in a nutshell, was that a specific style taught you a specific way of doing things and that with the limited vocabulary dictated by a particular style, you only had a set number of responses to what is normally a dynamic, ever changing, and most likely random chain of events based on the moment.

Perhaps Bruce could say it better:

Too much horsing around with unrealistic stances and classic forms and rituals is just too artificial and mechanical, and doesn’t really prepare the student for actual combat. A guy could get clobbered while getting into this classical mess. Classical methods like these, which I consider a form of paralysis, only solidify and constrain what was once fluid. Their practitioners are merely blindly rehearsing routines and stunts that will lead nowhere.

I believe that the only way to teach anyone proper self-defence is to approach each individual personally. Each one of us is different and each one of us should be taught the correct form. By correct form I mean the most useful techniques the person is inclined toward. Find his ability and then develop these techniques. I don’t think it is important whether a side kick is performed with the heel higher than the toes, as long as the fundamental principle is not violated. Most classical martial arts training is a mere imitative repetition – a product – and individuality is lost.

When one has reached maturity in the art, one will have a formless form. It is like ice dissolving in water. When one has no form, one can be all forms; when one has no style, he can fit in with any style.

— Bruce Lee

“Classical” methodologies have the same effect as Bruce felt “classicial styles” had. Talk to any team (better yet, ask the management team) and you will find that they develop software by “using RUP”, “being an XP team”, or “using SCRUM” rather than using a set of tools to develop software. The assumption by those above the team in implementing the methodology is that if you aren’t doing whats perscribed, your doing it wrong.

Unfortunately, when using a particular set of tools described as “the way”, one often winds up in situations in which they did not expect and have to find within the methodology the right solution to use, rather than doing what works to get the problem out of the way and get the project moving again. There is also a much greater chance that the team is put into a situation in which some “expert” will tell them that they “did it wrong” because they didn’t do it in accordance with the selected methodology. I’ve seen so much time wasted to figure out the “right” way to do something when everyone already knew what needed to be done, they just had to find a way that it would “fit” into the “official” structure of the team or organization.

The overarching structure of a particular methodology limits the possibilities a team has to act in a way that isn’t in accordance with the structure.

Notice that the stiffest tree is most easily cracked, while the bamboo or willow survives by bending with the wind.

— Bruce Lee

I believe that each development “methodology” is a collection of tools that could be valuable within a particular context. I do not believe that they are all or nothing. It is essential that you, as Bruce would say, “absorb what is useful, discard the rest, and add what is uniquely your own”.

Learning the Basic Tools Enough to Improvise

Within software development, we have a set of tools that we use in order to get our work done. Be it source control, languages, refactoring, test first development, etc, there are intricacies of the tools that we need to know in order to be able to improvise when situations arise that we are not expecting. Rarely, if ever, can we stick to one way of doing things on a project, so unencumbered knowledge of the basic tools are essential.

Again, Bruce addresses this within the philosophy of Jeet-Kune-Do:

I refer to my hands, feet and body as the tools of the trade. The hands and feet must be sharpened and improved daily to be efficient.

It is true that the mental aspect of kung-fu is the desired end; however, to achieve this end, technical skill must come first.

The techniques, though they play an important role in the early stage, should not be too restrictive, complex or mechanical. If we cling to them, we will become bound by their limitation. Remember, you are expressing the technique, and not doing Technique number two, Stance three, Section four?

Practice all movements slow and fast, soft and hard; the effectiveness of Jeet Kune-Do depends on split-second timing and reflexive action, which can be achieved only through repetitious practice.

When performing the movements, always use your imagination. Picture your adversary attacking, and use Jeet Kune-Do techniques in response to this imagined attack. As these techniques become more innate, new meaning will begin to emerge and better techniques can be formulated.

— Bruce Lee

Most of the time when implementing a “methodology”, more focus is put on the structure of the project than learning the tools around the methodology enough to be able to improvise. I like to focus my teams on becoming intimate with the basic tools that they need in order to work effectively.

Perhaps a practical example would help. We implemented Subversion as our version control tool in May of 2004, the day it went to version 1.0 (I would have had a hard time justifying a beta tool to the “enterprise architects“. During the transition to Subversion, I made the rule for the group that no GUI’s would be used with source control. Thats right, no Tortoise SVN, no Rapid SVN, nothing. The team was only allowed to use the command line tool. There was some “resistance” to this idea and to this day I’m not sure that I’ve ever really been able to communicate effectively why I made this rule. It all boils down to the ability to improvise.

I believe that in order to improvise, you have to have intimate knowledge of the tools at your disposal. This intimate knowledge of the capabilities of the core tool would enable each individual team member to aggregate a body of knowledge, internalize it, and be able to improvise when the unexpected occurred. This is one of the reasons why, as I’ve pointed out to my team members a number of times, that I have spent countless hours over my career retyping commands and / or code fragments rather than cutting and pasting – and why I still use emacs rather than the latest cool IDE. When it all comes down to it, repetition is the mother of skill. I believe that this is what Bruce was talking about in the above quote.

For our team, a GUI tool would have tied them to a fixed set of capabilities limited by someone elses interpretation of how the tool worked, rather than the true form of the tool and its capabilities.

Over time, I felt that this intimate knowledge of how the tool worked would enable the team members to generalize the concepts enough that they would be able to find new ways of doing things – that “new meaning will begin to emerge and better techniques can be formulated”. How the team used the tool was up to them, but I wanted them to use the tool in the purest form in order to learn it rather than a filtered version of it.

All or Nothing

When people talk about fighting schools they say that Kung Fu, or Karate, or this other style is the best. That is silly, and the problem becomes that the fighting style then becomes set in stone with no growth, and no adaptation, because what works well with me might not work for you.

— Bruce Lee

I was in a training class recently in which the instructor said “If you are not doing Pair Programming, you are not agile”. His assertion was that you have to use all the tools in the XP “methodology” in order to call yourself an agile team — that you cannot pick and choose the tools that work for the individual teams, but have to take the set as it is and use all parts.

I don’t necessarily agree. Pair programming works in many instances, but I have a hard time believing that this is the only way to do knowledge transfer and to create solid software. If it were, there wouldn’t be such successful distributed teams such as the Subversion Project or many of the different Linux teams (or the kernel team itself). Communication can be heightened without having people sitting next to each other watching or being watched by someone else. Bottom line is, depending on the individual, some might not be comfortable with this tool, and I think thats OK.

Its sad. Management teams overall want production line type processes to create software. They want to be able to predict schedules and everything that could possibly go wrong up front – and they want their predictions to be accurate. While most methodologies do not promise “precognition”, the interpretation by management teams most of the time is that precognition comes with the methodology they are buying. After all, why would they be doing something with a name if they are, more than likely, not going to better off in these areas by doing so?

[ This is where I ended.   After 2000 words I guess I was tired ]

Vacation Learning – PHP and Smarty Templates

I’m on vacation this week and next week. Since I rarely have time to learn anything technical (or blog for that matter anymore), I thought I would take some time during my time off to learn something new around development.

We have a system at work that is essentially a small portal. The core of it was written by me to learn PHP about 8 years ago and has been augmented by me and one other guy at workKeith and I over the years. Over that time, as we added new functionality to it, I used it to experiment with other languages as I was learning it. Other pieces were written in Java out of convenience. In total, we have pieces written in PHP, Java, Python, and PERL.

As I usually use this system to learn new things, I figured it would be a good candidate to use to learn how to use the Smarty templating system for PHP. I became interested in this templating system after working with Eventum over the last few weeks and figured that if I am going to do further work with Eventum, it would be helpful to understand the templating framework it uses.

So I’ve started using the system to take our 8 year old PHP code base and separate some of the presentation logic out. Smarty is pretty flexible and easy to use at a high level (I haven’t gotten into any of the really advanced stuff yet).

Here’s an example of how nicely the use of a templating system simplifies your code. Take this example, which enumerated entries from our internal wiki via an RSS feed into a section on the home page:

function getWikiEntries($url) {
$theHTML = "";

$rss = fetch_rss($url);

$theHTML .= "

"; $theHTML .= "";# foreach over each item in the array.
# displaying simple links$rowCount = 0;
$className = "modifications-evenrow";foreach ($rss->items as $item ) {if (($rowCount % 2) == 0) {
$theHTML .= "
"; } $theHTML .= "# truncate item title to 28 characters
$myTitle = $item['title'];if (strlen($myTitle) > 28 ) {
$myTitle = substr($myTitle, 0, 28) . " ...";
}$theHTML .= $myTitle;
if (($rowCount % 2) == 0) {
$theHTML .= "  ";
} else {
$theHTML .= "";
}
$rowCount++;if ($rowCount == 20):
break;
endif;
}$theHTML .= "

 

“; # get the channel title and link properties off of the rss object # $title = “Recent Wiki Entries”; $link = $rss->channel[‘link’]; #$theHTML .= “$title“; $theHTML .= “$title   “; $theHTML .= “
“; $theHTML .= ““;

“; return($theHTML); }

I’m sure you can appreciate how hard this would be to maintain, and all of the cruft that has accumulated over the years …

Now take the simplified version (sans error checking), written today in about 10 minutes:

function getWikiEntries($url) {
$rss = fetch_rss($url);

$template = new TemplateEngine();

$firstColumn = array_slice($rss->items, 0, 10);
$secondColumn= array_slice($rss->items, 10);

$template->assign("firstColumn", $firstColumn);
$template->assign("secondColumn", $secondColumn);
$template->assign("link", $rss->channel['link']);

return($template->renderString("wikiEntries.tpl"));
}

… along with its corresponding Smarty template:

{section name="entries" loop="$firstColumn"}{/section}

<a title="{$secondColumn[entries].title}" href="{$secondColumn[entries].link}">{$secondColumn[entries].title}</a>
<table width="80%" cellspacing="0" cellpadding="0" align="center">
<tbody>
<tr>
<th class="header-title" colspan="2">Recent Wiki Entries <a href="{$link}"><img src="{$applicationURL}/images/rss.png" alt="" border="0" /></a></th>
</tr>
<tr>
<td class="modifications-sectionheader" colspan="2"></td>
<td class="modifications-data"><a title="{$firstColumn[entries].title}" href="{$firstColumn[entries].link}">{$firstColumn[entries].title|truncate:28:" ..."}</a></td>
</tr>
</tbody>
</table>

I don’t know about you, but I think thats quite a difference in maintainability. I’d much rather modify the html in the template than in the original function. Not only that, but the code is actually code, not a bunch of code with a lot of simply horrid markup stuck in the middle of everything.

I’m pretty impressed with how much I’ve been able to use in a short amount of time this week. The libraries are obviously thought out and ramp up time for me was really minimal. I like libraries like that. It also addresses something that has annoyed me for a long time. Embedded HTML is a pain to maintain and I’ve dreaded going into this over the years just because of that.

At some point, I’ll investigate what it takes to write custom plugins, a functionality that the libraries also support.

I think I’ve been able to get a really good start at getting something maintainable. My goal over the next few of weeks is to templatize the whole system, then start taking the non-PHP pieces of the system and rewrite them in PHP. I’ll also add the ability to change configuration in one place, so that we can cut some of the pain that we have in keeping things maintained down – and perhaps be able to install the application in other places.

Should be fun. I’m definitely feeling productive over the past few days. I’ve always liked working in PHP over other languages. I definitely have to do work like this more often.

Video: How Open Source Projects Survive Poisonous People (And You Can Too)

Since getting a 80G iPod about a month ago two weeks ago, I’ve been really getting into watching the Google Tech Talks on Google Video. I recently watched How Open Source Projects Survive Poisonous People (And You Can Too), a lecture given by Brian Fitzpatrick and Ben Collins-Sussman from the Subversion team (now both Google employees) that summarizes a lot of information in Karl Fogels book Producing Open Source Software: How to Run a Successful Free Software Project.

If you haven’t had time to pick up and read Karls book, this video would be a good primer to some of the concepts in it and could very well motivate you to pick it up. Its an excellent book and one that I thoroughly enjoyed reading.

Jon Udell Screening Room Screencast on IronPython

Episode 8 of Jon Udell’s The Screening Room features Jim Hugunin, creator of Jython and now IronPython, a .NET implementation of Python. For Python fans, this is a fascinating screen-cast to watch. They’ve done some great work on IronPython, which just hit its 1.0 release this week.

Some of the cool things I liked about what I saw:

  • Integration of the Python language with the Visual Studio IDE (including the Debugger)
  • Full access to the .NET framework (The Avalon demo was cool)
  • Optional integration of CLR methods into Python objects (String.Trim() vs. String.strip())
  • The PowerShell demo was also pretty cool
  • In the early part of the screen-cast, Jim takes a benchmark program and increases performance of the program dramatically by refactoring one function into C#. Shows off the optimization opportunities possible with the platform, along with the great integration between IronPython and other .NET languages

IronPython looks like a really fun thing to start playing with and could be a really great addition to a Windows development group for prototyping. I am a big fan of dynamic languages and think this is a great addition to the .NET tool set.