Review of The Wisdom of Crowds by James Surowiecki

The Wisdom of CrowdsOne of the big trends in the tech and business communities for the past couple of years has been to begin to rethink sociological phenomenon. This is due to the great advances in communication technology in the past few years. During the Internet boom, the business and high tech communities were mostly focused on the emerging technologies such as distributed computing. The business interest in technology is focusing less on Moore’s Law and moving more towards the sociological and organizational impacts of technology. While academia has been focused on this for much longer, business books like this one have recently been gaining momentum.

The Wisdom of Crowds offers a nice introduction to relevant concepts in sociology and game theory and discusses them from a business perspective. While there may not be a whole lot of new information for readers already familiar with these topics, Surowiecki’s easy writing style is enjoyable and will appeal to a broad range of business decision makers.

One of the most beneficial new ideas to come out of this book is the addition of a fourth class of individuals affecting social epidemics, as described in Malcolm Gladwell’s book The Tipping Point: How Little Things Can Make a Big Difference. Gladwell identified the Salesman, Maven, and Connector as classes of individuals who start epidemics. Surowiecki identifies a class of individuals who can potentially disrupt an epidemic by merely by offering an independent voice. He doesn’t give a name to this class of individuals, but they could perhaps accurately be called Disruptors.

The title, derived from an influential nineteenth century book on large group dynamics called Extraordinary Popular Delusions & the Madness of Crowds by Charles Mackay, is a bit of a misnomer since the “crowds” to which Surowiecki refers can be defined as groups of individual agents working in an independent and more or less isolated fashion. Crowds, by contrast, are large groups that are acutely aware of each other’s presence and highly reactive, thus forming something of a single organism. The author’s foundational assumption, then, is that individuals can, under the right circumstances, express an individual agency that is not determined by crowd dynamics, in Mackay’s sense.

From an academic perspective, the populist crowd wisdom presented in The Wisdom of Crowds seems to be an extension, or perhaps an evolution, of the Hegelian concept of “thesis-antithesis-synthesis”, where the conclusion drawn from the reconciliation of two contradictory thoughts becomes a higher thought that transcends both original thoughts. While Surowiecki does not provide concise logical arguments concerning the notions of realism or social determinism, he does provide several pragmatic examples to back up his assumptions.

Surowiecki does a nice job of discussing effective management of these groups of individuals by creating a dichotomy of centralization versus aggregation. Centralization, according to Surowiecki, can either lose the benefits of collective wisdom by relying on a limited point of view, or worse, can actually destroy collective wisdom in groups through authoritarianism.

There is and important aspect of effective individual collectives that Surowiecki fails to fully explore. In each of Surowiecki’s examples, each group shares a common social context. Collective “wisdom”, by Surowiecki’s definition, can be described as merely a reflection of the prejudices and shared social reality of a collection of individuals (by social reality, I am referring to the idea that the way we view the world is highly dependent on the culture around us).

Diversity, for the author, is central to the effectiveness of collective wisdom. This wisdom, however, is dependent on a relevant social context, thus requiring a certain amount of homogeneity within the group. The author acknowledges this only one time in the book, when discussing Schelling points:

“…The existence of Schelling points suggests that people’s experiences of the world are often surprisingly similar, which makes successful coordination easier. After all, it would not be possible for two people to meet at Grand Central Station unless Grand Central represented roughly the same thing to both of them… The reality Schelling’s students shared was, of course, cultural. If you put pairs of people from Manchuria down in the middle of New York City and told them to meet each other, it’s unlikely any of them would manage to meet. But that fact that the shared reality is cultural makes it no less real. (p. 92)”

While diversity is important to collective wisdom, it only makes sense within certain parameters. For instance, the NASA Challenger discussions did lack a diversity of individual contributions, but that diversity was only relevant within a certain context. NASA’s findings would not have been the better for taking a random poll of non-NASA employees. The collective wisdom in this case only made sense within the context of highly specialized engineers. What we must now consider is the amount of diversity required to produce a “wise” result. While Surowiecki is quick to point out that the lack of diversity of perspective can be detrimental to collective wisdom, the over-abundance of diversity can result in arbitrary or faulty data.

Of course, now the discussion revolves around what is meant by the terms “wisdom” and “diversity”. Different situations require different kinds of solutions and different levels of diversity. This is an area that would be very beneficial to explore, especially in the context of the Web (the impact and usage of folksonomy, for instance). Unfortunately, Surowiecki offers very little discussion on these points. Surowiecki does allude several times to the fact that individuals likely to be involved in a particular collective have an adequate incentive to participate. While Surowiecki does not explicitly point this out, he assumes, probably fairly correctly, that people likely to participate in something like a financial or information market have placed enough investment (time, reputation, money, etc.) in that market that they will have a strong desire to attempt informed judgments; these investments should, according to Surowiecki, be enough to raise the bar of entry enough to create a system of self-filtration.

Pragmatically, Surowiecki has provided enough examples to create a workable hypothesis that business leaders and policy makers can use. As business books go, The Wisdom of Crowds is an intelligent and interesting read, well suited to a broad audience of business professionals.

Current Reading Queue

I’ve not been very motivated to read lately for some reason, but I have still been buying books – a very dangerous thing when I’m not motivated to actually read them. So far, I’ve accrued the following books, which I hope to read in the order presented here:

  1. Unite the Tribes: Ending Turf Wars for Career and Business Success by Christopher Duncan
  2. Crossing the Chasm by Geoffrey A. Moore
  3. God’s Debris : A Thought Experiment by Scott Adams

I’m in a huge reading slump. I’ve read so much this year that it is no longer a relaxing thing. Hopefully that will change soon.

Extremely Busy This Week

I realized as I was sitting here working this morning after getting up at around 4:30a that its already Wednesday and I haven’t really posted anything since Sunday.

I’ve been extremely busy with work this week. Additionally, I picked up a book called The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed Our Culture on the recommendation of a friend and have spent all available free time reading it. Its a facinating book. While I’m only half way through it, I feel very comfortable recommending it right now. The best way to explain it is the Hard Drive : Bill Gates and the Making of the Microsoft Empire of the search industry.

As I get some time this week, I also want to write up something on another book I read last week called The E-Myth Revisited: Why Most Small Businesses Don’t Work and What to Do About It, in which the author explains what they call the Franchise Prototype and the factors involved in creating a small business. Additionally, he explains why most small businesses started by ‘Technicians’ fail. Its a really great read. Pick it up if you get a chance.

Ok, enough writing, back to the office. I’ve got my iPod loaded and a posting on the web site. Time to get some work done!

The Cluetrain Manifesto – Still Reading it, but WOW.

I’m still working my way through The Cluetrain Manifesto: The End of Business As Usual (also available on cluetrain.com) and I’m finding it an extremely exciting text to read.

The book does a really great job of explaining what the web means to business culture as a whole. Unfortunately, as I look at it I see that while its first publication was in 2000, companies are obviously still stuck in their ruts as to how they think they should be run.

Last night while dinner was cooking I was reading the chapter by David Weinberger entitled The Hyperlinked Organization. In it, David outlines ten bullet points in which the “Company” does in order to accomplish certain things while the reality is that the company produces the exact opposite result. This section hit me really hard, because it strikes me as absolutely true. I’m hoping that my reproducing these isn’t a problem, because I think that the text deserves the additional exposure.

Here are the ten bullets outlined in The Hyperlinked Organization:

  • The company communicates with me through a newsletter and company meetings meant to lift up my morale. In fact, I know from my e-mail pen pals that it’s telling me happy-talk lies, and I find that quite depressing.
  • The company org chart shows me who does what so I know how to get things done. In fact, the org chart is an expression of a power structure. It is red tape. It is a map of whom to avoid.
  • The company manages my work to make sure that all tasks are coordinated and the company is operating efficiently. In fact, the inflexible goals imposed from on high keep me from following what my craft expertise tells me I really ought to be doing.
  • The company provides me with a career path so I’ll see a productive future in the business. In fact, I’ve figured out that because the org chart narrows at the top, most career paths necessarily have to be dead ends.
  • The company provides me with all the information I need to make good decisions. In fact, this information is selected to support a decision (or worldview) in which I have no investment. Statistics and industry surveys are lobbed like anti-aircraft fire to disguise the fact that while we have lots of data, we have no understanding.
  • The company is goal-oriented so that the path from here to there is broken into small, well-marked steps that can be tracked and managed. In fact, if I keep my head down and accomplish my goals, I won’t add the type of value I’m capable of. I need to browse. I even need to play. Without play, only Shit Happens. With play, Serendipity Happens.
  • The company gives me deadlines so that we ship product on time, maintaining our integrity. In fact, working to arbitrary deadlines makes me ship poor-quality content. My management doesn’t have to use a club to get me to do my job. Where’s the trust, baby?
  • The company looks at customers as adversaries who must be won over. In fact, the ones I’ve been exchanging e-mail with are very cool and enthusiastic about exactly the same thing that got me into this company. You know, I’d rather talk with them than with my manager.
  • The company works in an office building in order to bring together all of the things I need to get my job done and to avoid distracting me. In fact, more and more of what I need is outside the corporate walls. And when I really want to get something done, I go home.
  • The company rewards me for being a professional who acts and behaves in a, well, professional manner, following certain unwritten rules about the coefficient of permitted variation in dress, politics, shoe style, expression of religion, and the relating of humorous stories. In fact, I learn who to trust — whom I can work with creatively and productively — only by getting past the professional act.

This very accurately describes the corporate environment as I have experienced it, and its a sad, sad thing. As a matter of fact, I still remember the look of puzzlement I received at one company when I had asserted in a meeting that for development teams to be productive they have to have space to “play and make mistakes” without consequences. I received a look like I was from another planet. This was in response to a request to begin measuring defects on work in progress (pre-integration or QA) in order to measure developer productivity. Yes, that’s right, measuring defects on work that is still in major development. I never put this into action.

I’ve worked for some companies that I hated, due to their “factory” mentality of software development. One company that I despised working for I have a new respect for nowadays, because they actually had a newsgroup on UUNET (back in the 90’s) in which the development and technical support staff were allowed to contribute freely to. The lowly development staff were actually allowed to interact freely with customers. I always thought that that was a really cool thing for a customer – to have a problem with a piece of software that I had written and get answered by the guy who wrote it to either receive a way to work around it or be notified as soon as a patch made it to the tech support area of the web site. I haven’t worked for a company that understood the importance of this concept since then.

So what am I getting out of this book? I am getting confirmation, first and foremost, that I am not some nut with unrealistic ideas about the effects that the internet has on business and customers reactions to them. I am getting affirmation of the belief that if you just think about the experience you want on the internet as a customer, you can completely change the way your customers think about you.

And I’m learning that its the little things that create revenue opportunity, many of which you don’t make money with, but because they are base expectations. They are a barrier to entry for customers if you do not have them. One of these, I believe, is the conversations with real people rather than a corporate entity.

One thing this book obviously does is make you think — at least enough to brain dump some very disconnected but long dwelled upon concepts into your Labor Day Sunday blog entry.

Current Books on the Reading List

I’ve currently seen in increase in the reading queue. I just finished The Tipping Point: How Little Things Can Make a Big Difference by Malcolm Gladwell, and it was great. I hope to post a review on this one soon.

After that, I picked up Blink : The Power of Thinking Without Thinking by the same author. I got halfway through that (also very good by the way) when Tom The Architect hit me with another recommendation called The Cluetrain Manifesto: The End of Business as Usual by Rick Levine, Christopher Locke, Doc Searls (also in the Gillmor Gang) and David Weinberger. This one looked intriguing and so I set aside Blink to read this one.

This book is also available for free via an online edition, but for some reason I need the physical book lying around. This is some interesting stuff and even though I’m only a small way through the book I can already say this is something I would recommend those in business (and especially management) read.

Big Macs vs. The Naked Chef

I started reading the Joel on Software book yesterday after seeing it at Barnes and Noble and finding my self unable to put it back on the shelf.

This morning I hit Chapter 33, an article entitled Big Macs vs. The Naked Chef, a brilliant article on the effect of methodologies on a software development shop.

The book is full of great stuff like this, and I’m really enjoying being able to sit down and read it all at one time without clicking and scrolling.

If you haven’t picked up this book yet, I would highly recommend it. If you’re too cheap to buy it, hit his website, where all of the articles are available for free.

Its so nice to see a software development author out there with a healthy dose of common sense!

Sarzo’s Off The Rails Book Update

I received an email from the Diary of an Axeman Club forum, referring to another forum in which the moderator posted this message:

GOOD NEWS!!!

Just heard back from Rudy and he says the book is only delayed.
He says Amazon made a poor choice in wording the emails.
He appreciates everyone’s interest and support here.

The full message thread can be found on the randyrhoads.tk board. This board, according to the info in its rules section, is also frequented by members of Randy’s family and others like Kevin Dubrow, Kelli Garni (former bassist for Quiet Riot and Randys childhood friend), and Rudy Sarzo, so the source seems credible.

This is good news for Randy fans. I know I’ve been looking forward to the book since it was announced.

Off the Rails now Off the Press?

In May I reported that Rudy Sarzo had a book coming out called ” Off the Rails – My Adventures in the Land of Ozz”. It was due to be released in September and was available for preorder when I found it on Amazon. So I preordered it.

This morning, I received the following from Amazon:

We are sorry to report that the release of the following item has been
cancelled:

Rudy Sarzo “Off the Rails – My Adventures in the Land of Ozz”

Though we had expected to be able to send this item to you, we’ve
since found that it will not be released after all. Please accept
our sincerest apologies for the inconvenience we have caused you.

We have cancelled this item from your order.

Your credit card will NOT BE CHARGED for this item because you only
pay for items when we ship them to you.

Your order is now closed.

I’ve looked around for announcements as to why the book is not being published but can’t find any articles yet. There isn’t even an announcement on Rudy’s site. If you run into any, please send them over.

I’m sure there are a ton of Rhoads fans that were really looking forward to this one. This would have been the only book to chronicle Randy’s life, up to and including the night of his death.

Needless to say, for me, and I’m sure for all Rhoads fans around the world, this is pretty disappointing.