11.1 The Bishop

Free For All. Go to the Table of Contents. Vist the Gifcom.


Eric Raymond, a man who is sort of the armchair philosopher of the open source world, did a great job of summarizing the phenomenon and creating this myth in his essay "The Cathedral and the Bazaar." Raymond is an earnest programmer who spent some time working on projects like Stallman's GNU Emacs. He saw the advantages of open source development early, perhaps because he's a hard-core libertarian. Government solutions are cumbersome. Empowering individuals by not restraining them is great. Raymond comes off as a bit more extreme than other libertarians, in part because he doesn't hesitate to defend the second amendment of the U.S. Constitution as much as the first. Raymond is not ashamed to support widespread gun ownership as a way to further empower the individual. He dislikes the National Rifle Association because they're too willing to compromise away rights that he feels are absolute.


Some people like to call him the Margaret Mead of the free source world because he spent some time studying and characterizing the culture in much the same way that Mead did when she wrote Coming of Age in Samoa. This can be a subtle jab because Margaret Mead is not really the same intellectual angel she was long ago. Derek Freeman and other anthropologists raise serious questions about Mead's ability to see without bias. Mead was a big fan of free love, and many contend it was no accident that she found wonderful tales of unchecked sexuality in Samoa. Freeman revisited Samoa and found it was not the guilt-free land of libertine pleasures that Mead described in her book. He documented many examples of sexual restraint and shame that Mead apparently missed in her search for a paradise.


Raymond looked at open source development and found what he wanted to find: the wonderful efficiency of unregulated markets. Sure, some folks loved to label Richard Stallman a communist, a description that has always annoyed Stallman. Raymond looked a bit deeper and saw that the basis of the free software movement's success was the freedom that gave each user the complete power to change and improve their software. Just as Sigmund Freud found sex at the root of everything and Carl Jung uncovered a battle of animus and anima, the libertarian found freedom.


Raymond's essay was one of the first to try to explain why free source efforts can succeed and even prosper without the financial incentives of a standard money-based software company. One of the biggest reasons he cited was that a programmer could "scratch an itch" that bothered him. That is, a programmer might grow annoyed by a piece of software that limited his choices or had an annoying glitch. Instead of cursing the darkness in the brain cavity of the corporate programmer who created the problem, the free source hacker was able to use the Source to try to find the bug.


Itch-scratching can be instrumental in solving many problems. Some bugs in software are quite hard to identify and duplicate. They only occur in strange situations, like when the printer is out of paper and the modem is overloaded by a long file that is coming over the Internet. Then, and only then, the two buffers may fill to the brim, bump into each other, and crash the computer. The rest of the time, the program floats along happily, encountering no problems.


These types of bugs are notoriously hard for corporate testing environments to discover and characterize. The companies try to be diligent by hiring several young programmers and placing them in a room with a computer. The team beats on the software all day long and develops a healthy animosity toward the programming team that has to fix the problems they discover. They can nab many simple bugs, but what happens if they don't have a printer hooked up to their machine? What happens if they aren't constantly printing out things the way some office users are? The weird bug goes unnoticed and probably unfixed.
The corporate development model tries to solve this limitation by shipping hundreds, thousands, and often hundreds of thousands of copies to ambitious users they called "beta testers." Others called them "suckers" or "free volunteers" because once they finish helping develop the software, they get to pay for it. Microsoft even charges some users for the pleasure of being beta testers. Many of the users are pragmatic. They often have no choice but to participate in the scheme because they often base their businesses on some of the software shipped by these companies. If it didn't work, they would be out of a job.


While this broad distribution of beta copies is much more likely to find someone who is printing and overloading a modem at the same time, it doesn't give the user the tools to help find the problem. Their only choice is to write an e-mail message to the company saying "I was printing yesterday and your software crashed." That isn't very helpful for the engineer, and it's no surprise that many of these reports are either ignored or unsolved.


Raymond pointed out that the free source world can do a great job with these nasty bugs. He characterized this with the phrase, "Given enough eyeballs, all bugs are shallow," which he characterized as "Linus's Law." That is, eventually some programmer would start printing and using the Internet at the same time. After the system crashed a few times, some programmer would care enough about the problem to dig into the free source, poke around, and spot the problem. Eventually somebody would come along with the time and the energy and the commitment to diagnose the problem. Raymond named this "Linus's Law" after Linus Torvalds. Raymond is a great admirer of Torvalds and thinks that Torvalds's true genius was organizing an army to work on Linux. The coding itself was a distant second.


Of course, waiting for a user to find the bugs depended on there being someone with enough time and commitment. Most users aren't talented programmers, and most have day jobs. Raymond and the rest of the free source community acknowledge this limitation, but point out that the right person often comes along if the bug occurs often enough to be a real problem. If the bug is serious enough, a non-programmer may even hire a programmer to poke into the source code.


Waiting for the bug and the programmer to find each other is like waiting for Arthur to find the sword in the stone. But Raymond and the rest of the free source community have even turned this limitation on its head and touted it as an advantage. Relying on users to scratch itches means that problems only get addressed if they have real constituencies with a big enough population to generate the one true believer with enough time on his hands. It's sort of a free market in people's time for fixing bugs. If the demand is there, the solution will be created. It's Say's Law recast for software development: "the supply of bugs creates the talent for fixes."


Corporate development, on the other hand, has long been obsessed with adding more and more features to programs to give people enough reason to buy the upgrade. Managers have long known that it's better to put more time into adding more doohickeys and widgets to a program than into fixing its bugs. That's why Microsoft Word can do so many different things with the headers and footers of documents but can't stop a Word Macro virus from reproducing. The folks at Microsoft know that when the corporate managers sit down to decide whether to spend the thousands of dollars to upgrade their machines, they'll need a set of new compelling features. People don't like to pay for bug fixes.


Of course, corporations also have some advantages. Money makes sure that someone is actively trying to solve the bugs in the program. The same free market vision guarantees that the companies that consistently disappoint their customers will go out of business. This developer has the advantage of studying the same source code day in and day out. Eventually he'll learn enough about the guts of the Source to be much more effective than the guy with the jammed printer and modem. He should be able to nab the bug 10 times more quickly then the free source hobbyist just because he's an expert in the system.


Raymond acknowledges this problem but proposes that the free source model can still be more effective despite the inexperience of the people who are forced to scratch an itch. Again he taps the world of libertarian philosophy and suggests that the free software world is like a bazaar filled with many different merchants offering their wares. Corporate development, on the other hand, is structured like the religious syndicates that built the medieval cathedrals. The bazaars offered plenty of competition but no order. The cathedrals were run by central teams of priests who tapped the wealth of the town to build the vision of one architect.


The differences between the two were pretty simple. The cathedral team could produce a great work of art if the architect was talented, the funding team was successful, and the management was able to keep everyone focused on doing their jobs. If not, it never got that far. The bazaar, on the other hand, consisted of many small merchants trying to outstrip each other. The best cooks ended up with the most customers. The others soon went out of business.


The comparison to software was simple. Corporations gathered the tithes, employed a central architect with a grand vision, managed the team of programmers, and shipped a product every once and a bit. The Linux world, however, let everyone touch the Source. People would try to fix things or add new features. The best solutions would be adopted by oth ers and the mediocre would fall by the wayside. Many different Linux versions would proliferate, but over time the marketplace of software would coalesce around the best standard version.


"In the cathedral-builder view of programming, bugs and development problems are tricky, insidious, deep phenomena. It takes months of scrutiny by a dedicated few to develop confidence that you've winkled them all out. Thus the long release intervals, and the inevitable disappointment when long-awaited releases are not perfect," Raymond said.


"In the bazaar view, on the other hand, you assume that bugs are generally shallow phenomena--or, at least, that they turn shallow pretty quick when exposed to a thousand eager code-developers pounding on every single new release. Accordingly you release often in order to get more corrections, and as a beneficial side effect you have less to lose if an occasional botch gets out the door."