11.3 Bazaar or a Cathedral

Free For All. Go to the Table of Contents. Vist the Gifcom.


When Raymond wrote the essay, he was just trying to suss out the differences between several of the camps in the free source world. He noticed that people running free source projects had different ways of sharing. He wanted to explain which free source development method worked better than others. It was only later that the essay began to take on a more serious target when everyone began to realize that Microsoft was perhaps the biggest cathedral-like development team around.


Raymond said, "I think that like everyone else in the culture I wandered back and forth between the two modes as it seemed appropriate because I didn't have a theory or any consciousness."


He saw Richard Stallman and the early years of the GNU projects as an example of cathedral-style development. These teams would often labor for months if not years before sharing their tools with the world. Raymond himself said he behaved the same way with some of the early tools that he wrote and contributed to the GNU project.


Linus Torvalds changed his mind by increasing the speed of sharing, which Raymond characterized as the rule of "release early and often, delegate everything you can, be open to the point of promiscuity." Torvalds ran Linux as openly as possible, and this eventually attracted some good contributors. In the past, the FSF was much more careful about what it embraced and brought into the GNU project. Torvalds took many things into his distributions and they mutated as often as daily. Occasionally, new versions came out twice a day.


Of course, Stallman and Raymond have had tussles in the past. Raymond is careful to praise the man and say he values his friendship, but also tempers it by saying that Stallman is difficult to work with.
In Raymond's case, he says that he once wanted to rewrite much of the Lisp code that was built into GNU Emacs. Stallman's Emacs allowed any user to hook up their own software into Emacs by writing it in a special version of Lisp. Some had written mail readers. Others had added automatic comment-generating code. All of this was written in Lisp.


Raymond says that in 1992, "The Lisp libraries were in bad shape in a number of ways. They were poorly documented. There was a lot of work that had gone on outside the FSF and I wanted to tackle that project."
According to Raymond, Stallman didn't want him to do the work and refused to build it into the distribution. Stallman could do this because he controlled the Free Software Foundation and the distribution of the software. Raymond could have created his own version, but refused because it was too complicated and ultimately bad for everyone if two versions emerged.


For his part, Stallman explains that he was glad to accept parts of Raymond's work, but he didn't want to be forced into accepting them all. Stallman says, "Actually, I accepted a substantial amount of work that Eric had done. He had a number of ideas I liked, but he also had some ideas I thought were mistaken. I was happy to accept his help, as long as I could judge his ideas one by one, accepting some and declining some.


"But subsequently he asked me to make a blanket arrangement in which he would take over the development of a large part of Emacs, operating independently. I felt I should continue to judge his ideas individually, so I said no."


Raymond mixed this experience with his time watching Torvalds's team push the Linux kernel and used them as the basis for his essay on distributing the Source. "Mostly I was trying to pull some factors that I had observed as unconscious folklore so people could take them out and reason about them," he said.
Raymond says, "Somebody pointed out that there's a parallel of politics. Rigid political and social institutions tend to change violently if they change at all, while ones with more play in them tend to change peacefully."


There is a good empirical reason for the faith in the strength of free source. After all, a group of folks who rarely saw each other had assembled a great pile of source code that was kicking Microsoft's butt in some corners of the computer world. Linux servers were common on the Internet and growing more common every day. The desktop was waiting to be conquered. They had done this without stock options, without corporate jets, without secret contracts, and without potentially illegal alliances with computer manufacturers. The success of the software from the GNU and Linux world was really quite impressive.
Of course, myths can be taken too far. Programming computers is hard work and often frustrating. Sharing the source code doesn't make bugs or problems go away--it just makes it a bit easier for someone else to dig into a program to see what's going wrong. The source code may just be a list of instructions written in a programming language that is designed to be readable by humans, but that doesn't mean that it is easy to understand. In fact, most humans won't figure out most source code because programming languages are designed to be understood by other programmers, not the general population.


To make matters worse, programmers themselves have a hard time understanding source code. Computer programs are often quite complicated and it can take days, weeks, and even months to understand what a strange piece of source code is telling a computer to do. Learning what is happening in a program can be a complicated job for even the best programmers, and it is not something that is taken lightly.


While many programmers and members of the open source world are quick to praise the movement, they will also be able to cite problems with the myth of the Source. It isn't that the Source doesn't work, they'll say, it's just that it rarely works anywhere near as well as the hype implies. The blooms are rarely as vigorous and the free markets in improvements are rarely as liquid.


Larry McVoy, an avid programmer, proto-academic, and developer of the BitKeeper toolkit, likes to find fault with the model. It isn't that he doesn't like sharing source code, it's just that he isn't wealthy enough to take on free software projects. "We need to find a way for people to develop free software and pay their mortgages and raise a family," he says.


"If you look closely," he says, "there really isn't a bazaar. At the top it's always a one-person cathedral. It's either Linus, Stallman, or someone else." That is, the myth of a bazaar as a wide-open, free-for-all of competition isn't exactly true. Sure, everyone can download the source code, diddle with it, and make suggestions, but at the end of the day it matters what Torvalds, Stallman, or someone else says. There is always a great architect of Chartres lording it over his domain.


Part of this problem is the success of Raymond's metaphor. He said he just wanted to give the community some tools to understand the success of Linux and reason about it. But his two visions of a cathedral and a bazaar had such a clarity that people concentrated more on dividing the world into cathedrals and bazaars. In reality, there's a great deal of blending in between. The most efficient bazaars today are the suburban malls that have one management company building the site, leasing the stores, and creating a unified experience. Downtown shopping areas often failed because there was always one shop owner who could ruin an entire block by putting in a store that sold pornography. On the other side, religion has always been something of a bazaar. Martin Luther effectively split apart Christianity by introducing competition. Even within denominations, different parishes fight for the hearts and souls of people.
The same blurring holds true for the world of open source software. The Linux kernel, for instance, contains many thousands of lines of source code. Some put the number at 500,000. A few talented folks like Alan Cox or Linus Torvalds know all of it, but most are only familiar with the corners of it that they need to know. These folks, who may number in the thousands, are far outnumbered by the millions who use the Linux OS daily.


It's interesting to wonder if the ratio of technically anointed to blithe users in the free source world is comparable to the ratio in Microsoft's dominion. After all, Microsoft will share its source code with close partners after they sign some non-disclosure forms.[^9] While Microsoft is careful about what it tells its partners, it will reveal information only when there's something to gain. Other companies have already jumped right in and started offering source code to all users who want to look at it.


[9]: At this writing, Microsoft has not released its source code, but the company is widely known to be examining the option as part of its settlement with the Department of Justice. Answering this question is impossible for two different reasons. First, no one knows what Microsoft reveals to its partners because it keeps all of this information secret, by reflex. Contracts are usually negotiated under non-disclosure, and the company has not been shy about exploiting the power that comes from the lack of information.
Second, no one really knows who reads the Linux source code for the opposite reason. The GNU/Linux source is widely available and frequently downloaded, but that doesn't mean it's read or studied. The Red Hat CDs come with one CD full of pre-compiled binaries and the second full of source code. Who knows whoever pops the second CDROM in their computer? Everyone is free to do so in the privacy of their own cubicle, so no records are kept.


If I were to bet, I would guess that the ratios of cognoscenti to uninformed users in the Linux and Microsoft worlds are pretty close. Reading the Source just takes too much time and too much effort for many in the Linux world to take advantage of the huge river of information available to them.
If this is true or at least close to true, then why has the free source world been able to move so much more quickly than the Microsoft world? The answer isn't that everyone in the free source world is using the Source, it's that everyone is free to use it. When one person needs to ask a question or scratch an itch, the Source is available with no questions asked and no lawyers consulted. Even at 3:00 A.M., a person can read the Source. At Microsoft and other corporations, they often need to wait for the person running that division or section to give them permission to access the source code.


There are other advantages. The free source world spends a large amount of time keeping the source code clean and accessible. A programmer who tries to get away with sloppy workmanship and bad documentation will pay for it later as others come along and ask thousands of questions.


Corporate developers, on the other hand, have layers of secrecy and bureaucracy to isolate them from questions and comments. It is often hard to find the right programmer in the rabbit warren of cubicles who has the source code in the first place. One Microsoft programmer was quoted as saying, "A developer at Microsoft working on the OS can't scratch an itch he's got with Excel, neither can the Excel developer scratch his itch with the OS--it would take him months to figure out how to build and debug and install, and he probably couldn't get proper source access anyway."


This problem is endemic to corporations. The customers are buying the binary version, not the source code, so there is no reason to dress up the backstage wings of the theater. After some time, though, people change cubicles, move to other corporations, and information disappears. While companies try to keep source code databases to synchronize development, the efforts often fall apart. After Apple canceled development of their Newton handheld, many Newton users were livid. They had based big projects on the platform and they didn't want to restart their work. Many asked whether Apple could simply give away the OS's source code instead of leaving it to rot on some hard disk. Apple dodged these requests, and this made some people even more cynical. One outside developer speculated, "It probably would not be possible to re-create the OS. The developers are all gone. All of them went to Palm, and they probably couldn't just put it back together again if they wanted to."


Of course, corporations try to fight this rot by getting their programmers to do a good job at the beginning and write plenty of documentation. In practice, this slips a bit because it is not rewarded by the culture of secrecy. I know one programmer who worked for a project at MIT. The boss thought he was being clever by requiring comments on each procedure and actually enforcing it with an automated text-scanning robot that would look over the source code and count the comments. My friend turned around and hooked up one version of the popular artificial intelligence chatterbots like Eliza and funneled the responses into the comment field. Then everyone was happy. The chatterbot filled the comment field, the automated comment police found something vaguely intelligent, and the programmer got to spend his free time doing other things. The boss never discovered the problem.


Programmers are the same the world over, and joining the free source world doesn't make them better people or destroy their impudence. But it does penalize them if others come along and try to use their code. If it's inscrutable, sloppy, or hard to understand, then others will either ignore it or pummel them with questions. That is a strong incentive to do it right.