18.5 A Fork, a Split, and a Reunion

Free For All. Go to the Table of Contents. Vist the Gifcom.


Now, after all of the nasty stories of backstabbing and bickering, it is important to realize that there are actually some happy stories of forks that merge back together. One of the best stories comes from the halls of an Internet security company, C2Net, that dealt with a fork in a very peaceful way.


C2Net is a Berkeley-based company run by some hard-core advocates of online privacy and anonymity. The company began by offering a remailing service that allowed people to send anonymous e-mails to one another. Their site would strip off the return address and pass it along to the recipient with no trace of who sent it. They aimed to fulfill the need of people like whistleblowers, leakers, and other people in positions of weakness who wanted to use anonymity to avoid reprisals.


The company soon took on a bigger goal when it decided to modify the popular Apache web server by adding strong encryption to make it possible for people to process credit cards over the web. The technology, known as SSL for "secure sockets layer," automatically arranged for all of the traffic between a remote web server and the user to be scrambled so that no one could eavesdrop. SSL is a very popular technology on the web today because many companies use it to scramble credit card numbers to defeat eavesdroppers.


C2Net drew a fair deal of attention when one of its founders, Sameer Parekh, appeared on the cover of Forbes magazine with a headline teasing that he wanted to "overthrow the government." In reality, C2Net wanted to move development operations overseas, where there were no regulations on the creation of cryptographically secure software. C2Net went where the talent was available and priced right.


In this case, C2Net chose a free version of SSL written by Eric Young known as SSLeay. Young's work is another of the open source success stories. He wrote the original version as a hobby and released it with a BSD-like license. Everyone liked his code, downloaded it, experimented with it, and used it to explore the boundaries of the protocol. Young was just swapping code with the Net and having a good time.


Parekh and C2Net saw an opportunity. They would merge two free products, the Apache web server and Young's SSLeay, and make a secure version so people could easily set up secure commerce sites for the Internet. They called this product Stronghold and put it on the market commercially.


C2Net's decision to charge for the software rubbed some folks the wrong way. They were taking two free software packages and making something commercial out of them. This wasn't just a fork, it seemed like robbery to some. Of course, these complaints weren't really fair. Both collections of code emerged with a BSD-style license that gave everyone the right to create and sell commercial additions to the product. There wasn't any GPL-like requirement that they give back to the community. If no one wanted a commercial version, they shouldn't have released the code with a very open license in the first place.
Parekh understands these objections and says that he has weathered plenty of criticism on the internal mailing lists. Still, he feels that the Stronghold product contributed a great deal to the strength of Apache by legitimizing it.


"I don't feel guilty about it. I don't think we've contributed a whole lot of source code, which is one of the key metrics that the people in the Apache group are using. In my perspective, the greatest contribution we've made is market acceptance," he said.


Parekh doesn't mean that he had to build market acceptance among web developers. The Apache group was doing a good job of accomplishing that through their guerrilla tactics, excellent product, and free price tag. But no one was sending a message to the higher levels of the computer industry, where long-term plans were being made and corporate deals were being cut. Parekh feels that he built first-class respectability for the Apache name by creating and supporting a first-class product that big corporations could use successfully. He made sure that everyone knew that Apache was at the core of Stronghold, and people took notice.


Parekh's first job was getting a patent license from RSA Data Security. Secure software like SSL relies on the RSA algorithm, an idea that was patented by three MIT professors in the 1970s. This patent is controlled by RSA Data Security. While the company publicized some of its licensing terms and went out of its way to market the technology, negotiating a license was not a trivial detail that could be handled by some free software team. Who's going to pay the license? Who's going to compute what some percentage of free is? Who's going to come up with the money? These questions are much easier to answer if you're a corporation charging customers to buy a product. C2Net was doing that. People who bought Stronghold got a license from RSA that ensured they could use the method without being sued.


The patent was only the first hurdle. SSL is a technology that tries to bring some security to web connections by encrypting the connections between the browser and the server. Netscape added one feature that allows a connection to be established only if the server has a digital certificate that identifies it. These certificates are only issued to a company after it pays a fee to a registered certificate agent like Verisign.


In the beginning, certificate agents like Verisign would issue the certificates only for servers created by big companies like Netscape or Microsoft. Apache was just an amorphous group on the Net. Verisign and the other authorities weren't paying attention to it.


Parekh went to them and convinced them to start issuing the certificates so he could start selling Stronghold.


"We became number three, right behind Microsoft and Netscape. Then they saw how much money they were making from us, so they started signing certificates for everyone," he said. Other Apache projects that used SSL found life much easier once Parekh showed Verisign that there was plenty of money to be made from folks using free software.


Parekh does not deny that C2Net has not made many contributions to the code base of Apache, but he doesn't feel that this is the best measure. The political and marketing work of establishing Apache as a worthwhile tool is something that he feels may have been more crucial to its long-term health. When he started putting money in the hands of Verisign, he got those folks to realize that Apache had a real market share. That cash talked.


The Stronghold fork, however, did not make everyone happy. SSL is an important tool and someone was going to start creating another free version. C2Net hired Eric Young and his collaborator Tim Hudson and paid them to do some work for Stronghold. The core version of Young's original SSLeay stayed open, and both continued to add bug fixes and other enhancements over time. Parekh felt comfortable with this relationship. Although Stronghold was paying the salaries of Young and Hudson, they were also spending some of their spare time keeping their SSLeay toolkit up to date.


Still, the notion of a free version of SSL was a tempting project for someone to undertake. Many people wanted it. Secure digital commerce demanded it. There were plenty of economic incentives pushing for it to happen. Eventually, a German named Ralf S. Engelschall stepped up and wrote a new version he called modSSL. Engelschall is a well-regarded contributor to the Apache effort, and he has written or contributed to a number of different modules that could be added to Apache. He calls one the "all-dancing-all-singing modrewrite module" for handling URLs easily.


Suddenly, Engelschall's new version meant that there were dueling forks. One version came out of Australia, where the creators worked for a company selling a proprietary version of the code. C2Net distributed the Australian version and concentrated on making their product easy to install. The other came out of Europe, distributed for free by someone committed to an open source license. The interface may have been a bit rougher, but it didn't cost any money and it came with the source code. The potential for battle between SSLeay and modSSL could have been great.


The two sides reviewed their options. Parekh must have felt a bit frustrated and at a disadvantage. He had a company that was making a good product with repeat buyers. Then an open source solution came along. C2Net's Stronghold cost money and didn't come with source code, while Engelschall's modSSL cost nothing and came with code. Those were major negatives that he could combat only by increasing service. When Engelschall was asked whether his free version was pushing C2Net, he sent back the e-mail with the typed message, "[grin]."


In essence, C2Net faced the same situation as many major companies like Microsoft and Apple do today. The customers now had a viable open source solution to their problems. No one had to pay C2Net for the software. The users in the United States needed a patent license, but that would expire in late 2000. Luckily, Parekh is a true devotee to the open source world, even though he has been running a proprietary source company for the last several years. He looked at the problem and decided that the only way to stay alive was to join forces and mend the fork.


To make matters worse, Hudson and Young left C2Net to work for RSA Data Security. Parekh lost two important members of his team, and he faced intense competition. Luckily, his devotion to open source came to the rescue. Hudson and Young couldn't take back any of the work they did on SSLeay. It was open source and available to everyone.


Parekh, Engelschall, several C2Net employees, and several others sat down (via e-mail) and created a new project they called OpenSSL. This group would carry the torch of SSLeay and keep it up-to-date. Young and Hudson stopped contributing and devoted their time to creating a commercial version for RSA Data Security.


Parekh says of the time, "Even though it was a serious setback for C2Net to have RSA pirate our people, it was good for the public. Development really accelerated when we started OpenSSL. More people became involved and control became less centralized. It became more like the Apache group. It's a lot bigger than it was before and it's much easier for anyone to contribute."


Parekh also worked on mending fences with Engelschall. C2Net began to adopt some of the modSSL code and blend it into their latest version of Stronghold. To make this blending easier, C2Net began sending some of their formerly proprietary code back to Engelschall so he could mix it with modSSL by releasing it as open source. In essence, C2Net was averting a disastrous competition by making nice and sharing with this competitor. It is a surprising move that might not happen in regular business.


Parekh's decision seems open and beneficent, but it has a certain amount of self-interest behind it. He explains, "We just decided to contribute all of the features we had into modSSL so we could start using modSSL internally, because it makes our maintenance of that easier. We don't have to maintain our own proprietary version of modSSL. Granted, we've made the public version better, but those features weren't significant."


This mixing wasn't particularly complicated--most of it focused on the structure of the parts of the source code that handle the interface. Programmers call these the "hooks" or the "API." If Stronghold and modSSL use the same hook structure, then connecting them is a piece of cake. If Engelschall had changed the hook structure of modSSL, then the C2Net would have had to do more work.


The decision to contribute the code stopped Engelschall from doing the work himself in a way that might have caused more grief for C2Net. "He was actually planning on implementing them himself, so we were better off contributing ours to avoid compatibility issues," says Parekh. That is to say, Parekh was worried that Engelschall was going to go off and implement all the features C2Net used, and there was a very real danger that Engelschall would implement them in a way that was unusable to Parekh. Then there would be a more serious fork that would further split the two groups. C2Net wouldn't be able to borrow code from the free version of OpenSSL very easily. So it decided to contribute its own code. It was easier to give their code and guarantee that OpenSSL fit neatly into Stronghold. In essence, C2Net chose to give a little so it could continue to get all of the future improvements.


It's not much different from the car industry. There's nothing inherently better or worse about cars that have their steering wheel on the right-hand side. They're much easier to use in England. But if some free car engineering development team emerged in England, it might make sense for a U.S. company to donate work early to ensure that the final product could have the steering wheel on either side of the car without extensive redesign. If Ford just sat by and hoped to grab the final free product, it might find that the British engineers happily designed for the only roads they knew.


Engelschall is happy about this change. He wrote in an e-mail message, "They do the only reasonable approach: They base their server on modSSL because they know they cannot survive against the Open Source solution with their old proprietary code. And by contributing stuff to modSSL they implicitly make their own product better. This way both sides benefit."


Parekh and C2Net now have a challenge. They must continue to make the Stronghold package better than the free version to justify the cost people are paying.


Not all forks end with such a happy-faced story of mutual cooperation. Nor do all stories in the free software world end with the moneymaking corporation turning around and giving back their proprietary code to the general effort. But the C2Net/OpenSSL case illustrates how the nature of software development encourages companies and people to give and cooperate to satisfy their own selfish needs. Software can do a variety of wonderful things, but the structure often governs how easy it is for some of us to use. It makes sense to spend some extra time and make donations to a free software project if you want to make sure that the final product fits your specs.


The good news is that most people don't have much incentive to break off and fork their own project. If you stay on the same team, then you can easily use all the results produced by the other members. Cooperating is so much easier than fighting that people have a big incentive to stay together. If it weren't so selfish, it would be heartwarming.