1. Introduction

Business is changing after the expansive thinking of the late 1990s followed by the lessons learned in the early 2000s: It no longer makes sense for every company to make and own every aspect of its business. Many companies--perhaps all of them--require software, and many create software as an integral part of running their businesses; and some even produce software as part of their offerings. Every such company knows deeply that producing software is not a cakewalk and that the pace of creating final, usable software is slow. A naïve promise of open source is that it is somehow free of cost. But the real opportunities of open source come from many directions.

In fact, there are so many business and strategic reasons for a company heavily invested in creating software to adopt open-source activities that a company that does not can be considered to be out of touch with its own destiny. Even if a company decides that public (or pure, all-volunteer) open source is not for it, using open source within its firewalls could be a smart decision for pure software development methodology reasons.



Open Source: A Different Way of Doing Business

When businesspeople first encounter the idea of public open-source software, they are usually attracted by the fact that it is cost-free and that they get access to the underlying source code--open-source software is like coming across a blueberry bush bursting with fat ripe blueberries. But then it hits them that open-source software comes on an as-is basis with no warranty, no indemnification, and no support. No one is officially responsible--there is no one to contract with and no one to sue for liability.

Also confusing is the open-source development process. Who is in charge? How are decisions made? What is the development schedule? Finally, the idea of giving work done by their developers away to anyone--possibly including their business competitors--seems to violate common sense.

All in all this is not how they are used to doing business. But open source does make sense.

Innovation Happens Elsewhere

Companies who wish to create wealth are always interested in productivity. Productivity includes being able to innovate effectively. Effective innovation is not merely being able to invent and improve, but also being able to determine what to invent and how to improve. High productivity requires doing less to produce as much or more--a company that requires its own employees to labor hard and long to make its products or perform its services will be less profitable, in general, than one that can take advantage of the efforts of others. In most cases, this means that a company wishing to innovate productively must recognize that valuable work and talent exist outside the confines of the company and that it must find ways of using that outside material and expertise while still maintaining a competitive edge.

Until now, the most effective way of accomplishing this has been to purchase a company that has a technology or product of interest to the buyer along with the personnel who can maintain and move it forward. One of the best-known examples of this was Turbo Pascal, one of Borland International's most successful products--one that helped put Borland on the map.

Turbo Pascal started out as Blue Label Software (BLS) Pascal, developed and marketed by a company called PolyData based in Copenhagen, Denmark, and written primarily by Anders Hejlsberg. It was a subset of the full Pascal language along with an editor, compiler, and run-time libraries. BLS Pascal evolved into a complete Pascal implementation, changing its name to COMPAS and then PolyPascal as it moved to other platforms and further evolved. In 1983 PolyData entered into a license agreement with Borland. Borland wrote a new editor and menu system for PolyPascal, and the resulting product became Turbo Pascal. PolyPascal and Turbo Pascal were marketed in parallel for a few years, but thereafter PolyPascal was discontinued. When Anders Hejlsberg moved to the United States in 1987, he became a full-time employee of Borland.

One of Borland's most significant innovations with PolyPascal was to take a product that had previously sold for about $500 and sell it for $49.95. At that time, fully professional-grade development environments including a language compiler sold for around $3000, and Turbo Pascal was of similar quality. Therefore, this price innovation had a dramatic effect on the market. Moreover, not only was Borland able to get a top-quality development tool, but it was able to hire a top programming-language designer and implementor. By not having a stake in how hard it had been to develop the product, Borland was able to price the package so that it would sell dramatically well. Few people at that time--the early 1980s--had any idea that software developed for sale could be sold reasonably at commodity prices. And even after Borland entered the scene with Turbo Pascal, the lesson was learned only slowly.

Nevertheless, an important part of the Turbo Pascal story is that intellectual property rights were well defined and protected for both PolyData and Borland. Borland would not have entered into the agreement without the exclusive rights the agreement provided. Deciding whether to incorporate open source as part of a business strategy requires coming to grips with which property rights and control over the technology are important to your company.

The phrase "innovation happens elsewhere" captures the essence of the idea of adding just the smallest amount of innovation necessary for competitive advantage. It is common for people working for a technology company to suffer, at least a little, from the belief that all the really innovative people in their particular technology happen to work at that company. This can cause such a company to work too hard to produce every last bit of related technology, which is often not the best competitive approach. Many advantages accrue when a company adopts the attitude that most innovation happens elsewhere and focuses on choosing the best outside innovations and figuring out the right distinguishing features to make its products competitive.

Traditional open source is one of the best examples of this approach. The Internet started out as the ARPAnet, and many of the underlying technologies powering it--Domain Name Service (DNS via BIND), email transport (sendmail), and various open TCP/IP implementations--were developed in the public domain and remain open-source implementations, even when there are commercial versions available. Working in the public domain in the style of an open university research environment was the norm when the foundations of the Internet were being laid in the 1970s and early 1980s. Although there were software companies and software for sale, software was considered secondary to hardware, which made up the bulk of the purchase price of a computer system. Even in the late 1980s the concept that the software for a system might cost more than the hardware was laughable--the hardware is real, after all. Today it is quite common for the software costs to dominate the cost of a desktop computer system, and this trend has paralleled the rise of the software market. Open source is the continuation of this earlier style of software production, and the increase in activity in open-source projects has likewise mirrored the expansion of the commercial software segment, largely as a reaction to it.

Despite their apparent political or philosophical tendencies, the open-source and free-software activities point the way toward harnessing outside innovation for companies. What we mean is that not only is the software produced by open-source projects useful for improving productivity for companies, but so are the open-source practices and the idea of companies participating in and starting open-source projects.

By engaging an outside community, a company can learn which innovations to make and how to make them. All that is required is to play by some rules, give up the commodity part of the product, and skillfully retain the high-value part. In exchange, the community will, in general, act as a co-author of the company's product and innovate in unexpected ways.

Open source is not simply a resource--it is a viable business strategy.

Jumping In

A number of companies have jumped into open source, some of them basing their businesses entirely on it, such as Red Hat, VA Software, and CollabNet, but many more have started open-source projects both inside and outside their corporate firewalls. Among the larger or more widely known companies using open source as part of their business strategies are General Electric (GE), Sun Microsystems, IBM, Apple, Hewlett-Packard (HP), SGI, Oracle, Cisco, Intel, and Symbian.

Apple is an interesting example because it is not generally known as embracing open source but is thought of as being a highly proprietary company from its earlier years. Apple's OS X is based on Darwin, which is an open-source project. Here is what Apple's developer website says about it:

Apple's open source projects allow developers to customize and enhance key Apple software. Through the open source model, Apple engineers and the open source community collaborate to create better, faster and more reliable products for our users.
Beneath the appealing, easy-to-use interface of Mac OS X is a rock-solid foundation that is engineered for stability, reliability, and performance. This foundation is a core operating system commonly known as Darwin. Darwin integrates a number of technologies, most importantly Mach 3.0, operating-system services based on 4.4BSD (Berkeley Software Distribution), high-performance networking facilities, and support for multiple integrated file systems.[^955960]

However, an important part of this strategy is hidden. Because Darwin is one of the Unix family of operating systems, a fair number of other open-source packages happen to work on the platform, which helps Apple dramatically because the Macintosh platform is regarded as hosting relatively few applications. Even though the current crop of open-source Unix applications is not especially appealing for businesses per se, they are appealing to the scientific and engineering communities, which are well represented in the business sector, and as Linux makes inroads into the server and client markets, the application base for OS X can only get better.

For example, there is an open-source project called Fink dedicated to bringing some of the open-source Unix programs to Darwin:

The Fink project wants to bring the full world of Unix Open Source software to Darwin and Mac OS X. We modify Unix software so that it compiles and runs on Mac OS X ("port" it) and make it available for download as a coherent distribution.[^955963]

There are currently about 3500 packages in the Fink project. The packages by and large are not integrated into the full Aqua user interface, but they do provide significant functionality to the platform.

[^955960]: http://developer.apple.com/darwin/

[^955963]: http://fink.sourceforge.net/

Understanding Open Source

Open source forms a commons where certain types of development take place in the open and the artifacts are available for general use. The primary problem software-related companies face at the beginning of the twenty-first century is how to innovate effectively. Innovation is relatively common and easy, but being able to choose which innovations will make the most business sense and then monetizing them is not easy. There is in general a trade-off between doing things in the commons, where innovation is easy because of the large number of diverse people and unfamiliar ideas, and doing things behind closed doors, where it is easy to keep a competitive advantage through secrecy and intellectual property laws. The first step is understanding innovation and creativity and how a commons can be a good venue for them.

Open source itself seems new and is mysterious in many ways. Why would anyone work for free? Where did these ideas come from? What sorts of software are created using an open-source methodology? Open source seems like it comes from a mob, but in reality it comes from a set of communities. These communities are tied together through culture, vocabulary, customs, practices, values, ethics, and morals. Being able to work with an open-source community requires understanding all these things well enough not to be made a fool of immediately and then learning how to work effectively within the chosen community. There is an arc of maturation as you enter an open-source community as a newcomer or foreigner and then progress to becoming an inhabitant, then perhaps to leader, and then on to respected elder. Open source works on principles common to gift economies, which are generally more fundamental, or primitive, than free-market economies (which tend to be overly rational in how they treat value and compensation). Perhaps it's best to call gift economies precapitalistic.

Nevertheless there are many business reasons to embrace open source, all the way from just using open-source software such as Linux or Apache to starting and running open-source projects. The reasons to engage with open source include the following:

All-volunteer open source is the purest form. Projects such as Linux and Apache, before they were noticed and embraced by companies, were examples of all-volunteer open source; most of the projects on SourceForge are also examples. When a company joins an open-source project and especially when a company starts an open-source project, it brings to the table experts and processes that are not usually part of open-source projects. These include usability experts, release management, quality assurance, specification writing, documentation writers, and project management. These other process appurtenances can be carried over to varying degrees, but inevitably they alter the all-volunteer nature of open source, and therefore a company needs to know about how open source works in order to bring these other roles and practices to the table effectively. This is the central topic of this book.

An important legal key to open source is licensing. There is a pervasive myth that open-source software is not owned by anyone and that a company doing open source must give up ownership and control of its property. Actually, open source recognizes ownership and generally the primacy of ownership and its concomitant rights. Software licenses can yield many levels of rights because there is a vast gap between having no rights and all rights. Licenses can give the right to distribute and make changes to source code, but also can limit what sorts of distributions and changes are allowed. The license establishes, to an extent, the legal basis for an open-source project and community. There are a variety of licenses ranging from granting the right to view source, to gated communities, to open source, to free software, to public domain.

Engaging in an open-source project requires understanding the community, its culture and customs, its tools, and its way of working. Open-source projects are distributed and work through email, websites, and other written documents. There are standard versioning tools, compilers, bug-tracking tools, and customs for using them. Building, testing, support, and releases are handled in particular ways. There is a sort of hierarchy to the developers centered on module owners and the concept of a meritocracy, where rights are expanded only after abilities have been demonstrated.

In general, a company starting an open-source project needs to expend energy and resources creating a sufficiently large and robust community that will contribute, regardless of the type of contribution expected. Some managers and executives, upon beginning to engage in open source, are surprised that it seems more like a social activity than a development exercise, but this is the reality. To work with an open-source project--especially to start one up--requires an understanding of the culture and customs of the open-source way.

Beyond figuring out the business reasons for doing open source, a company or project needs to look at a variety of issues to determine whether it makes sense to use an open-source approach. These include whether you and your management buy into the open-source lifestyle, whether the source code is suitable and ready for source licensing, whether you have the resources, and whether your resource expectations are reasonable. Then you have to get the source code ready, get the company ready, choose a license, create a development plan and roadmap, make a budget and get funding, educate your developers, and build a website. Beyond these, there are commonsense things to do to build the community and maintain credibility in the open-source community.

After you've started an open-source project, there is a lot of work to do to keep its momentum building: It's not a matter of throwing the source code over a wall and watching a crowd gather. You need to craft and evolve the vision for the project, keep resources applied to the community, actively bring in contributors and users, evolve the website, keep things active and looking active, grow and mature your community members, and communicate transparently.

There are things you need to do to succeed. You really need to understand and buy into the open-source lifestyle. You need to start a relevant and useful project, not one that duplicates an existing effort. You need to get your code in shape and choose a good license--both for the open-source community and for your company. If you don't seem to have a way to make money or other commercial gain, the open-source community may consider you not with-it enough to risk expending effort on your project. And there are things to avoid. The biggest mistakes you can make involve control. The one characteristic of companies that open-source developers despise is overcontrol. It is natural to want to control a project, but the level of control that makes sense for an open-source project may seem too lax for development managers and executives. In many ways, the development processes in open source seem inefficient because they are based on written communication, and it's easy to fall into taking shortcuts, for instance by making important decisions in local, face-to-face meetings. This can cause tensions inside and outside the project in your company.

Moreover, the community will not arise just by itself. There is nurturing to do. And it's easy to buy into this concept intellectually while still failing to understand or appreciate what it means for everyday work. Most open-source communities need to have an email (or push-based) culture in order to seem vigorous and viable. Pull-type websites need to have compelling content for people to visit every day, and an open-source project is unlikely to have that.

Nevertheless, many companies have embraced the use of open source. In 2002, a study by Berlecon Research (FLOSS--Free/Libre Open Source Software: Survey and Study) conducted for the European Commission found that eight of the world's 25 largest software companies, including IBM, HP/Compaq, SAP, Hitachi, and Sun Microsystems, had a major involvement with open source and that another three companies engaged in lesser open-source activity. This was based on public announcements by the companies of their involvement, and so the actual use of open source is undoubtedly higher.

Communities

One of the hardest lessons to learn about working in open source is that the work involves community building, politics, citizenship, principles, and governance. An open-source project consists of some shared artifacts--source code, documentation, and so forth--and some mailing lists, newsgroups, and perhaps some other social software. That is, there are the things, on the one hand, and the people, on the other. The people form a community, a community that interacts, that forms customs and traditions, that develops friendships and affiliations, and that, in short, creates a culture. Many software developers ignore the social aspects of open source and might even be surprised, in many cases, to hear people talk of open source as a social or community activity. At the first Jini community meeting, one developer remarked, "Is Jini a technology or a sociology experiment?"

For a company to succeed at open source, and more so than for an individual or an established open-source project, it must take community seriously and be explicit about it. This is because there is always some suspicion by outsiders that a company has some hidden, unpleasant agenda.

Community building consists of the activities done to foster a culture and make it pleasant and rewarding to be a member of the community. There must be ways for individuals to visit the community, learn about it, join it as newcomers, become productive members, develop into experts, and finally become respected elders.

Politics enters the scene when companies or other competing interests are involved. How are compromises made? How are the positions of established expertise respected? These are political questions.

Citizenship has to do not only with the ways individuals mature through an arc of roles, but with whether individuals have the opportunity to feel that being a member of the community distinguishes them from others and whether this distinction is a source of pride. Perhaps the community has a significant name (Linux), a logo or other identifiable sign (a penguin), or established customs (yearly community meetings). Perhaps the community is known for its level of technical expertise or its important contributions.

Principles form the backbone of any open-source project. Software developers and engineers are overwhelmingly principled and ethical,[^955966] so that any open-source project started by a company needs to embrace and make explicit its dedication to principles. Open-source projects started by individuals generally don't have to be explicit about their principles because such principles are so prevalent within the developer community. But a company does not have this luxury.

Governance has to do with how decisions are made about the workings of the community. For the Linux project, Linus Torvalds makes many decisions about what code gets accepted into the source tree and who else has the authority to make such decisions, so he is part of the governance of the project; he decided that he makes such decisions. For companies, governance generally needs to be more explicit and, obviously, equitable and fair. As non-company-based open-source projects mature, they are finding that governance needs to be made explicit.

A good example of the importance of community is the project that designed Common Lisp, which was one of the first large network-based design collaborations, begun in 1981. One of us (RPG) was the originator of the Common Lisp effort. It began in the summer of 1981 as a consolidation effort between descendents of an older dialect of Lisp called MacLisp. After a series of short face-to-face meetings, the bulk of the work shifted to an email-based discussion on the ARPAnet (the predecessor of the Internet). The acknowledged list of contributors contains 62 individuals who exchanged over 2000 email messages over a 2-year period along with two or three face-to-face meetings and four drafts of the specification written primarily by Guy L. Steele Jr. Over its lifetime, the group exchanged well over 10,000 email messages.

The culture of this community was central to how it worked. The community developed an interaction style and a particular online culture in which a person's participation as well as "influence" depended on his or her expertise. Keep in mind that at that time (1981) there were no or very few online communities. Network email was limited to some universities and Department of Defense-related companies. It turned out--as we know only so well today--that people are more prone to be insulting and forthright in email than in person. Many of the discussions carried on over the network were quite spirited, but they had the advantage of being written down. There was no need to rely on dim memory or pale ink because all the mail was automatically stored in a centralized place, so there was no chance of losing or misplacing it.

The discussions were often in the form of proposals, discussions, and counterproposals for the design of Common Lisp. Code examples from existing software or proposed new syntax were often exchanged. And all was subject to quick review by community members. New members of the community wishing to come up to speed could go to the archives.

This online style had some drawbacks. Foremost, it was not possible to observe the facial and body reactions of other people, to see whether some point angered them, which would mean the point was important to them. There was no immediate way to see that an argument had gone too far or had little support. This meant that time was wasted and that carefully crafted written arguments were required to get anything done.

The leader of the group was Scott Fahlman of Carnegie-Mellon University, and there was an inner circle of decision makers--Scott Fahlman, Guy Steele, David Moon, Daniel Weinreb, and Richard P. Gabriel--called the Quinquevirate or Gang of Five. Just outside this circle was the semi-official Common Lisp Group, which included 33 individuals. This was its governance.

In all, the benefits of the contentious interaction style characteristic of the culture outweighed its problems. Moreover, the expert leadership, the small core of decision makers, the excellent specification-writing talent, and the freedom to work continuously without needing to schedule meeting times meant that the result was quite extraordinary. Despite the diverse and oddball nature of the individuals, a definite culture emerged with strong bonds, a private language, emergent roles, and definite traditions.[^955969]

The Common Lisp community was based on the principles of openness and the celebration of technical expertise that characterized the MIT AI (Artificial Intelligence) Lab culture. This is the culture that formed and still supports the views of Richard Stallman, who is the founder of the Free Software Foundation--which kicked off the open-source movement--and who was an active member of the Common Lisp community.

Politics was central to the community. One goal was to bring together Lisp machine companies that had developed new and different Lisp language features to distinguish themselves from each other and other Lisp vendors (both hardware and software vendors) to agree on a common standard--if it was successful, this standard would diminish their differentiators.[^955972] Moreover, software Lisp vendors working on general-purpose computer implementations, academics, and Lisp users were invited into the community, with their conflicting and unusual perspectives and requirements--some commercial, some scientific, some engineering-based, and some just oddball. All these groups were expected to come to a common agreement. This is politics par excellence. Pride of citizenship was conferred by the attentiveness of ARPA to the group and the fame of some of its principals.

Finally, the curious fact about the Common Lisp effort was that even though its apparent product was a specification of Common Lisp in the form of a book, the members of the community never worked on the text of that book. There were no artifacts that formed the center of the community; people, in general, did not contribute code, text, or editing expertise. The writing was done entirely by Guy Steele and the other members of the Quinquevirate. The community was there to make decisions, much as do legislatures and constitutional conventions. A newcomer to the Common Lisp community needed to listen and read the email archives--until Steele's first draft there was no artifact available to study and change; and after, only Steele could change the draft. The day-to-day activity consisted of political and technical debate followed by making specific decisions, which means that the Common Lisp project was essentially all community.

[^955966]: Virus writers and crackers notwithstanding.

[^955969]: The transcript of the Common Lisp email interactions was studied by JoAnne Yates and Wanda J. Orlikowski of the MIT Sloan School, resulting in two reports: " Knee-Jerk Anti-LOOPism and Other E-mail Phenomena: Oral, Written, and Electronic Patterns in Computer-Mediated Communication ," MIT Sloan School Working Paper #3578-93, and " From Memo to Dialogue: Enacting Genres of Communication in Electronic Mail ," MIT Sloan School Working Paper #3525, Sloan School, MIT, Cambridge, MA, July 1993.

[^955972]: And, in fact, some have argued that the eventual success of Common Lisp as a standard language combined with improvements in general-purpose computers and compiler technology dealt all the Lisp machine companies a death blow.

Who This Book Is Intended For

We wrote this book to help business executives understand when and how an open-source strategy can help them to achieve their company's business goals. We also want to provide support for the managers charged with implementing that strategy in their day-to-day work running a project that makes use of open source.

The book is also aimed at the engineers who may need to work on open-source projects. We want to give them an idea of what they will experience and what will be expected of them. We also want to give them the information they will need to educate their managers and co-workers about open source.

Third, the book is for anyone interested in a better understanding of open source--its larger history, its philosophy, and its future prospects.

Software Developers

Since we wrote the first draft of this book, the depth of the recession of the early twenty-first century has made itself felt in the software industry. We don't want to make this book specific to a particular era, but we want to point out to software developers of any time period the educational and experiential advantages of working with open source. It is unlikely that open-source code will disappear. This means that open-source systems will form the basis of a significant portion of our computing infrastructure for many years. An individual who has experience working with open-source software will enjoy a variety of advantages.

Experience with open source can add value to your skill set. Operating systems, web servers, scripting-level languages, wikis, weblogs, email infrastructure, and many other facilities come from the open-source world, and experience with these pieces of software is valuable. Experience working with open-source projects can be of value to companies that have or plan to have open-source projects of their own. Knowing the roles of the members and having a good idea of how the process works is essential for companies to succeed with open source, and knowledge comes more from experience than from books.

Working with an open-source project will give you experience working in a distributed development environment and perhaps working with non-English speakers from different cultural backgrounds. Increasingly companies are locating development groups around the world, both in-sourced and out-sourced, and experience in this sort of environment can be essential.

Open-source projects work using documents and written communications. The documents can be as simple as an emailed specification or design rationale, but they are documents nevertheless. Working through writing is a way to bring more discipline to the development process. Moreover, you can become a better writer by practicing writing, and working on an email-heavy project will give you lots of practice.

In an open-source project, the source code is available for inspection, and the designers and implementors of the software are generally available and willing to answer questions. Reading (and critiquing) source code is one of the best ways to learn about design and implementation. Open-source software is available for continual improvement, so it might be higher quality than commercial code and, thus, a better educational experience.

Before open source became prevalent, self-training in many of these areas was a hit-or-miss affair. Learning software development requires interaction, and it was difficult before Internet-based open-source projects existed to find places to get the sort of interaction needed for training.

Open Source as Business Strategy

Making a public open-source project successful takes more than a press release and putting some code outside the firewall. Open source is a development methodology, but companies whose thinking stops there will find the experience disappointing. Open source is sometimes called a lifestyle, which means that successful open-source projects operate under a set of cultural conventions that may be foreign or difficult for some companies to accept. Public-open source is not a cost-saving mechanism, and it takes a careful business analysis to justify using it.

This book focuses on what a company needs to do to make open source a part of its business. This starts with understanding the business reasons for using open source and goes on to the details of day-to-day project activities. Included are examples of how companies such as Sun, IBM, HP, and Cisco Systems have successfully benefited from their use of open source.

Our examples tend to be drawn from Sun projects. The reason for this is that we have had the opportunity to observe not only the reaction of the community and pundits to company-sponsored open-source projects and the reports of people working on the inside on these projects, but also the internal discussions, pro and con, good and bad, of the decision makers. We were able to see what advice was taken and not taken, and what the results were. Sun is neither the best practitioner of open source as a business strategy nor the worst. Sun's record is spotty with some good successes and several embarrassments; the company embraces some of the fundamental tenets of openness and working with a community that are required for a successful use of open source as a strategy, but it often fails to embrace them thoroughly enough to realize all the benefits.

We do not advocate using open source blindly as a strategy, and we are not die-hard promoters of open source. We hope this book is a balanced look at the pros and cons of open source, along with tested advice on how to succeed once open source has been chosen as a part of a company's strategy.