An objective definition of open standards

Computer Standards & Interfaces 28 (2006) 495 – 507 www.elsevier.com/locate/csi

An objective definition of open standards Michael Tiemann Red Hat Corporate Headquarters, 1801 Varsity Drive, Raleigh, North Carolina 27606, USA Received 13 December 2004; accepted 13 December 2004 Available online 7 January 2005

Abstract There is much debate among IT executives and policy-makers as to whether Open Standards, Open Source, and/or something else, are necessary and sufficient to ensure that their software procurements add long-term value to their IT systems. However, a lack of definition of Open Standards makes debate on their meaning, let alone their merits, challenging. This lack of objective definition provides a convenient loophole for vendors to make claims which sound good in theory while protecting bad practices. Given the size of the world-wide software market, and the significant investment that it therefore represents, it is time to define what the software industry (vendors, customers, and users) all claim to want: Open Standards. This paper defines the lower limit of what can be called an Open Standard and then defines a framework for grading Open Standards that exceed that minimum. This framework is then extended to the subject of file formats. This paper does not take the position that Open Source is intrinsically good, nor that Open Source in and of itself is a de facto Open Standard. But it does evaluate the role that Open Source can play to make a good Open Standard better, and argues forcefully that Open Standards without Open Source implementations offer little protection from vendor lock-in. D 2004 Published by Elsevier B.V. Keywords: Open Standards; Open Source; Information technology

1. The need for Open Standards in commerce When customers procure systems, they want the best value for the money. They want systems that work when delivered, that work robustly over time, that can be updated as needed, and that work well with other systems.1 Customers do not want to be locked into a single vendor (who can then raise rates, reduce service arbitrarily, or otherwise act strategically), nor do they want products that tilt against or are insensitive to the benefits of free-market competition (which reduces 1

See http://www.cio.com/archive/101501/wasting.html.

0920-5489/$ - see front matter D 2004 Published by Elsevier B.V. doi:10.1016/j.csi.2004.12.003

value beyond the strict borders of the system).2 And because it is impossible to predict what the future will bring (though we believe it will somehow be better), customers want systems that can interoperate with possible future products, or at least systems that provide a viable exit strategy to some better future platform. In 1900 the US Government set out to solve this problem for the industrial world, creating the National Bureau of Standards, now called the National Institute 2

Ibid.

496

M. Tiemann / Computer Standards & Interfaces 28 (2006) 495–507

of Standards and Technology.3 Standards of commerce and interoperability were defined, and within a few years, according to NIST, these problems were solved. In the 1980s, the IT industry took preemptive action and declared that Open Standards would deliver users from the actions of previous regimes that had created confusion in commerce, difficulty in conducting fair transactions, and most obviously, great difficulty in getting systems from different vendors to fit together properly. More than 20 years later, we remain not only without an accepted definition of an Open Standard, but with the problems stemming from a lack of standards multiplied by the prevalence of technology’s role in society today.4

which mentions Open Standards prominently in its executive summaries, does not define this term except within a sub-domain of the overall scope. Moreover, the definition given6 is so broad that it covers virtually anything to which it can be applied: To be considered open, a standard does not have to meet all criteria listed. These criteria are listed below in priority order for consideration in selecting preferred standards. 5

2. Open? Standard? It is hard to argue against Open Standards, but it is even more difficult to agree on what the term actually means. Are they vendor-neutral? Interoperable? Published? Can they be implemented royalty-free? All of the above? Not necessarily any of the above? If the term bOpen StandardQ is to be used in any IT policy document, it had better both be clearly defined and substantially useful as defined. And because the term Open Standard is now being considered in some contexts alongside Open Source as a means to empower users against vendor lock-in and improve software procurement generally, it makes sense to learn what we can learn from the process that has made Open Source both so legitimate and so widely understood, and to apply that process of inspection, rigorously, to Open Standards. Creating a rich and meaningful definition of Open Standards is no small task, conceptually, politically, or otherwise. Within the US Department of Defense (DoD), for example, the Open Systems Joint Task Force has made an attempt to define an Open Standard as something that is bwidely used, consensus based, published, and maintained by recognized industry standards organizations.Q5 So far so good, yet the latest DoD Joint Technical Architecture report,

5 5

5

5

WS.AV.1.5.1.2.1 Widely Used—Widely used is conceptual in nature and as a result difficult to define. There can be a wide range of users, from one to thousands. Typically, the concept requires some judgment; e.g., if there are two standards, and one has a single user and the other has multiple users, the standard with multiple users would be preferred. WS.AV.1.5.1.2.2 International—Standards that are accepted by more than one nation or international organizations are preferred. WS.AV.1.5.1.2.3 Consensus Based—Consensus based means that more than one entity, or a standard development organization representing more than one entity, has agreed upon or promulgated the standard. WS.AV.1.5.1.2.4 Public Domain—Public domain means the standard is not owned by a single company and is publicly available. Any company could use the standard without paying license or royalty fees. WS.AV.1.5.1.2.5 Well Defined (Verifiable)—A well-defined standard contains readily available documentation that is complete enough for use by a design team, and includes verification criteria to check the design solution for compliance.

According to the above definition, a proprietary system with a single user, no international acceptance, and no published documentation could, conceivably be called an Open Standard (though that surely was not the intention). In contrast, Eric Raymond, President of the Open Source Initiative,7 has declared that bIf [the so-called

3

See http://www.technology.gov/Mission.htm. United Nations Conference on Trade and Development (UNCTAD), 2003, p. 95. 5 See http://www.acq.osd.mil/osjtf/html/terms.htm. 4

6

See http://jta.disa.mil/jta/jta-vol-I.pdf, Vol I–122, pdf page 146–147. 7 See http://www.opensource.org.

M. Tiemann / Computer Standards & Interfaces 28 (2006) 495–507

Open Standard] doesn’t have an open-source reference implementation, the term bstandardQ is an abuse of the language.Q8 What follows is an attempt to bridge these two widely divergent views. To build this bridge we must now address two problems where before there were one: 1. 2.

What is an Open Standard? What problems do they exist to solve? How should one Open Standard be compared in quality or preference to another?

To address the first question, consider a 1993 report Investment Strategy Study published by the Institute for Defense Analyses.9 This study reported that: The use of Open Standards in [automated testing systems] has been projected to provide the following five benefits: 1.

Improve the [test] acquisition process by creating an [automated testing system] framework that can meet functional and technological needs, and promote automation in software development, rehostability, and portability of [test program sets]. 2. Decrease the use of custom hardware from approximately 70% today to 30%. 3. Reduce engineering costs 70%. 4. Reduce [test program set] integration time and cost 50 to 75%. 5. Provide an iterative improvement in the quality of [test] by the reuse and refinement of libraries.

497

productive to rehabilitate the term than to try to find a new one (OMB Circular A-119 appears to define bVoluntary, Consensus StandardQ as an alternative, but as Appendix 1 explains, the circular definition leads nowhere). Taking on that challenge, we propose distinguishing four levels of Open Standards: Open Standard 0: The standard is documented and can be completely implemented, used, and distributed royalty free (just as the W3C requires of any standards submitted to them). Implementations of the standard may be extended, or offered in subset form. However, certification organizations may decline to certify subset implementations, and may require that extensions also satisfy the criteria of an Open Standard. Anything less than this is not an Open Standard, period. Open Standard 1: There is specified OSS that can interoperate with the standard. Moreover, if there is an interoperability issue between a product claiming Open Standard 1 and the specified OSS product, it means that the vendor of the Open Standard 1 product will be responsive to interoperability questions or issues raised by developers of the specified OSS product. Open Standard 2: There is an OSS reference implementation of the standard. Open Standard 3: This implementation of the standard is an OSS implementation.

4. Rationale Other research and references (see Appendix 3) provides numerous variations of the same themes: Open Standards promote a variety of beneficial environmental, process, and market factors that generally give buyers much better value for money, and this is not limited merely to software testing (the subject of the IDA report).

3. Defining Open Standards The term Open Standard may have been poorly or loosely defined in the past, but it may be more

Open Standard Principle 0 encapsulates the factors that ensure integrity through transparency, independent participation, and due process. Moreover, such a standard must be technology and platform neutral or else the requirement of complete implementation cannot be met. Indeed, although it falls short of the definition demanded by Eric Raymond (an open source implementation is not a requirement), or what Bruce Perens suggests in his Open Standards Principles,10 it is consistent in scope and intent with what the DoD’s Open System Joint Task Force report. More importantly, this definition might survive a referen-

8

See http://www.eweek.com/article2/0,1759,1544770,00.asp. Referenced from JTA Version 6.0, Final, 3 October 2003, Vol I–97. 9

10 See http://perens.com/OpenStandards/Definition.html, particularly principles 1, 3, 5, and 6.

498

M. Tiemann / Computer Standards & Interfaces 28 (2006) 495–507

dum as the bare minimum of what 1.1 million OSS developers11 would accept as their definition of Open Standard. To them, this level of Open Standard would be what bFreedom ZeroQ is to the Free Software community—the minimum below which it is more ethical to disobey the law than to obey it. Note that standards promoted by ISO, ANSI, IEEE, ECMA, and other standards bodies often do not meet the test of Open Standard 0 because these organizations do not universally accept the royaltyfree clause. This is their choice to make, but it potentially defeats the goal of technology neutrality and we should not let ourselves be goaded into thinking that proprietary standards are Open Standards. Moreover, we should not accept that a standard that necessarily discriminates against the development and business models of Open Source is an Open Standard. We should understand that if we accept standards that eliminate the possibility of an open source implementation, not only is competition is reduced, but the positive values of the network effect (see Appendix 2) are disproportionately reduced. Standards bodies that maintain proprietary standards may continue to meet the letter of the law (OMB Circular A-119) as it relates to bVoluntary, Consensus Standards,Q but we should not for a minute confuse those with Open Standards.12 Open Standard Principle 1 provides to users, first and foremost, the guarantee of an exit strategy. Namely, users have source code which can either be used directly, or can legally be modified to be used, to get data out. Full exportability of data is the only sure defense against vendor lock-in. From the market’s perspective, Open Standard 1 provides not only the integrity of a standard, but the genuine (not merely theoretical) potential for competition among software that produces or consumes the standard. Thus, if there is a document format that can be read and written by an open source word processing program, or a web protocol that can be interpreted by an open source 11 The 2004 study by Evans Data measuring the size of the Open Source development community is referenced here: http:// www.evansdata.com/n2/pr/releases/DPS2004.shtml. 12 Andrew Updegrove of Gesmer Updegrove LLP observe that as important as the product known as the standard is the process by which that standard is created and maintained. This idea is fully developed here: http://www.consortiuminfo.org/bulletins/ mar04.php#feature.

web client, then selection of this standard will invite competition from open source alternatives, which creates more competition than if Open Source is excluded. Open Standards Principle 2 aims beyond merely giving users a guaranteed exit strategy: it provides the ability to review the actual workings of the standard in an implementation. While there clearly is a distinction between a standard definition and a reference implementation, a reference implementation can be extremely valuable in identifying gaps or hidden assumptions that may underlie a standard definition. The availability of reference implementations of both web servers and web clients proved indispensable in corroborating or impeaching claims of competing proprietary vendors in reference to HTTP and HTML standards, for example. The IETF’s Best Practices paper (quoted at the conclusion of this paper) generalizes this principle, arguing that standards without implementations are not trustworthy, and implementations that cannot be examined cannot be verified. The success and robustness of the Internet is a direct consequence of building on a foundation of standards that rose to the level of Open Standard 2. This is the minimum level that Eric Raymond would endorse as a true bOpen Standard.Q Open Standard 3 recognizes that not only is there an open source implementation that can be examined, this particular implementation is open source and can be examined. This provides the most direct route for determining whether a fault lies in the standard, the implementation of the standard, or an understanding of the standard, and is most useful in providing a means for advancing the standard over time as practices improve, as well as providing a safeguard against fragmentation when a proprietary implementation extends the standard but the extensions have not been reincorporated into the open source reference implementation. This is the level that Eric Raymond believes will be the norm in 3–5 years. There are some examples where this delineation (and its consequences) can be observed. The Berkeley TCP/IP Networking Stack, when distributed with its source code, is an example of software satisfying all four levels of Open Standards. Closed-source versions of the Berkeley TCP/IP Networking Stack (such as reported to be in Microsoft’s Windows products and Apple’s OSX products) satisfy Open Standards 0, 1,

M. Tiemann / Computer Standards & Interfaces 28 (2006) 495–507

and 2. The Linux TCP/IP Networking Stack (which is different than Berkeley’s) also satisfies all four levels, showing that there need not be only one single reference implementation. Similarly, the Apache Web Server, when distributed with its source code, also satisfies all four levels of standards for the HTTP server protocol whereas the Microsoft IIS web server satisfies at most level 2. South Korea offers a cautionary example of why a higher level of Open Standard implementation is a good thing. Numerous open source evangelists living in Korea, who run open source software whenever and where ever possible, have found it virtually impossible to navigate the web without using Microsoft’s Internet Explorer (IE) to connect to the Microsoft IIS webservers so prevalent in that country. Virtually everywhere else in the world, where Apache enjoys approximately 70% market share, numerous web browsers are all quite usable for most tasks. How did Microsoft’s IE and IIS, which are supposedly standards-conforming programs, turn Korea into a decidedly non-standard web environment? Well, that is a question that is difficult to answer without source code, though it may be related to whether IE and IIS implement HTTP extensions that violate Open Standard 0.

5. Open file formats While there is tremendous macro-economic benefit for the industry to broadly adopt Open Standards13 (a shift greatly facilitated by the availability and adoption of open source implementations of these standards) this discussion would be incomplete without a discussion of file formats, and in particular the notion of Open File Formats. While file formats are governed by standards (either open or proprietary, explicit or implicit), file formats represent a disposition of data rather than an artifact of a program. The distinction is important when it comes to questions of ownership: many users feel no need to bownQ the programs they use, just the right to use them legally. But most users also feel that the data they create by using programs, operating systems, computer hardware, etc., rightfully

13

See Appendix 2 for a detailed explanation.

499

belongs to them (or to the employer paying for the creation of such data). When a program writes data in a proprietary file format, the data may well be legally owned by the party creating it, but it cannot be accessed later without maintaining a right and ability to run a format-aware application. Proprietary file formats thus create an invisible tax on data. The network value analysis in Appendix 2 illustrates that the btaxQ on proprietary data formats will increase as the popularity of the format increases. The tax is a transfer of value from the customer to the vendor—one that can increase as the customer produces more documents in the format. The tax also results in an overall barrier to increased adoption of the format, and thus a decrease in the value through interoperability of the format.14 If, for whatever reason, it is necessary to separate the notions of Open Standards (which primarily apply to applications and their programming interfaces) and Open File Formats (which apply strictly to the formatting of data to be stored and retrieved), the logical extensions of the Open Standard principles to file formats are as follows. Open Format 0. The format is published as a standard (which means governed by a standards body), and as a standard it can be implemented and practiced without royalty or restriction by any third party. Examples of Open Format 0 include Postscript, PDF, ASCII text, HTML, and (as far as we know, but certain Microsoft patent assertions could change this) XML. Under this definition, Open Format 0 does not include the Microsoft .doc format today because .doc is not a published standard recognized by any independent standards body (the fact that there may be 3rd-party applications that claim to read and write this format does not change this, nor does and the fact that such applications are available as free or open source software).

14 Tim Berners-Lee writes on his website: bThe decision to make the Web an open system was necessary for it to be universal. You can’t propose that something be a universal space and at the same time keep control of it.Q Berners-Lee was not the first to implement or release a hypertext system, but was the first to recognize that a proprietary file format would prevent his system from reaching critical mass. See http://www.w3.org/People/Berners-Lee/ FAQ.html#What2.

500

M. Tiemann / Computer Standards & Interfaces 28 (2006) 495–507

Open Format 1 adds to Open Format 0 the availability of an Open Source reader. This does not tilt the market to prefer Open Source, it only says that if an open source reader exists, then the file format is more likely to be readable on more platforms (including platforms that have not yet been defined, such as those that will exist 25 years into the future). Open Format 2 adds to Open Format 0 the availability of an Open Source writer. This does not tilt the market to prefer Open Source, it only says that if an open source writer exists, then the file format is more likely to be writable on more platforms (including platforms that may no longer be supported by their original vendors). Open Format 3 adds to Open Formats 1 and 2 the availability of an open source translator. However, rather than being binary like the other definitions of this document, its measure is a function of the number of open formats that this translator can convert to and from.

6. Freedom by any other name still needs protection If we are going to nominate Open Standards as a measure of quality in making procurement decisions, we should do so knowing 1. 2.

3.

4.

5.

there are strong economic incentives for subverting these standards; while rivalrous vendors suffer large penalties and large risks when they try to subvert Open Standards, that is not in and of itself a sufficient deterrent from behaving irrationally; dominant vendors suffer little penalty and little risk in subverting Open Standards while enjoying considerable upside. Absent a robust definition and requirement of Open Standards, there is little to deter such vendors from acting rationally; the availability of open source, which tilts the economic incentives from the micro- to the marco-, provides a highly robust mechanism to achieve the benefits that Open Standards promise, and therefore: some Open Standards are better and more robust than others.

In closing, consider the significant factors that made the Internet the hallmark of engineering that it is. In the words of the Internet Engineering Task Force (IETF), in their document titled The Internet Standards Process – Revision 3:15 Status of this Memo This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Distribution of this memo is unlimited. Abstract This memo documents the process used by the Internet community for the standardization of protocols and procedures. It defines the stages in the standardization process, the requirements for moving a document between stages, and the types of documents used during this process. It also addresses the intellectual property rights and copyright issues associated with the standards process. [. . .] 1.1 Internet Standards The Internet, a loosely-organized international collaboration of autonomous, interconnected networks, supports host-to-host communication through voluntary adherence to open protocols and procedures defined by Internet Standards. There are also many isolated interconnected networks, which are not connected to the global Internet but use the Internet Standards. The Internet Standards Process described in this document is concerned with all protocols, procedures, and conventions that are used in or by the Internet, whether or not they are part of the TCP/IP protocol suite. In the case of protocols developed and/or standardized by non-Internet organizations, however, the Internet Standards Process normally applies to the application of the protocol or procedure in the Internet context, not to the specification of the protocol itself. In general, an Internet Standard is a specification that is stable and well-understood, is technically competent, has multiple, independent, and interoperable implementations with substantial operational experience, enjoys significant public support, and is

15

See http://ietf.org/rfc/rfc2026.txt.

M. Tiemann / Computer Standards & Interfaces 28 (2006) 495–507

recognizably useful in some or all parts of the Internet. The robustness, reach, and utility of the Internet, which crosses more implementation boundaries than perhaps any other system in history, is clearly the result of applying these best standards practices, and it is no accident that quite often, Open Source implementations have played a major role in satisfying the criteria of what makes a good standard. That is not to say that Open Source is the only way to implement a standard, but that when some implementation can interoperate with specified open source, when there is a reference open source implementation of the standard, when the implementation in question is open source, it is easy to see how multiple, independent, and interoperable implementations can exist, how the transparency of such implementations can generate substantial operational experience, and how, through widespread distribution and the network effect, such standards can enjoy broad public support. Open Standards are the key to creating a level playing field between proprietary and open source software. And the key to Open Standards is a NISTlike definition that is transparent, objective, and most of all, understood to mean the same thing by everybody involved. Then, perhaps, innovation will lead to systems that are better, not only different.

7. Legislative implications As Alexis de Tocqueville observed in 1835, the principal feature of a democratic government is that the People do not serve the State, but that the State serves the People. In 2001 Dr. Edgar David Villanueva Nun˜ez, a congressman from Peru´, began working on Bill Number 1609, also known as bFree Software in Public Administration.Q16 Dr. Villanueva’s justification for writing such a bill is as revolutionary as the notion of democracy itself: We agree that in the private sector of the economy, it must be the market that decides which products to

16 See http://linuxtoday.com/mailprint.php3?action=pv<sn= 2002-05-20-006-26-IN-LF-PB.

501

use, and no state interference is permissible there. However, in the case of the public sector, the reasoning is not the same: as we have already established, the state archives, handles, and transmits information which does not belong to it, but which is entrusted to it by citizens, who have no alternative under the rule of law. As a counterpart to this legal requirement, the State must take extreme measures to safeguard the integrity, confidentiality, and accessibility of this information. The use of proprietary software raises serious doubts as to whether these requirements can be fulfilled, lacks conclusive evidence in this respect, and so is not suitable for use in the public sector. This declaration of IT independence,17 more than any other, sparked the movement to give Open Source software legislative preference over proprietary software for use in the public sector, a movement that continues to gather momentum at all levels of government around the world. But is this the best, or the only way to solve the problems so frequently observed in public-sector IT projects dominated by proprietary software? The proprietary software industry has said no: Open Source is overkill—Open Standards are all that are needed. My response is that unless and until Open Standards are defined in a meaningful way–one that enables public officials to act in accordance with the rule of law as Dr. Villanueva suggests–Open Source software, when available, may indeed be the only choice for safeguarding the public’s information. If a legislature does want to address the question of what is in the public interest when it comes to information technology, I would suggest first defining Open Standards as I have here, and then secondly deciding whether more needs to be done to remedy the failings of the market. In Washington State, for example, such a definition is already overdue. The most recent Information Technology Investment Policy 18 draft of the Information Services Board (ISB) of that State’s Department of Information Services begs the question bwhat is an Open Standard?Q [emphasis

17 See http://www.opensource.org/docs/peru_and_ms.php and also http://www.gnu.org.pe/resmseng.html. 18 See http://www.dis.wa.gov/portfolio/PDFs/ itinvestmentpolicy.pdf.

502

M. Tiemann / Computer Standards & Interfaces 28 (2006) 495–507

mine] in the context of trying to act in the public interest: The purpose of this policy is to ensure that information technology (IT) resource investments by agencies of the executive and judicial branches of state government: 5

5

5

5

are included in an agency’s IT portfolio and support that agency’s strategic business plan, business requirements, and risk management process. are wise uses of state resources made after consideration of multiple alternatives and a range of technical, functional, and business options. are obtained through the use of open, vendorneutral specifications and standards that support each agency’s IT portfolio and the state’s overall IT infrastructure. are obtained through fair and open competition among vendors, when possible.

On the opposite side of the earth, The Department of Information Technology of the Government of Punjab, India has this to say: Information Technology (IT) Resources Acquisition and Disposal Policy 19 PURPOSE The purpose of this policy is to provide requirements for the acquisition of information technology (IT) resources which: 5 5

5 5

19

Meet Departments’ functional and business needs while providing timeliness, cost efficiency, and flexibility in the acquisition process. Allow Departments to obtain the best value for money by considering multiple alternatives, giving consideration to a range of technical factors, functional and business requirements, and the quality. Promote fair and open competition. Promote the use of open, vendor-neutral specifications and standards to ensure that acquisitions support Department strategic directions and the Punjab State Technical Standards (STS).

http://www.doitpunjab.gov.in/it_manual/policy.zip.

These objectives are certainly consistent both with each other and with the definitions put forth in this paper. However, when a legislator finds no vendor-neutral specification (either of a software standard, a protocol, or a file format) in their IT portfolio and/or overall IT infrastructure, or finds that there is no possibility of a fair and open competition among vendors, should that legislator accept the status quo or should they, like Dr. Villanueva, seek to make a change? The US Departments of State, Commerce, and Justice are silent on this topic, but the People, increasingly, are not. Of course merely legislating preference of Open Source technologies over proprietary technologies does not address this issue, either, but legislating what is an Open Standard, and holding vendors accountable to implementing and not subverting those standards, is an idea whose time has come.

Acknowledgments This paper benefited enormously from the review, comments, and criticisms of the following people: Alan Cox, David A. Wheeler, L. Jean Camp, Eric Raymond, Andrew Updegrove, Will A. Rodger. This paper drew enormous inspiration from Tim O’Reilly and Richard Stallman’s discussion of bFreedom Zero.Q Thanks to the Honorable Bruce Mehlman for pointing me to the origins of NIST and its enabling role in modern commerce. Thanks also to Tim Berners-Lee for catalyzing the World Wide Web, not only the most compelling example of the power and scope of Metcalfe’s Law, but a darn useful resource for tracking down research and citations as well. The author welcomes endorsement of this paper by other parties, but does not claim any such endorsements except his own.

Appendix 0 The framework of delineating Open Standards and Open File Formats for the purposes of answering two questions: (1) what are they, and (2) which ones are better than others, was inspired by the delineation provided by the Free Software Definition and the

M. Tiemann / Computer Standards & Interfaces 28 (2006) 495–507

Open Source Definition. The Free Software Definition20 defines four freedoms: 5

The freedom to run the program, for any purpose (freedom 0). The freedom to study how the program works, and adapt it to your needs (freedom 1). Access to the source code is a precondition for this. The freedom to redistribute copies so you can help your neighbor (freedom 2). The freedom to improve the program, and release your improvements to the public, so that the whole community benefits (freedom 3). Access to the source code is a precondition for this.

5 5 5

and asserts that if all four freedoms are present, the software may be considered bfree software.Q If the software lacks one or more of these four freedoms, it is considered bnon-free software.Q Similarly, the Open Source Definition21 defines 10 criteria: 1.

Free Redistribution: The license shall not restrict any party from selling or giving away the software as a component of an aggregate software distribution containing programs from several different sources. The license shall not require a royalty or other fee for such sale. 2. Source Code: The program must include source code, and must allow distribution in source code as well as compiled form. Where some form of a product is not distributed with source code, there must be a well-publicized means of obtaining the source code for no more than a reasonable reproduction cost preferably, downloading via the Internet without charge. The source code must be the preferred form in which a programmer would modify the program. Deliberately obfuscated source code is not allowed. Intermediate forms such as the output of a preprocessor or translator are not allowed. 3. Derived Works: The license must allow modifications and derived works, and must allow them to be distributed under the same terms as the license of the original software. 20 21

See http://www.fsf.org/philosophy/free-sw.html. See http://opensource.org/docs/definition.php.

503

4.

Integrity of the Author’s Source Code: The license may restrict source-code from being distributed in modified form only if the license allows the distribution of bpatch filesQ with the source code for the purpose of modifying the program at build time. The license must explicitly permit distribution of software built from modified source code. The license may require derived works to carry a different name or version number from the original software. 5. No Discrimination Against Persons or Groups: The license must not discriminate against any person or group of persons. 6. No Discrimination Against Fields of Endeavor: The license must not restrict anyone from making use of the program in a specific field of endeavor. For example, it may not restrict the program from being used in a business, or from being used for genetic research. 7. Distribution of License: The rights attached to the program must apply to all to whom the program is redistributed without the need for execution of an additional license by those parties. 8. License Must Not Be Specific to a Product: The rights attached to the program must not depend on the program’s being part of a particular software distribution. If the program is extracted from that distribution and used or distributed within the terms of the program’s license, all parties to whom the program is redistributed should have the same rights as those that are granted in conjunction with the original software distribution. 9. License Must Not Restrict Other Software: The license must not place restrictions on other software that is distributed along with the licensed software. For example, the license must not insist that all other programs distributed on the same medium must be open-source software. 10. License Must Be Technology-Neutral: No provision of the license may be predicated on any individual technology or style of interface. If a software license conforms to all 10 criteria, it may be submitted for approval by the OSI board, and upon such approval, is bOSI Certified.Q There are dozens of licenses22 that are now certified. 22

See http://opensource.org/licenses/.

504

M. Tiemann / Computer Standards & Interfaces 28 (2006) 495–507

The industry has acknowledged the four freedoms of the Free Software Definition. That is not to say that everybody agrees with them, but they are there, consistently understood. The industry has acknowledged the Open Source Definition, driving billions of USD in server and software sales each quarter,23 demonstrating its commercial relevance and appeal. If the industry can accept, or at least debate, a delineation of Open Standards that illuminates, rather than obscures, both the principles and specifics of every given standard, then programmers, managers, and procurement agents will all be better equipped to make the best decisions. And when Open Standards are made objective, as both Free Software and Open Source software have been, customers who truly want Open Standards will be able to get what they want.

Thus, we have a benefit of using standards defined in terms of our ability to benefit from using standards. Now, as to the question of bwhat is a standard?,Q the circular continues that a definition of standards that includes all of the following: (1)

(2)

Appendix 1 OMB Circular A-119 establishes policies on Federal use and development of voluntary consensus standards and on conformity assessment activities (http://www.whitehouse.gov/omb/circulars/a119/ a119.html). As if to prove that the Government is not without a sense of humor, this circular answers our question of bWhat is the benefit of using [Open Standards]Q in a circular fashion. In the background section the document defines the goals of Government using bVoluntary Consensus StandardsQ a.

Eliminate the cost to the Government of developing its own standards and decrease the cost of goods procured and the burden of complying with agency regulation. b. Provide incentives and opportunities to establish standards that serve national needs. c. Encourage long-term growth for U.S. enterprises and promote efficiency and economic competition through harmonization of standards. d. Further the policy of reliance upon the private sector to supply Government needs for goods and services.

23

bLinux server revenues surpassed $1 billion in quarterly factory revenue for the first time in 3Q04.Q Source: IDC http:// www.idc.com/getdoc.jsp?containerId=pr2004_11_22_120318.

Common and repeated use of rules, conditions, guidelines, or characteristics for products or related processes and production methods, and related management systems practices. The definition of terms; classification of components; delineation of procedures; specification of dimensions, materials, performance, designs, or operations; measurement of quality and quantity in describing materials, processes, products, systems, services, or practices; test methods and sampling procedures; or descriptions of fit and measurements of size or strength.

We note in passing that if we consider Open Source software under this light, we have directly and completely the ability to examine common all of the above properties at any level of detail. In contrast, proprietary software can arbitrarily hide one or more aspect of its properties from standardization. This is not to say that mere publication of source code automatically elevates that which was published to the level of a Voluntary, Consensus Standard, but the availability of source code means that those defining the standard can make their decisions independent of the artifacts the software vendor may choose to detail. OMB A-119 does not directly define bOpen Standards,Q but it does define bVoluntary, Consensus StandardsQ as being defined by the following attributes: (i) (ii) (iii) (vi) (v)

Openness. Balance of interest. Due process. An appeals process. Consensus, which is defined as general agreement, but not necessarily unanimity, and includes a process for attempting to resolve objections by interested parties, as long as all comments have been fairly considered, each objector is advised of the disposition of his or her objection(s) and the reasons why, and the consensus body members are given an oppor-

M. Tiemann / Computer Standards & Interfaces 28 (2006) 495–507

tunity to change their votes after reviewing the comments. Presumably, these are in priority order, and hence presumably bOpennessQ is important. Yet nowhere does OMB A-119 explain what bOpennessQ actually means. This confirms our hypothesis that while the OMB also seeks the benefits of Open Standards, it does not rise to the level of defining them.

Appendix 2 If we can identify the mechanism by which the long- and short-term economic benefits of an Open Standard are obtained, we can provide a meaningful definition of what is, or is not, an Open Standard. Metcalfe’s Law declares that if there are N users in a potential network, the value of the network is proportional to the number of interconnections that can be made in the network. Thus, if nobody can talk with anybody, the value of the network is zero. If a person can talk only to the person to their right, the value of the network is the sum of those N discrete connections. But if everybody can talk with everybody, the value of the network is proportional to N^2, the total number of two-way connections possible. As the Internet has demonstrated, large interconnected communities are vastly more valuable than small, isolated communities. Open Standards are those things that promote the network effect–an effect widely recognized and greatly valued–while proprietary standards are interfaces that exclude connections, hence disrupt or destroy the network effect. Consider three competing, incompatible networks run by three market participants, A, B, and C. According to Metcalfe’s Law, the total economic value of these networks is approximately the sum of the values of these discrete networks: A2 þ B2 þ C 2 If the three networks were perfectly interoperable, the value would be approximately ð A þ B þ C Þ2

505

Alan Cox, a Red Hat Fellow, proposes these two equations can be related by considering an incompatibility factor, X:
A2 þ B2 þ C 2 4X þ ð A þ B þ C Þ2 4ð1  X Þ When internetworking is useless, X is near 1 and when internetworking is seamless and efficient, X is near 0. Mathematically, for values greater than 1, the sum of the squares is strictly less than the square of the sums, so total network value is maximized when X is 0. However, the economic value of each opponent (as typically set by the capital markets) is a function of both the value of the network and their relative share of the network. When all opponent’s shares are equal, their relative shares are the same whether X is 1 or 0, though the value of the network is approximately three times greater when X is 0 instead of 1. In that case, all opponents have an incentive to cooperate on interoperability because they all stand to gain from the growth of the value of the network and none stand to lose because of relative valuations. Ironically, this is the reverse of what happened in the Unix market in the 1980s and 1990s. Although AT&T licensed its source code to virtually all the major computer vendors (making them Unix vendors in the process), these vendors began introducing incompatibilities that first fragmented, then virtually destroyed the economic value of Unix. In spite of spending years and millions of dollars developing bSpec 1170,Q the so-called Open Standard that Unix was supposed to be, the real compatibility and interoperability that enable the network effect degraded to the point where they became almost non-existent. And with a correlation that would make any economist proud, the failure induced by a lack of true Open Standards was mirrored by the decline and fall of Unix market share growth from 1990 to the present day. Interestingly, while the GPL (and numerous open source licenses for that matter) grants the freedom to modify software, hence the mechanism to fragment technology, the GPL also protects the freedom to share innovations, however received. Consider the impact the GPL has had on the Linux kernel: because market rivals can so easily adopt innovations of their competitors, commercial Linux kernels

506

M. Tiemann / Computer Standards & Interfaces 28 (2006) 495–507

are more alike than different; if there are differences, it is because vendor X does not like what vendor Y did, not because vendor Y can prevent vendor X from using it. For this reason, there are hundreds of Linux distributions compatible with, if not identical to, the Linux market leader, meaning that the network effect is in full force, and that vendors are as incented to grow the total market as they are to compete against one another. When one opponent has significantly more share than the others, that opponent enjoys a much larger share of the value when X is 1 then when X is 0. Consider what happens when opponent1 has 70%, opponent2 has 20%, and opponent3 has 10% share. opponent1’s 70% number share becomes 91% value share when X is 1 because 91% of the viable connections are contained within opponent1’s network reach, as this table illustrates: Total value

A B C

Mkt %

70 20 10

X=1

X=0

1.0

1.85

Network %

Network %

90.7 7.4 1.9

129.6 37.0 18.5

% Gain

43% 400% 900%

In this scenario, the total value of the network would be 85% greater when interoperability is perfect, but would opponent1 give up the market and pricing power of being an effective 91% share holder, 10 larger than the rest of the competition combined, in order to access the additional 85% that interoperability would allow? History of the Unix market and current practice tells us they would not, even though the economic benefit could be as much as 42% more (or 20% more after pricing a 20% discount into the goods sold to a larger market). In contrast, because interoperability is easily achieved under the GPL, a semi-dominant player effects no interoperabilty lock-out, and all players have full access to their share of the greater value of the market. A corollary to this analysis is that the longer it takes to switch from the major supplier to standards, the more likely the market is to become a monopoly (because the sum of users of the rivals is not enough to counteract).

Consider this hypothetical case of three incompatible file formats for word processing (we will treat the subject of open file formats as a special case of Open Standards later in this document): Total Value

A B C

Mkt %

93 4 3

X=1

X=0

1.0

1.15

Network %

Network %

99.7 0.2 0.1

107.2 4.6 3.5

% Gain

8% 2400% 3233%

Why would opponent1 give up a virtual monopoly on the market and leave themselves exposed to competition merely so as to enable the market to increase 15% in total value (and of that increase, grant half of it immediately to the competition)? Thus, while the macroeconomic benefit of Open Standards is clear, so are the microeconomic reasons why dominant players would seek to prevent or subvert such standards. And this is why it is so important to be careful when accepting any definition of interoperability and especially any promises that a standard makes about interoperability (including the ones proposed herein!).

Appendix 3 What follows are the best alternative attempts I have found to define bOpen StandardsQ Excerpted from Bruce Perens web page Open Standards Principles and Practice 24 1.

Availability: Open Standards are available for all to read and implement. 2. Maximize End-User Choice: Open Standards create a fair, competitive market for implementations of the standard. They do not lock the customer in to a particular vendor or group. 3. No Royalty: Open Standards are free for all to implement, with no royalty or fee. Certification of compliance by the standards organization may involve a fee. 24

See http://perens.com/OpenStandards/Definition.html.

M. Tiemann / Computer Standards & Interfaces 28 (2006) 495–507

4.

5.

6.

No Discrimination: Open Standards and the organizations that administer them do not favor one implementor over another for any reason other than the technical standards compliance of a vendor’s implementation. Certification organizations must provide a path for low and zero-cost implementations to be validated, but may also provide enhanced certification services. Extension or Subset: Implementations of Open Standards may be extended, or offered in subset form. However, certification organizations may decline to certify subset implementations, and may place requirements upon extensions (see Predatory Practices). Predatory Practices: Open Standards may employ license terms that protect against subversion of the standard by embrace-and-extend tactics. The licenses attached to the standard may require the publication of reference information for extensions, and a license for all others to create, distribute, and sell software that is compatible with the extensions. An Open Standard may not otherwise prohibit extensions.

The Open GIS Consortium (OGC) defines an open standard as [emphasis from OGC site]:25 1.

2.

3.

[Created] in an open, international, participatory industry process, as described above. The standard is thus non-proprietary, that is, owned in common. It will continue to be revised in that open process, in which any company, agency, or organization can participate. Has free rights of distribution: An bopenQ license shall not restrict any party from selling or giving away the specification as part of a software distribution. The bopenQ license shall not require a royalty or other fee. Has open specification access: An bopenQ environment must include free, public, and open access to all interface specifications. Developers are allowed to distribute the specifications.

4.

5.

Michael Tiemann is a true open source software pioneer. He made his first major open source contribution over a decade ago by writing the GNU C++ compiler, the first native-code C++ compiler and debugger. His early work led to the creation of leading open source technologies and the first open source business model. In 1989, Tiemann’s technical expertise and entrepreneurial spirit led him to co-found Cygnus, Tiemann contributed in a number of roles from President to hacker, helping lead the company from fledging start-up to an admired open source leader. Tiemann serves on a number of boards, including the Open Source Initiative and the GNOME Foundation. Tiemann also provides financial support to organisations that further the goals of software and programmer freedom, including the Free Software Foundation and the Electronic Frontier Foundation.

27

See http://www.opengis.org/resources/?page=faq#37.

Does not discriminate against persons or groups: bOpenQ specification licenses must not discriminate against any person or group of persons. Ensures that the specification and the license must be technology neutral: No provision of the license may be predicated on any individual technology or style of interface.

By this definition, a de facto standard established by one company or an exclusive group of companies or by a government is not an Open Standard, even if it is published and available for use by anyone at no charge. The Web, and the Spatial Web, cannot depend on proprietary standards. The Open Group26 requires 27 that a standard be open according to a set of Interface Adoption Criteria28, but it also provides substantial wiggle room around these criteria. It says that bany standard recorded in the Standards Information Base [m]ust be an open standard as defined by the Open Group interface adoption criteria.Q But in the criteria it also says bNot all of the criteria defined are absolute requirements, but all must be considered before the final decision is taken on adoption of a specification.Q Finally, it does not say objectively whether something meeting these criteria is less than, more than, or precisely an open standard, the very state of affairs that this paper attempts to remedy.

26

25

507

28

See http://www.opengroup.org/. See http://www.opengroup.org/tech/procedures/standards.htm. See http://www.opengroup.org/tech/procedures/iac.htm.