Technology in Society 26 (2004) 385–390 www.elsevier.com/locate/techsoc
Engineering ethics and society Wm. A. Wulf National Academy of Engineering, 2101 Constitution Ave. NW. Washington, DC 20418, USA
Abstract The engineering profession has a strong tradition of ethical standards for its members. However, the practice of engineering is changing in ways that raise ethical issues not previously considered. These ‘‘macro ethical’’ issues are ones for the profession as a whole rather than for individual engineers. This paper lays out some of the issues as a way of encouraging further discussion. # 2004 Elsevier Ltd. All rights reserved. Keywords: Engineering; Ethics; Compexity
At the annual meeting of the National Academy of Engineering (NAE), the president is expected to present a brief lecture on a relevant engineering issue. For the 1999 meeting, in anticipation of the new millennium, I thought it would be appropriate to talk about the achievements of engineers in the twentieth century and the challenges facing them in the twenty-first. Preparation of the first part of the lecture was relatively easy. We had worked with the engineering professional societies to create a list of the twenty greatest engineering achievements of the twentieth century—achievements we had selected based on their impact on people’s lives rather than for their technological ‘‘gee whiz’’. The resulting list1 is impressive and includes electrification, the automobile, the airplane, clean water, electronics, radio and television, agricultural mechanization, computers, the telephone, air conditioning and refrigeration, and ten more. It is striking that our lives are dramatically different and (mostly) better than those of our ancestors because of these engineering innovations. However, preparing the second half of the lecture was more difficult, at least at first. As Niels Bohr said, making predictions is difficult, especially predictions
Tel.: +1-202-334-3201; fax: +1-202-334-1680. E-mail address:
[email protected] (W.A. Wulf). 1 The list can be seen by following the link on the lower left side of the NAE website: www.nae.edu.
0160-791X/$ - see front matter # 2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.techsoc.2004.01.030
386
Wm.A. Wulf / Technology in Society 26 (2004) 385–390
about the future. Conventional wisdom about the near-term future of technology suggests that we there is a wealth of technical opportunities and challenges that will continue to transform our lives. But my crystal ball is no better than anyone else’s, and it goes completely foggy after a dozen years or so. How could I anticipate the challenges for a full century? When I looked back at the list of the twentieth-century achievements, I was struck by two things. First, I was awed by how much the work of engineers matters. Arguably, engineers and their creations did more to shape our lives than anyone or anything else in the last century! Just imagine how our lives would be different if even one thing were missing from that list—electricity, for example. Second, I realized that the immense social impact of most of our inventions was not predicted by their inventors. As Norm Augustine said: ‘‘The bottom line is that the things engineers do have consequences, both positive and negative, sometimes unintended, often widespread, and occasionally irreversible’’ [1]. Because of the enormous impact of engineers on individuals and society, they also bear deep moral and ethical responsibilities. That realization changed the nature of my quest, and in the end I decided I would pose only one challenge for the twenty-first century—engineering ethics. Specifically, I am convinced that the nature of engineering is changing in ways that raise new kinds of ethical issues, ones that engineers have not faced before and hence have not thought deeply about. Those new issues, I believe, have ramifications not only for the behavior of individual engineers, but also for the profession as a whole and, indeed, for the process of engineering itself. In this article I would like to elaborate on that theme and its implications for both the engineering profession and society. Before proceeding, let me be clear that I believe the vast majority of engineers behave ethically; I do not believe we are facing some sort of crisis or moral decline. The purpose of this paper is to explore the ethical implications of new circumstances and to help the profession and society prepare to deal with them, not to suggest that there is a current problem. Nonetheless, the issues I raise are complex, extremely important, and increasingly urgent, so we need to get on with thinking them through. Let me start by saying that I believe the new ethical issues are ones for the profession rather than ones for the individual. Issues for the profession are called macro ethical questions in contrast to those for the individual, which are called micro ethical questions. (This is not to suggest that the macro questions are more important than the micro ones; they are not!) An example from medicine may help explain the difference. In medicine the micro ethical issues are much like those in engineering. The ‘‘First do no harm. . .’’ of the Hippocratic oath is much like the ‘‘Hold paramount the safety, health and welfare of the public. . .’’ of the NSPE code.2 Both focus on the behavior of the individual practitioner. 2
The National Society of Professional Engineers (NSPE) code is the model for the ethics codes of many other professional engineering societies.
Wm.A. Wulf / Technology in Society 26 (2004) 385–390
387
Medicine has had to face other macro ethical problems, such as the allocation of scarce organs, drugs, or physicians’ time. When there are fewer organs than patients needing transplants, who gets them? Problems like allocation are not for the individual doctor to decide; rather, the profession, or perhaps society guided by the profession, needs to make these decisions. These are the macro ethical issues, the kinds that changes in the nature of engineering practice (and our perception of it) are raising. Enough generalities! Let’s consider some specifics. A class of macro ethical questions arise because of our increasing inability to predict all the behaviors of the systems we build, and the near certainty that some unpredicted behaviors will be deleterious and possibly catastrophic. How does one engineer such systems ethically? The inability to predict all the behaviors of a system may arise from several sources: 1. Complexity. Any sufficiently complex system will have emergent properties— those that cannot be predicted a priori. Starting from quantum mechanics, one would not predict the personality of Bill Wulf, for example; that personality is an emergent property. There is an increasing number of engineered systems that have reached the threshold of complexity where emergent properties occur, for example, current proposals to ‘‘remediate’’ the effects of previous man-made changes to the Everglades. The complexity of the Everglades ecosystem is such that one cannot completely understand, much less predict, the outcome of all the interactions of its components, and hence of an engineered change to them. Similarly, many of the bugs reported in computer software are not errors or mistakes in the way we usually use those words, but emergent properties that are impractical to predict a priori. 2. Chaotic systems. Even when we have a nice, correct mathematical model of a system, it may be chaotic, that is, the long-term behavior of the system may be exquisitely sensitive to its initial conditions. The classic example of the flap of a butterfly’s wings that just might cause a hurricane thousands of miles away suggests that, although unlikely, the consequences of our engineering decisions can indeed be catastrophic. 3. Discrete systems. For both of the above examples, because their mathematical descriptions are continuous, we can at least reliably predict their near-term behavior. That is, because of the old argument from our introductory calculus class, ‘‘For every E there exists a d. . .’’, we know that for physical systems, small changes in the input produce small changes in the output. Unfortunately, not so for discrete (digital) systems. A one-bit change in the pattern of 0s and 1s in my computer memory can completely change the interpretation of all the others. Thus a small change in the ‘‘input’’ can cause an huge change in the ‘‘output.’’ Coupled with the enormous number of possible states (patterns of zeros and ones),3 the lack of continuity in digital systems not only prohibits effective 3
There are on the order of 10100,000,000,000,000,000,000 states in the memory of my 64-megabyte laptop. By comparison, there are something like 10120 atoms in the universe!
388
Wm.A. Wulf / Technology in Society 26 (2004) 385–390
prediction of their behavior but inhibits exhaustive testing processes as well. For physical systems, one can test a representative sample of states and let continuity ensure than others will behave similarly—but not so for digital systems. It is important to understand that in each of the cases above, it is impossible to predict all of the behaviors of the systems under consideration. Not that it’s hard to predict them, and if only we were more careful we could do it. No, it’s impossible! Because the systems we engineer are becoming ever more complex and are increasingly critically dependent on digital systems, we are now building systems with behaviors we cannot predict. How do we ethically engineer such systems? The answer is not ‘‘don’t do that.’’ Consider the remediation of the Everglades, for example. We know that past actions by developers and others have damaged the ecosystem. So doing nothing is essentially a decision to perpetuate the damage. Yet there is no way to know with absolute confidence what different and possibly worse damage might result from any particular action. Moreover, we have plenty of evidence of well-intentioned decisions—‘‘the best we knew at the time’’—turning out to have unpredicted and negative consequences, for example, damming western rivers to prevent flooding. What, then, is the ethical thing to do about the Everglades, and how do we proceed in a manner that minimizes the possibility of unpredicted damage? Even doing nothing is in fact doing something, and doing anything may have an unpredicted negative outcome. So what is ethical behavior? Another example of macro ethical questions concerns the combined effect of many individual decisions. I don’t think anyone would suggest that individual engineers working for automobile manufacturers are unethical simply because of their employment. Yet 40,000 Americans die each year because of automobile accidents; the emissions from those autos contribute to the increase of greenhouse gasses that might (or might not) cause global climate change. I can’t quite bring myself to say that collectively we engineers have no responsibility for the deleterious effects of the cars we design. Nor can I argue that any particular individual engineer is responsible. In somewhat the same vein is the collective effect of many local decisions on our global environment. Paving one parking lot and blocking the entrance to an aquifer that is used hundreds of miles away will not destroy the environment, nor will the diversion of one stream to provide irrigation for a farm field. But thousands or millions of such decisions can. Moreover, the really important effect may not be the direct one, but one resulting from the highly connected and interdependent web of ecological relationships. As Brad Allenby has pointed out, the earth is already an engineered artifact because of just such decisions, and it is high time for the engineering profession to (1) take responsibility for the consequences, and (2) devise a process of engineering that takes them into account when making local decisions [2,3]. In a sense, these examples of macro ethical issues are not new. There have certainly been engineering decisions in the past with enormous societal impact, but it does seem like we are crossing a qualitative threshold. For instance:
Wm.A. Wulf / Technology in Society 26 (2004) 385–390
389
. The machines that my father designed could not achieve sufficient complexity to exhibit emergent properties; today a single computer engineer or bio-engineer can, and routinely does, build systems of that complexity. . The collective effect of our use of fossil fuels is having a measurable effect on greenhouse gas concentrations which, combined with recent evidence of rapid climate change in the past, raise a worrisome specter. . Because of their low cost, digital systems are being introduced into the control of nearly every engineered product and process. Because of the inability to thoroughly test any but the most trivial of these systems, there will be catastrophic failures. The question, then, is what should we do? I have the sense that engineering will change profoundly over the next few decades in response to these issues, although I have no idea what the nature of the change will be. Certainly our concept of engineering ethics must expand beyond the cozy codes of individual behavior we are familiar with. We will have to establish a new understanding with society about what constitutes ethical behavior on the part of the profession. But it is much larger than that. The very process of engineering design, which evolved in an environment where all behaviors of a system could be predicted, will have to change to accommodate an environment where they cannot be predicted and hence one that supports our new societal compact. Even the notion that one can completely specify the properties of a system before building it may need to be re-examined unless some new approach allows us to bound our uncertainty. If we cannot a priori completely specify the properties of a solution to an engineering problem, then engineering will become a more interactive process with society, and the culture of engineering will change as well. Engineering and technology have always been influenced by society as well as influencing it, but the time constants have been large and, except for large infrastructure projects, mostly indirect. Typically, engineers have been reluctant to participate in public policy and political debates, even when there is a major technical component to the issue under discussion. But that may be precisely the form of societal interaction needed in the future. The first step is to recognize that we have a problem, begin to discuss it, and encourage scholarship on it. That was my purpose in this paper. The community of engineers and ethicists must engage in a dialogue; the intersection of these communities is tiny and separately they do not share a vocabulary, much less a scholarly value system. Until we enlarge the community that can communicate with each other through such dialogue, I fear we cannot make progress on the substantive issues we face. I hope to use the NAE to encourage this dialogue, but I would like to urge the professional societies to do so as well in order to bring their disciplinary expertise into the discussion. It is my hope that from these discussions a body of scholarship will emerge which reflects the ethical issues of the changing nature of engineering, and from which we can begin to rethink the way we educate engineers, the way we practice engineering, and the way we engage with society.
390
Wm.A. Wulf / Technology in Society 26 (2004) 385–390
References [1] Augustine N. Ethics and the second law of thermodynamics. The Bridge, Fall 2002. [2] Allenby BR. Earth systems engineering and management. Technology and Society 2000;19(4):10–24. [3] Allenby BR. Observations on the philosophic implications of earth systems engineering and management. Batten Institute Working Paper. Darden Graduate School of Business, University of Virginia, Charlottesville, VA, 2002. Wm. A. Wulf was elected President of the National Academy of Engineering (NAE) in 1997.He is currently on leave from the University of Virginia, where he is a University Professor. His research spans computer architecture, computer security, programming languages, and optimizing compilers. In 1988– 90 Dr. Wulf was Assistant Director of the National Science Foundation. Prior to joining the University of Virginia, Dr. Wulf founded a software company, Tartan Laboratories, based on research he did while on the faculty at Carnegie-Mellon University. Dr. Wulf is a member of the National Academy of Engineering, a Fellow of the American Academy of Arts and Sciences, a Corresponding Member of the Academia Espanola De Ingeniera, and a Foreign Member of the Russian Academy of Sciences. He is also a Fellow of four professional societies: the ACM, the IEEE, the AAAS, and AWIS. He has authored over 100 papers and technical reports, written three books, holds two US patents, and has supervised over 25 PhDs in Computer Science.