A value-oriented policy generation methodology for technology assessment

TECHNOLOGICAL

FORECASTING AND SOCIAL

CHANGE

8,401-420

401

(1976)

A Value-Oriented Policy Generation Methodology for Technology Assessment * MARK BERG, KAN CHEN and GEORGE ZISSIS

ABSTRACT examples of, a value-oriented policy This paper presents the rationale for, and preliminary generation methodology for technology assessment. The 6 step procedure is intended to facilitate the development of normatively based policy options for the technology under assessment. The relationship between policy generation and the other aspects of technology assessment is then discussed along with some recommendations for future research.

Introduction

Of the many definitions of technology assessment (TA) found in the literature, most tend to reflect the concise statement by J. Coates, that TA is “the systematic study of the effects on society that may occur when a technology is introduced, extended, or modified, with emphasis on the impacts that are unintended, indirect and delayed”. [2] Clearly this view represents a major and important thrust in the TA movement. One must beware, however, of taking too narrow a reading of the definition. For a narrow reading would suggest a model of the technology-society interaction which is one sided and overly deterministic. In this model society is basically passive, like a quiet pond; technology is active, like a skipping stone; and technology assessment is a neutral observer, standing on shore reporting the effects of the ripples as they interact and approach their secondary and tertiary impacts on the shoreline. In many cases this may be a helpful, if not totally valid, image. In others, however, we need a more complex image of TA and a less deterministic model of the interaction between technology and society. In this alternative model the reciprocal relation between technology and society is emphasized. Society takes on a more active role, like the turbulent water of a fast moving river. Technology, while still active, is now also acted upon, much like a small boat. And MARK BERG is a Senior Research Associate at the Center for Research on Utilization Scientific Knowledge, Institute for Social Research, University of Michigan. KAN CHEN is a Professor of Electrical and Computer Engineering, University of Michigan. GEORGE ZISSIS is Chief Scientist of the Environmental Research Institute of Michigan.

of

* The basic methodology and examples in this paper were supported by NSF Grant GI-34899 at the Environmental Research Institute of Michigan. The concluding generalization and final documentation was supported by NSF Grant GK-43465 at the University of Michigan. An abridged version of this paper appeared as part of Chapter 3 in Perspectives on Technology Assessment, by Arnstein, Cristakis, et al. [l] .

0 American

Elsevier Publishing

Company,

Inc., 1976

402

MARK BERG. KAN CHEN AND GEORGE ZISSIS

technology assessment, as part of the guidance system, becomes a more active participant subject to an occasional drenching in the turbulent waters of the political arena. From this perspective, the “. . unintended, indirect and delayed, . . .” effects of a technology depend not just on the characteristics of the technology and the society at time to, but also on the particular policy options implemented by society to guide and affect the technology (see Fig. 1). Thus, for this type of technology assessment, methodologies are needed for the generation of policy options to be incorporated into the contextual scenarios needed for assessing impacts. The remainder of this paper is devoted to a description of a policy generation methodology given preliminary development in a recent partial technology assessment of remote sensing of the environment. 1 A policy generation methodology was developed in response to the highly policy-sensitive nature of the problem. Impacts of remote sensing of the environment are not dominated by theoretically deterministic physical/chemical characteristics, such as automotive engines with ‘predictable’ pollutants and health effects. Nor is it a process such as snowfall augmentation with probabilistically predictable physical effects and related environmental consequences. It is instead, a ‘passive’ information technology whose most important impacts will be determined by policies with respect to what information is obtained, who obtains it, and how it is used. While technical capabilities and limitations will set specific boundary conditions for policy options, the impacts which flow from the use of remote sensing will be dominated not by technical considerations but by the choice of policies and the value judgments (often hidden) attached to them. While remote sensing is, perhaps, more policy sensitive than many other technologies, we would argue that some type of policy generation methodology is needed in almost all TA projects due to the policy-oriented nature of technology assessment [2, 31. One cannot realistically talk about the full range of impacts of a technology without identifying those impacts within the context of the range of institutional settings and policy options under which the technology might feasibly operate. (This is, of course, an iterative process in that as impacts or opportunities are assessed, new policy variations may be generated in response.) In the long run, TA will be judged to be a successful enterprise only to the degree to which it aids societal decision-makers to identify and choose technology policy options which facilitate achievement of societal goals while inhibiting the potential for unintended negative effects. This requires that TA devote significant, serious and creative effort to the generation of policy options. Policies are the instrumental judgments which arise from the complex interplay between reality judgments of the possible and value judgments of the desirable [4]. Thus, policies are clearly normative, not neutral or value free. The scientific tradition would suggest that underlying assumptions and propositions should be explicitly stated for all to understand and judge. From this perspective then, a policy generation methodology should acknowledge its underlying normative components and expose its value-oriented content as explicitly as possible. A strong case can be made that without this explicit

‘Remote sensing, the technology for acquisition of information about objects or events at an appreciable distance from the information-gathering or sensing system, has been developed with special intensity over the past ten years for gathering information needed for environmental monitoring and resources surveys, an application called Remote Sensing of Environment. Perhaps the most striking example of this technology is the pair of Earth Resources Technology Satellites which have been successfully operated by the National Aeronautics and Space Administration (NASA) since July 1972. [2a].

METHODOLOGY

A.

Keciprocal

and

society

FOR TECHNOLOGY

relationship between witbout technology

403

ASSESSMENT

technology assessment. B.

Technology assessment modifies technological development.

certain

Fig. 1. The role of technology assessment. exposure of values and goals the analysis may be subject to drift, inadequate evaluation, and charges of antidemocratic elitism and manipulation. However, technology assessment is not just a scientific/technical enterprise; it is also political. And our political tradition would suggest that under!ying assumptions and propositions (especially values) are often best left unsaid or ambiguous in the interest of building the winning coalitions needed for democratic action. One cannot ignore the political costs which may accrue to an explicit statement of values or goals. Thus, it can be expected that in some TA areas where the issues, impacts and policy options are heavily politicized, a normative approach to policy generation, as presented here, may not be the most useful approach. In these situations, TA’s attempting to operate from a commonweal perspective, may have all the controversy their scientific/technical expertise can handle just from the identification of potential impacts. However, the normative approach might still be useful here for the internal use of various stakeholder groups operating from a more advocacy oriented model. As suggested above there are alternative models for performing TA ranging from uninhibited advocacy to the more commonweal, or public interest, perspective which attempts a more apolitical, evenhanded public image. From this latter perspective especially, the effectiveness, or power, of TA within the social-political decision-making process depends heavily on the perceived legitimacy of its scientific/technical expertise [5, 61. TA cannot avoid dealing with values, but that is not its area of ‘expertise’. Therefore, it has been our working hypothesis that to protect its legitimacy and its utility, TA and its policy generation methodologies, when operating from a commonweal perspective, should attempt to be ‘values explicit’ and ‘evenhanded’, and neither pretend to be value free nor try to be value advocates. For this reason and for the additional reasons discussed immediately below, the policy generation methodology used in our remote sensing project did attempt to be value explicit and evenhanded. The particular procedure was stimulated by a general approach briefly described by Harold Lasswell in A Preview of Policy Sciences [7]. Starting from this very general approach we have developed a procedure and applied it to the policy and institutional issues surrounding remote sensing of the environment. The appeal of the procedure is two-fold. First, it provides a systematic, yet flexible, method for organizing the large volume of multidimensional data relevant to our assessment. Second, and more importantly, it provides the necessary linkage between the social value orientation we have chosen to emphasize and the policy-and-action orientation which we feel is appropriate for TA. There are a number of additional important reasons for placing an emphasis on values in TA. And despite the problems presented by the fact that the general areas of values, value change, value acquisition, etc., are terribly complex and not well understood, we

404

MARK BERG, KAN CHEN AND GEORGE

ZISSIS

feel it is important to start and hopefully provide a basis for encouraging further work with respect to the value implications of new technologies. A major impetus for investigating a value-oriented approach stems from the practical need for bounding a TA project. It is impossible to identify and assess all possible impacts of a technology exhaustively. In addition to the practical constraints of time and money available to any assessment project, we are faced with our very limited ability to trace accurately the impact of any particular new technology, despite the increasing repertoire of methods available for such purposes [1, 81. Our formal models of the impacts of technological innovation are simply nonexistent, inadequate or untested. And our informal mental models are usually fuzzy and inconsistent, and simply inadequate to deal with the complexity and emergent nature of the socio-technical environment. In view of these limitations on the exhaustive approach we adopted the hypothesis that by identifying families of important affected values, we can both aid policy-makers in understanding the sometimes hidden value implications of their policy options and also give insight into the kind of indicators most appropriate for monitoring the emergent impacts of an ongoing technological innovation. In this manner, the elaborations of the reality system and the value system of the policy-makers would proceed together, as far as their policy considerations with respect to the new technology are concerned. And this is critical since, to quote from Sir Geoffrey Vickers, “Facts are relevant only to some standard of value; and values are applicable only to some configuration of fact” [9]. One final benefit in emphasizing a value-oriented approach stems from the need for getting coherent outputs from an interdisciplinary project team. Milton Rokeach makes the point well [lo] : Consider,. . ., the relative ubiquitousness of the value and attitude concepts BCIOSS disciplines. While attitudes seem to be a specialized concern mainly of psychology and sociology, values have long been a center of theoretical attention across many disciplines-philosophy, education, political science, economics, anthropology, and theology, as well as psychology and sociology. All these disciplines share a common concern with the antecendents and consequents of value organization and value change. Around such a shared concern we may realistically anticipate genuine interdisciplinary collaboration.

We have found Rokeach’s anticipation to be well-founded, and we can add to his list the concerns of engineers, lawyers, physicists, geographers, environmentalists, and planners. The problems of disciplinary and team integration are critical to the long-run success of TA, and as a focus of shared disciplinary and individual concern the value-oriented approach proved to be a useful source of group integration. As described below, our procedure for policy analysis is clearly value-oriented in the sense that it translates specific value-based goals into operational policy options. Obviously, not all sectors of society will share all of the goals selected to be operationalized, and consequently not all sectors will find relevance or satisfaction in the resulting policy options. It is part of the final policy evaluation process and the political process to examine these value conflicts from a variety of perspectives and look for compromises and innovations which can facilitate overall goal achievement, while still respecting and protecting the value differences among subsectors. In our project, six general societal values were selected for analysis: survival, distributive justice, privacy, material welfare and consumption, law abidingness, and rationality. These were chosen based on a subjective estimate of (1) the degree to which they might be impacted by the use of remote sensing, and (2) the degree of interest (on the part of policy-makers and the

METHODOLOGY

FOR TECHNOLOGY

ASSESSMENT

405

public) in the relationship between these particular values and remote sensing of the environment. Clearly we have not encompassed all relevant values in our selection, but that was not our intent here. We are instead merely trying to test the feasibility of the methodology with the expectation that any future study would consider a larger set of relevant values after consultation with policy-makers and stakeholder groups. In the discussion below, we describe the six major steps of our policy generation methodology and relate the insights, problems, and limitations we have encountered in its preliminary use. Examples of the results of the first five major steps, for the general societal values of survival, justice, and privacy, as impacted by remote sensing, are summarized in the attached tables (see Appendices A, B, and C). Description

of Procedure

1. GOAL CLARIFICATION

Starting from general societal values (or values of a subset of the society), we ask, What current or potential events or trends in society (including, but not limited to, those caused by the introduction of remote sensing) pose potential threats to the realization or maintenance of the desired value?-e.g., nuclear war is a threat to survival. Secondly, we ask, What operational goals for the use of remote sensing would enhance the general values by either relieving the potential threats or by taking advantage of new opportunities for value realization?-e.g., the use of remote sensing for off-site inspection allows for improved enforcement of, and security in, nuclear non-proliferation treaties. Threats and promises are, of course, interrelated to the degree that there is usually the opportunity to alleviate a threat and usually the threat of missing an opportunity. Our objective in this first step is to provide criteria and operational goals for the use of remote sensing based on a foundation of societal values. In doing so we attempt to relate the maintenance or realization of specific values to identifiable societal problems (i.e., threats to values) or opportunities (i.e., improved value realization) in which remote sensing might play a part through its primary, secondary, or tertiary impacts. By creating this linkage between abstract values and more concrete problem/opportunity sets, we translate the analysis to a more operational level at which analysis of data and specific policy options are more readily handled. This operationalization of goals is characterized by greater clarity and precision than the often elusive character of abstract values. This crystallization of, and strong linkage to, goals is a critical step in insuring that the policy process does not lose sight of its overall objectives because of immersion in instrumental details. By definition, the analysis is a value laden one, and obviously not everyone will agree with the values chosen for analysis or the ways in which those values are expressed and interpreted. Such disagreements can be expected in all policy-oriented processes because of their inherent content of value judgments. The importance of the goal clarification step in this analysis is that the value judgments being operationalized are made explicit, and thus, available for challenge and negotiation by policy-makers and stakeholders. 2. GOAL REALIZATION

STATUS

The objective of this step is to describe the extent to which events and trends have thus far approximated the desired operational goal. This is done by identifying and describing relevant events, budgets, legislation, attitudes, programs, etc. (e.g., remote

406

MARK BERG, KAN CHEN AND GEORGE

sensing is currently used for off-site arms inspection This step, with its short summary type descriptions, data are available. 3. ANALYSIS

ZISSIS

and R&D is continuing in that area). draws heavily on whatever empirical

OF CONDITIONS

In this step we attempt to identify and analyze those factors which have causal or influencing connections with the events and trends described in step 2. Special attention is paid to those factors which represent “leverage variables”, [l I] (i.e., those variables which are amenable to policy control and provide efficient leverage for influencing events and trends toward desirable outcomes). An example of such a leverage variable in our study would be the R&D budget for surveillance satellites. Once identified and understood, these variables may provide valuable insight into potential secondary and tertiary impacts, as well as stimulation for the identification of policy options in step 5. In searching out relevant factors, we often found it helpful to stimulate our thinking by organizing our analysis in terms of technological, economic, and socio/political factors and by raising a series of keyword issues and questions. For example, what trends or factors represent obstacles to goal attainment? And conversely, what incentives are operating toward goal attainment ? What are the relevant time frames and magnitudes, and what legal, economic, and budgetary factors are involved? Who are the involved groups, and what are their relationships? Who has the responsibility, control, and power? What are the influencing historical factors, social attitudes, and traditions? And finally, what technological factors such as complementary and competitive technologies will be influencing factors? Not all of these questions can be answered, nor are they appropriate in every case. The reader can, no doubt, think of other questions which would help fulfill the general purpose of stimulating thought. In many cases the answers will be complex, uncertain, or simply unknown. In our own attempts to work through the procedure, we have in fact found that many questions were either beyond our collective knowledge and expertise or at least require extensive research well beyond the time available. However, even when adequate answers cannot be provided, this step remains critical to the overall process as an important mechanism for structuring areas of certainty and uncertainty within the analysis. This structuring of uncertainty is a necessary input for the projections of step 4 and for those policy options of step 5 aimed at study, evaluation, and monitoring. 4. PROJECTION

OF DEVELOPMENTS

In this step we attempt to forecast the probable future goal realization status based on the trends and factors previously identified-assuming no significant policy changes and no significant surprises caused by external events. In much the same way as step 2, the forecasts proceed in terms of expected events, economic factors, legal factors, technological factors, attitudes,‘programs, etc. For example, it can be expected that satellites will continue to be used for arms surveillance and will be upgraded in quality. Our experience, after carrying out this step a number of times, has been that several types of forecasts seemed helpful. The first involved statements resulting from extrapolation of current trends and influencing factors. These statements usually take the general form of “X will occur” or “it can be expected that . . .“. They do not claim to predict what will in fact occur, but rather give insight into what, in our judgment, is the most probable state of affairs if the same forces continue to operate. It is not necessarily our assumption, however, that these same forces will continue to operate.

METHODOLOGY

FOR TECHNOLOGY

ASSESSMENT

407

The second type of forecast usually takes the form of “there is a potential for . . .” Here we deal with events of lower probability which would have significant impact if they did, in fact, occur. It is especially this type of forecast that brings many of the secondary and tertiary impacts into the policy analysis so that countervailing forces can be developed through appropriate policy options. A simple example would be the possibility that an arms surveillance satellite would be attacked and destroyed, leading to a serious confrontation exacerbated by a lack of precedent or contingency planning. One final type of forecast was also found to be appropriate. It tends to involve issues with high degrees of uncertainty and usually takes the general form of “it is uncertain whether or not . .“. The uncertainty may be the result of the nature of the issue or event, or it may stem from lack of knowledge or awareness on the part of the analyst. Especially in the latter case, the forecast might be substantially improved by extensive interview or Delphi [ 121 questioning of involved parties. In view of the limited and preliminary nature of this project, no such techniques were employed. Instead, we merely settled for consensus on the part of project personnel. In general, the time horizon for these projections is in the short to medium term range of 2-10 years. However, where it was judged that additional insight could be gained through a longer time horizon, appropriate statements were included. It should be stressed once again that these projections are not intended to predict the future. Rather, technology assessment, in general, and the policy options of the next step, in particular, are specifically intended to modify the trends and factors which have led us to these projections. The probability of undesirable projections actually occurring and/or the potential for positive goal realization can be significantly altered by the judicious choice of leverage variables and policy options.

5. IDENTIFICATION

OF POLICY

OPTIONS

In this step we attempt to identify alternative policy and institutional options for realizing the desired operational goal. In a sense we create a “menu” of policy options [ 1 I] which can be used to make preliminary policy choices as well as to determine what general options should be more fully detailed, studied and evaluated. It should be stressed that the outputs of this step represent options to be elaborated and evaluated, not recommendations. Some of the options are aimed at broad program goals, while others are aimed at achieving subgoals or affecting secondary and tertiary impacts. In most cases several options may be listed, all aimed at the same goal. Such options will sometimes be conflicting and incompatible, while in other cases they can be combined into compatible parallel programs. In general the options will reflect the leverage variables identified in step 3, and will thus concentrate on economic factors, potential regulations and legislation, treaties, education, etc. It is not the objective of this step to identify those policy options which while consistent with the goals of one value (e.g., survival) are conflicting with the goals of other important values (e.g., privacy) under consideration. This critical and problematic function of balancing one value against another conflicting or inconsistent value is left to the evaluation of step 6, and finally to the judgment of the appropriate policy-makers. The purpose here is to provide a wide range of goal-oriented policy and institutional options which can be synthesized with the options flowing from other goals, leading to a more holistic and comprehensive evaluation in terms of effectiveness, goal conflicts, barriers, acceptable compromise, further research, social experimentation, etc.

408

MARK BERG, KAN CHEN AND GEORGE ZISSIS

6. SYNTHESIS AND EVALUATION The

synthesis

OF POLICY OPTIONS

and

evaluation step is, perhaps, the most difficult one. Strictly speaking, this step is beyond policy generation as it is the synthesis and evaluation of the policies which have been generated in the first five steps. It is also the step in which the least progress was made in our project, and hence is not illustrated in the attached table. However, we do envision that the synthesis and evaluation step would be comprised of at least four overlapping substeps. The first substep involves the fairly mechanical process of bringing together the individual policy options flowing from each of the operational goals and structuring them into relevant categories such as time frame (i.e., short- or long-term policy), levels of action, or actors involved (i.e., legislative, judicial, executive, or private sector, state, national, or international agencies, etc.), and readiness for implementation (i.e., immediate, far-off, problematic, etc.). The second substep would examine the interaction between policies in terms ofvalue conflicts and inconsistencies. Here we would evaluate each value-based policy in terms of its impacts on other values. For example, a specific policy designed to upgrade privacy might be found to have a negative effect on survival and law abidingness. Such evaluations would be difficult and, no doubt, controversial; but where possible, it would be desirable to present this information in a format consistent with that of the attached tables. The third substep would involve an evaluation of the expected effectiveness of the various options. Here a historical review of the experience of similar policies would be useful, along with a review of current literature on the subject. One might examine factors such as incentives for (and ease of) implementation, along with political feasibility, available leverage and enforcement power. A natural outcome of this step would be recommendations for further study and/or monitoring of pilot projects, etc. The fourth step would attempt to bring together and evaluate the implications and side effects of particular options or sets of options. That is, the policy options provide a contextual setting or scenario for the traditional TA study “. . . of the effects on society that may occur when a technology is introduced, extended, or modified, with emphasis on the impacts that are unintended, indirect and delayed” [2] . In this step, in particular, one would like to have close interactions with the relevant policy-makers and affected parties. One output of this step would be new policy options or, at least, new factors which would be fed back into the next iteration of the policy analysis process. Once again, none of these substeps has been accomplished in a systematic or rigorous fashion, and none is presented in the attached tables. They do, however, represent the types of analyses which we feel would be helpful to those making choices and recommendations among alternative policy and institutional options. In this same spirit of work yet to be done and methods still to be evaluated we conclude with some comments relating the policy generation process to the rest of TA, and to the future directions of TA and the policy sciences in general. Conclusions

Most of the conventional methodologies in systems science (systems cost-benefit analysis [ 141 , decision analysis [ 151, mathematical modeling ming [16] , etc.) have put emphasis on the quantitative and comprehensive the policy consequences which have already been identified and the policy have already been generated. While these methodologies may be useful for

analysis [13], and programevaluation of options which the 6th step in

409

METHODOLOGY FOR TECHNOLOGY ASSESSMENT

our procedure, they fall short of guiding the tasks in the first 5 steps.’ In TA (as in broad policy research [l l] and futures research [19]) it is the identification of policy consequences and the generation of policy options which take on a central importance and are often the most difficult and most poorly performed tasks. It is hoped that the policy generation methodology presented here will aid in this task and stimulate further work in this area. In the end, of course, policy recommendations, decisions, and implementations must be based on the careful evahation of the policy options and consequences. The conventional methodologies in systems science will therefore also play a useful and important role. The coupling of policy generation and impact identification methodologies with the conventional evaluation methodologies in systems science is portrayed in Fig. 2. The upper portion of the diagram shows that the identification of technological impacts and the generation of relevant policy responses are mainly “fan-out” operations. The former is exploratory in the sense that it extrapolates from the given technology to its potential impacts. The latter is normative in the sense that it generates meaningful policy options in order to meet certain goals predicated on a set of social values. The dominant consideration of both fan-out operations is to expand the range of the reality, the value, and the instrumental systems operative in the minds of the policy-makers and the policy researchers. In contrast, the lower portion of the diagram in Fig. 2 shows that the evaluation of policy options and consequences is mainly a “fan-in” operation. The dominant consideration of the fan-in operation is to narrow down to a specific action to be taken by a certain time. Much of the fan-in operation will have to be judgmental and thus largely intuitive. The art of judgment [4] is and will remain qualitative. However, some steps in the fan-in operation, especially the final comparison and evaluation of a manageable number of policy combinations, can be quantitative and explicit by using the conventional methodologies of systems science. This will help in explaining the rationale of the final policy decision, or will facilitate intelligent public debate on the policy issues. The overview provided by Fig. 2 suggests several broad directions for future research in methodologies for technology assessment. In the area of impact identification methods3 there is a need for more effective utilization of the well-established methods within the broad disciplinary aspects of social, example, the time proven methods consumer

survey,

evaluation

political, economic, and environmental impacts. For of environmental modeling [20], market analysis,

research,

attitudinal

and

political

polls,

etc.,

have

been

at best in some TA projects. There is a need for a systematic examination of these methods to determine where and when they should be used, and how they can be effectively integrated with the interdisciplinary methods for TA. In the area of value-oriented policy generation, there is a need for much original research. As mentioned earlier in this paper, the policy generation procedure presented here represents only an initial attempt at this kind of research. Much more work in this employed

only

in ad hoc manner

‘Decision analysis proved to be a powerful approach to analyze the decision to seed hurricanes. and for assessing the policy consequences (the costs of government responsibility as well as property damage) which had already been identified. However, it failed to warn the decision maker and the public about the possible shrinking of the temperate zone as a result of hurricane seeding. [ 181. 3For a taxonotiy of these impact identification methodologies, see [ 24 ] .

[ 171 It was particularly useful for organizing the uncertainties

MARK BERG, KAN CHEN AND GEORGE ZISSIS

410

IN PERSPECTIVE

METHODOLOGIES

43

Fan Out

Fan Out

fi

/‘\

SYNTHESIS

Qualitative \

EVALUATION

/ Fan In

Quantltatn’e

\/

V

POLICY DECISIONS

Fig. 2. Methodological coupling among technology, values, and decisions in technology

assessment.

area is needed. Moreover, those who take a holistic approach to social problem analysis and social forecasting have criticized the often simplistic cause-effect relationships used in many technology assessment projects as grossly inadequate. Such criticisms have come from both historians [21] and futuristis [22] _However, we lack adequate methodologies for translating holistic perspectives to practical policy actions, especially when different holistic perspectives are in conflict. We lack adequate methodologies to make the knowledge in the realm of humanities functional, as we confront integrated problems with disintegrated skills [23], There is a need for methodological research that will draw from the wisdom and knowledge of historians, philosophers, anthropologists, and other holistic thinkers. Finally, in the area of policy synthesis and evaluation, there is a need for coupling the art of judgment [4] with the political process and the conventional techniques of systems science. We do not know how one can develop and improve on the art of judgment beyond the acquisition of a broad education and relevant experience. However, there can be better methodologies for coupling TA with the present and future political processes. These linkages are indispensable. For unless the TA perspective has some influence on the critical political question of “who gets what, when and how”, TA will quickly become a futile endeavor, of little impact on the real world [5]. political and economic decision-making systems, then future methodological development in this area will have to learn from, and integrate with, the knowledge about the social-psychological pressures on decision-makers [25] , the nature of political decisionmaking, citizen participation strategies, etc. TA will have to learn to avoid the dangers of elitism and technocratic analysis without falling into the equally dangerous traps offered

METHODOLOGY

by “the

FOR TECHNOLOGY

conventional

wisdom,”

411

ASSESSMENT

disjointed

incrementalism,

and interest

group politics

]261. It is generally agreed that TA is still at an embryonic stage of development. Its objectives are being refined; its relevant methodologies and techniques are evolving; and its potential impact on improved policy-making is still uncertain [27]. In other words, TA is a new field without a paradigm. The direction of future research in this field, in our view, points toward value-oriented procedures, combinations of humanistic and scientific approaches, and participative policy making processes-all of these being characteristics of a new policy science paradigm [l 1, 281. If the future research in these directions turns out to be successful, these characteristics will define a paradigm for TA, which in turn will help define a new paradigm for policy science in general. See pages 412-419

for Appendices,

and page 420 for references.

SNOlllaN03

JO SlSAlVNV

‘E

IKMS~~

_

:V XIaNSddV

4.

PROJECTION

OF DEVELOPMENTS

-

’ .’

I

Same resource and econon~~c enplo~tat~on, and potential for explortation, ~‘111 cont#nue although the questlon ~111 be addressed and steps taken Lo alleviate the problem. Contmued and lncrcasmg unevenness 1” the ab,,lty and canac~tv of obtalnLnz us,“_ -data can be exoected. 1” sn~te of ‘,“pen. po11cy. ” ” Some exnloitatlon and ‘shady deals’ based on R.S. data can be expecied. Policy and mstttuttonal adjustments can be expected to reduce the mamnltude of ““t~nt~al oroblcm. e.r.. tram~nr. cooperatwe dr~elopment; leg,sl&n, etc. Further advances ,n the hardware and understandlna needed fur exploitation can be expected. Other c”u”tr~es can be expected to pay a” ,ncreasmg share of the c”zLs for R S. data.

T 5 IDENllFlCATlON

3

OF CONDITIONS

OF POLICY

OPTIONS

,

a;phcatm

_

Early imlttary history ¤t R&D nature of R.S trchnology has permlfted tile general avoidance of the explmlat~on issue. At present. many operat,unal orwnted &es are conducted under the “lore relaxed standards of ‘pure’ scle”cE. A sl&md~cant portlo” of potential data users would see the use of legally obtamed, but onrevealed mformat~on, as sound and fax business practxe rather than exp,“,tati”n. St111 others would justdy ‘explollatlon‘ on the grounds of ‘survival of the fittest,’ etc.

ANALYSIS

fn~tute a study and evaluat~oo of alternative lnternafional pnlicles and mstltutlonal strut tures n,hxh would avoid exploitlve relationships and encourage cooperallve development. The U N. has begun such an evaluation, which should be supported. Along with studying substantive issues such as bilateral & nultllateral agreements, tax structures and mcentives, information poluz~es, the role of nult~-natlonal corporations, etc this study d fully supported and acted upon by the U.S. would increase both trust and awareness with respect to the issue. Continue to actively increase d”niesUc & lnLeruati”nal awareness of, and sensi(~v~ty to, the potcntml uses and IIIISUSES of R.S. technolopv. ?“ntmue and upgrade programs I” U.S. technology trammg and transfer. Avoid pncmg and or partlclpatlon fees which would be prohlbltive to smaller groups or countries. Prnwde data formats, etc. which do not necessarily require elaborate technlcal equipment or expertise to be useful. Pass legislation (with appropriate sanctions) to require mandalory disclosure, to all partIes at Interest, of information “btalned from ERS sponsored programs. Regulale and or license R.S equ~“ment. data centers, and ,,b”t”,nteroreters. ,ay,ne out specifx guidelines for informai~on disclosure. uhere and when Iposslble, the dWrtbuti”nal effects flowing from control and developnl~nt of resources discovered by remote senstng. Input these impacts to the pohcy making process for constderat~on. Aclwly encourage and fxllltate earlv par,,clpal~“n, of a,, leg!t,mate data users, I,, the R.S svslem plannmg process (e g., operational satelllle systems) toensure the consIderatmn of the needs and capabllltws of a,, users I” Ihe deve,opn,ent “1 system requ!remen,s and s,,er,fxa~~ons.

hlon~lur,

Appendix A (continued)

REALIZATION

STATUS

Enwwnmental and, or geophysical war fare based on R.S. information.

Some of the basic knowledge and technology for such uses is currently available, although R.S. components are more advanced than other “ecessary elements. Such uses have been considered and gwe” minimal exposure in the press. U.S. Senate has recently passed a resolut,“” to seek a” international agreement to outlaw such warfare. Allegations have been made that floodmg via cloud seeding was used in Viet Nam.

GOAL

B: VALUE-SURVIVAL

ainimize potential for the use $ R.S. information in warfare hrough artificially caused natural calamities such as looding, earthquakes, droughts, ?tC.

2.

APPENDK

B.

/ GOAL

1 Military currently uses satellites and aircraft in pursuit of this goal. The ‘Big Bird’ satellite is currently used for surveilling the adherence of the involved parties to the SALT agreements. Military currently supports R&D in bath reconnaissance and ceea” surveillance. Current advanced technical capability provides for ground resolution from satellites on the order of l-2 feet or better, resulting in detailed intelligence information (Greenwood, 1973). Other advanced technical capabilities have enhanced the lifetime and data transmitting capabilities of reconnaissance satellites.

OPPORTUNITY

CLARlFlCAllON

rse R.S. surveillance to provide ” early warning capabillty and D minimize the threat of surcise arms build-up, or other hreats, which might lead to rar.

THREAT

GOAL

A. Warfare

1. 3.

OF CONDITIONS

Knowledge needed to avoid these natural disasters overlaps with that needed to cause them. Avoidance of natural disasters is a goal currently being pursued (with open acknowledgement). Effectiveness and controlability of such weapons is not fully understood. The availability of new R.S. data has provided a stimulus and tool for research into natural disasters. The ma]ority of research in this area is under the control of NOAA and the Dept. of Inter,“= and is funded at a low level. Some countries because of their geographic locatmn are more susceptible to such warfare than others, No mternational treaty or convention concerning these weapons exists. However, they are generally viewed as ‘unfair’ and ‘inhumane’ weapons. The technical and financial ability t” carry “ut such research is unevenly distributed throughout the world. The technical and financial requirements to “se such weapons have a high variability, e.g., cloud seeding Vs causatmn of tidal waves.

An increasing number of nations now have, or can develop, both nuclear and surveillance capabilities. There is the potential danger of complacency due to ‘oversell’ of remote sensing effectiveness.

There are a multiplicity of data users with responsiblhty for surveillance. Early R.S. development evolved out of the 100 yr history of military surveillance. There is some controversy as to desirability of offsite weapons inspection. In general, more information serves as a stabilizing factor bv reducine uncertaintv. The usi of satellites and aircraft for military surveillance and treaty enforcement may reduce the potential for developing trust between countries, i.e., surveillance being symbolic of a lack of trust, Complete intelligence data will always require m”re than just remote sensing.

ANALYSIS

THREAT

Enwronmental Degradation

E.

Disasters

Natural

D.

OPPORTUNITY/

CLARIFICATION

GOAL

for monitorand predict-

Use remote sensing for monitoring and predlctlng pollution and assoctated environmental degradation.

Use remote sensing ing, understanding, ing natural disasters.

Use remote sensing technology to help provide the timely and accurate information, about the quantity, quality and location of crops & range lands needed for increased productivity and improved distribution.

I Khk)

I

GOAL

& Starvation

Hunger

C.

(Cont.)

1. GOAL REALIZATION

STATUS

B (continued)

Current R&D is oriented toward this goal; both application and demonstration project oriented. R.S. of crops & range lands IS currently being used in some areas at ‘primitive’ levels of technology--‘Total information system,’ at present, laes behind hardware caoabilitv. Dipt. of Agriculture m&es quasi-operatmnal use of B&W photography from low altitude aircraft in crop &rve;s. NASA’s program is aimed at global crop survey capabilities, however there are no operational systems and mstitutional adjustments lag behind. West African drouth areas are being surveyed by Skylab to collect data on possible water and food sources. Data are being given top priority for processing. U.S. Department of Agriculture, an important agent in producing and distributing world food supples, has taken more of a ‘show me’ attitude rather than a more active stance with respect to R.S. In U.S. and other countries no timely and exhaustive system of inventorying crops and rangelands now exists.

(Cont.)

2.

Appendix 3.

ANALYSIS

OF CONDITIONS

Social/Political Factors R&D for R.S. applications in agriculture is primarily supported by NASA aided to some extent by the Department of Agriculture in the area of information systems. Systems for emergency food delivery tend to be relatively small scale and often ad hoc. Provision of adequate food is viewed more as the responsibility of individual nations than as a cooperative international responsibility. Potential for inadeauate food suoolv has not decreased in recent &rs despite”g&?n revolution.’ Technical and financial ability to use R.S. for monitormg and mventory is not distributed evenly (either wlthm or between countries). In the private sector, large agricultural Interests wll be more able to fully obtain and utilize R.S. data. An elaborate operational R.S. system may prowde the information needed for increasing centralized management of crops and rangeland. This may be considered a threat to individual freedom, etc. which would be ddficult to stop once started. Many nations consume food in far excess of nutritional needs. - Food supply is endangered by increasmg enwronmental pollutmn.

Technical Factors - Processine. interoretation. and data handline capability require further hevelopment and &IIonstration before long range commitment can be expected. While sensor capabilities could be improved, they are currently adequate. Economic Factors Significant economic incentive for reaching the goal exists in terms of efficiency, disease control, and speculation. Uneven distribution (domestically and mternationally) of R.S. agriculture data could lead to ‘unfair’ and ‘underarable speculation in commodities markets, etc. - The cost to individual users may be prohibitive wthout governmental assistance and/or subsidy.

Cont.)

C.

A.

4.

-

-

-

-

-

PROJECTION

OF DEVELOPMENTS

R&D pace cm be expected to be fund- rather than conceptlimited. The time horizon for access to timely, accurate and complete world crop information is uncertain but not likely withm 10 years. Some successes I” specific countries or regions can be expected, possibly resulting from NASA sponsored demonstration projects. Airborne R S for crop evaluation can be expected to I”creqse 1” use. Pressures for mcreased agricultural output can be expected to continue. Emergency food dehvery systems can be expected to contmue to be less than fully adequate. Severe local food shortages throughout the world can be expected to continue throughout the next decade. Some xncrease m the efflclent use of food can be expected, but the potentla, for rapld changes I” the ‘excessive’ demand for protem on the part of affluent nat,ons IS uncertam and doubtful Pressure on the USDA to upgrade their mvolvement I” the ut,,,rat,on of R.S for agricultural apphcatwns, can be expected to contmue with sonle successful results.

Basic research into natural disasters can be expected to continue at a” mcreasing pace. National social attitudes can be expected to continue to disapprove of such warfare.

MlfOXe).

The knowledge and technology needed for the large scale use of natural disasters as weapons, is not expected to be available in the “ear term. Legislation and treaties to bar the use and development of such weapons can be expected to be passed (but dlfflcult to

Continued use of R.S. for military surveillance can be expected. Pragmatw and conservative pressures can be expected to override any desire to create atmosphere of mutual trust through “on-surveillance. Total information systems can be expected to continue to emolov much more than just R.S. systems. Co&&ed, but not neces&ily largk, improvement I” technical capabiltties can be expected. l71e fut& organizational sYructure for R.S. surveillance activities IS uncertain, however, it can be expected that the military will continue to play a dominant role among the numerous involved agencies. Consolidation of surveillance actwties LS unlikely.

+

B

A

5.

-

B (continued) IDENTlFlCATlON

OF POLICY

OPTIONS

and weaponry

on worldwide

scale

so that

motivation

for

surveillance

will

,

_

Increase level of R&D L” NASA for crop and range land applications. Implement similar user oriented R&D within Department of Agriculture, e.g., demonstration projects, etc. Create a U.N. structure to support an agriculture information system based, perhaps, m FAO. Create a World Food Bank, a new mstltutlon outside of the U N , with managerial capabilitles centering on an information system usmg global R.S. technology. Bee~n an immediate study of the imuact of timely aericultural data on commodltles swculation, tradmg, hoarding and other shbrtage related phenomena. Study and develop improved technwxs and mstltut~onal arrangements for emergency food relief systems. lntenslfy basic research mto nutrltlon needs and means of fulfillment. increase public sensltlvity and awareness with respect to these results and their implications. Increase R&D m various aspects of food supply systems, e g dehvery, storage, etc., and develop agricultural and food processing policies accordingly. Actively implement these policies through non-partlsan funding at the approprmte levels, e.g., local, state, national and international.

Promote international treaty outlawma such weapons. Ban R&D work directly related to using ‘naturalbisasters’ as weapons. Develop clear and visible pol~y statements for U.S. on this issue. Increase awareness (both domestically and internationally) with respect to the potential threat of this use of R.S. and Its related knowledge. Monitor enwronmental & geophysical knowledge in order to be aware of and control potential dangerous uses. Ownlv distribute environmental data and other relevant information which identifv the actual “be oi such warfare techniques, i.e., meteorologxal R.S. data, in situ sensors etc., may provide some of the needed information. Abandon warfare as means of settlinc disoutes.

Abandon warfare be abrogated.

Increase trust and pressure for treaty compliance, etc by adopting a” open pol~y of making the technolow and data avatlable to other countries or mternatlonal organizations, e.g., U.N., Peace institutes, etc. Mamtai” closed or sem-closed pohcy which would limit access to critical data. Develop techniques to protect our own surveillance capabllity from sabotage 01’ interference. Smce R.S. data requtres interpretation, develop redundant systems to provide confirmation and Increased certamty of ,“terpretat,on. Where possible, ~“crease efficiency and control of data by centralizing those functions where redundancy is not useful. Continue R&D efforts to imorove mteroretation caoabilltles and to mmlmize the need for ‘ground truth.’

Appendix

act”:1

“Xy

pr,-

of bndivid.

‘rlahts’

andnat1ona1

the

ual

of

for

R.S.

and d!ssemlnatton.

poiirles

and eiilrlency.

along wth

to ,nap-

and lnadver

leadlng

dorngrad,nR

tent

proprlate

data

and

data

;sciuln&

arquisition

I” deternunmg

data

of “rosary.

the con-

polirles

to ensure

of r,ehts

of R.S.

slderation

I‘“~SCI”U~

mechanisms

Develop

OPPORltJNlTY/GOAL

CLARIFICATION

and eii~r~ency

THREAT

GOAL

ante “1 uselul”ess

1.

1 on both

-

-

loss. U.S. was

usmg

a competitor’s

altltuda.

lb71

data

ar-

between

on a

pho-

case

trade

secrets.

an mproper

aer~at

c&t

orodurtaan

of home

data

and relat,valy

ilelds.

surveys

se”tIc

e.g.,

espionage

whatever

industrial

H”wever;a

of dlscovermg

that

etc.

and heat

plane

can collect

a small

durlna

,n sigh,

mccha-

agreement

Awarc-

oi pr,

less are inter-

and

other

than

societal

““rrns

and ~nterna-

rule on pr~vac” ISSUBS) to this I&W !.s.limited

domestxally

Greatest awareness 1s expressed with respect to mamtammg privacy against those trying to “b,aln data, rather than protertlng the pr,vary of those who have gathered data whtch they wish to protect.

both

may on “rraslon and sensltrvlty

but ~ncreasmg, tl”“ally

the c”“rts Awareness

respect given betwi=en (although

wth

over the dlssemlnatlon of countries irom an opera-

to privacy, there 1s no social mechamsm the speclflc respo”slbllltY Of medtatlne pr~vary and the-uses of ¬e se”,,“,,

At present.

have expressed ~“ncern lnformatlon about their LIonal system.

natIona conventions requiring perm~sslon to fly lnt” national air soace. However. there IS disaereement as to terrlto&xl boundarles. Several rountrics

even There

the const,tut,“n.

lnternatlanal privacy ‘rights’ are specific and under some dispute.

can be found 1” lea~slat~an. eomman or statutory law.

the US. has agreed to provide a master copy of the data from experlmental satetllte programs. Legal recognition of a ‘right’ to privacy LS rather recent and vaeue in the U.S. Some leeal basla for a ‘rlght’ of p&acy (although not an &“I& r,ght)

Current UnIted States ERS pohcy allows for taking images wherever deslred by the U.S. and dlstrlbut1”~: all data under a” ‘“pen to all’ policy. Should an international data dhstrtbution center be created.

means

tography,

ruled

processes,

wth

ad hoc basis,

from

scale

maintenance

small

R.S.

acreptable

duztangolsh

information

cannot

equpment

anyone

mexpenslve

‘private’

equrpment and

or

IS not ,mmed~ately

of the problem

!ts allev~at!“n

level.

~nvab,“n

however,

lor

SIAIUS

and national

potential

IS ~ncreaslng,

At present,

quwtlo”.

‘public’

Tcchnlcal

for

a real

REALIZATION

the personal threat

1s to the scope xsms

GOAL IS at present.

ICSS of this

,acy

rhere

2.

APPENDIX C: PRNACY T

3.

to the

of

l-2

have

R.S.

both

quality

that

to

1s

by the

e.g.,resolutlon

consideratmns,

diiilcult

resoluhon

satellite

approved

pr,vacy,

securzty R.S.,are

affectlhg

of natlonal

factors of spareborne

the

better

expected

once

and politically

than

Yet

effects. speclflcations s”ch

large

Imagery

,nvestment

or

IS

be very high. such as crop mineral reserves. etc, will issue for many concerned

(Continued

on next

page)

The cost of protectleg privacy from ground based “hservatton e.g.,fenres & buffer zones, ,s much less than the cost of protertmg aga,nst airborne “bservatlon. e.g.,r”ois, tunnels, etc.

of stock, a sertous

of privacy, could of apphcatlons

,nvas~“n ,mp”rtancr

occasional Economic

yield. quantity make privacy partws

of momtorlng & rontrohng the ,nass,ve of data of an oprratlonal system to avold

The costs quanttties

aimed at delineat11, which this IS

to be, and often IS, cheaper and rich than other data sources. Current R&D activity IS partially ‘ng and lnrreaslng the apphcat~ons true.

R.S. 1s expected more lnformatlon

this

of addltlonal small.

Once he qute

cost

scale. the marginal should

relatively made,

processmg

would to prl-

IS

for

lnformatmn provided by R.S. for operational “se on a

the same order of magmtude, yet the aircraft probably be perceived as the greater threat vacy due to its higher visibility

The technology IS such that a low flymg R.S. aircraft and orbitlog R.S. satellite could have a resolutionof

mlhtary.

capacity

techmcal

On the baas

been

produced

have

and its use.

it IS socially the data

available, control

and the better

launched

expected.

than

imagery

have

hardware

and Skylab

“ew

ERTS

for

1973)

have

the potentml satellites

(Greenwood,

Mlhtary it,

it 1s virtually

IS occurring.

of R.S. R.S.

aircraft

that

nature

Inches.

capability

of 2-3

low flymg

to detect

‘pawre

OF CONDITIONS

and are lImIted only by atmospheric The complex and often conservative

resotutlon

resolution

At present.

lmposslble

Due

ANALYSIS Factors

Economic Factors - The unique & synoptic has created pressure

-

-

-

-

-

Technical

5111VlS

NOllVZllV3~

lVO9

(panu~wo~) 3 xfpuaddy

‘Z

NOllV314lllVl3

lVO9

lV38HI

‘I

PROJECTION

to

OF DEVELOPMENTS

usefulness of much R.S. data along with high cost and difficulty of protecting some aspects of privacy may be downa lowering of standards and reduction in application

The technoloeical potential for invasion of urivacv can be expected to increase. It can be expected that privacy considerations will play some role in determining future data acquisition and dissemination policies at both the national and international levels. Whatever data acquisition and dissemination policies are imolemented e.g.. onen dissemination: need to know: etc.,it &n be expecleh that criticism and lack of trust will be forthcoming due to the wide variety of interests and standards on the part of those involved. ‘Die marginal cost of data acquisition and processing can be expected to decrease as R.S. systems shtft from R&D to operational in nature. Continued success in the use of KS. for environmental and resource applications can be expected to create pressure for increased use of R.S. on lwal levels where ihreats to privacy will be more readily perceived and resisted, e.g., census, building condition, heat loss, etc. It can be expected that the military will continue to set standards as to acceptable spaceborne resolution for non-military applications, and that the gap between expected image quality and achieved image quality will decrease with additional experience.

I

The perceived the potentially Privacy, suggest graded through its domains of

Invasion of orivacv can be exoected continue as an issue of some importance to some groups. In general, however, privacy issues will be outweighed by the perceived need for and utility of R.S. information.

4. 5.

IDENTlFlCATlON

OF POLICY

OPTIONS

In an effort to build trust & awareness with respect to R.S. and privacy, Congress could pass highly visible legislation creating a ‘watch dog’ agency (or giving a ‘watch dog’ responsibility) which would monitor, license, and regulate R.S. equipment, users, and applications. In an effort to prevent unauthorized or inappropriate use of R.S. information the U.S. could alter its oresent ‘oven* oolicv to a oolicv of more limited availability in which authorization by the party-at-interest’wouid be required before the data could be placed man open file. In an effort to build awareness and sensitivity to potential invasion of privacy from the sky, Congress or the appropriate agency could put forward clear and visible policy statements or legislation with regard to R.S. use. This might start with preliminary lists of either acceptable or unacceptable uses which could serve as a focus for national debate via public hearings on the issues. Along similar lines Congress could create an independent commission which would monitor and make invasion of privacy judgements with respect to new technical innovations and application areas. Such a commission if given clearance to classified material could be better prepared for the impacts of more advanced R.S. systems and, if so charged, could evaluate imagery obtained by the military which might have useful civilian application and would not compromise national security, e.g., there may be critical periods with respect to floods, earthquakes, etc. in which classified military R.S. could provide imagery when civilian satellites or aircraft were not in the proper position. Such a commission could also provide the coordination necessary to insure that any classified imagery released to the civilian sector would adhere to the same data dissemination policy as nonclassified Imagery, e.g., ERTS-A. Continue U.S. interaction with the U.N. Working Group on Remote Sensing of the Earth by Satellite, to develop data acquisition and dissemination policies which satisfactorily respect national privacy without overly jeopardizing the usefulness and efficiency of an operational satellite system. Continue to support the proposed U.N. task force to study this issue. Investigate through R&D, alternative methc&for acquiring or disseminating data which would ‘blank out’ those areas whose privacy is to be protected and still permit desirable coverage of adiacent areas. Require that demonstration and pilot projects for the application of R.S. techniques to local levels, be evaluated in terms of their impact on individual and group privacy. Basic research lo provide increased understanding of the need for privacy could be encouraged and funded as a parallel related program.

Appendix C (continued)

420

REFERENCES 1.Sherry R. Amstein,

MARK BERG, KAN CHEN AND GEORGE

ZISSIS

A. N. Christakis, et al., Perspectives on Technology Assessment, Chapter 3 by Berg, Chen and Zissis, published by The Science & Technology Publishers, Tel Aviv, 1975. 2. Joseph E. Coates, Technology assessment: The benefits, the costs, the consequences, The Futurist 5(6) (December 1971). 2a.A Partial Technology Assessment of Remote Sensing, Draft, ERIM Report No. 195200-2-F, January 1974. 3. Francois Hetman, Society and the Assessment of Technology, Organization for Economic Cooperation and Development (Paris), 1973. 4.Geoffrey Vickers, The Art of Judgment, Methuen, New York, 1965. 5. Mark R. Berg, The Politics of Technology Assessment, forthcoming in The Journal of the International Society for Technology Assessment, 1, (4) 1976. 6. Guy Benveniste, The Politics ofExpertise, Glendessary Press, San Francisco, 1972. 7. Harold Lasswell, A Preview of Policy Sciences, American Elsevier, New York, 1971. 8See [l] , Methodologies in Perspective, by Mark Berg. 9.Geoffrey Vickers, The Multi-Values Choice, in Value Systems and SocialProcess, Pelican, Gretna, La., 1970. lO.Milton Rokeach, Beliefs, Attitudes and Values, Joseey-Bass, Inc., San Francisco, 1968. Il. Kan Chen, Karl Lagler, Mark Berg, et al., Growth Policy: Population, Environment, and Beyond, University of Michigan Press, Ann Arbor, 1974. 12. Norman C. Dalkey, Delphi, Rand Paper P 3704, October 1967. 13. Roland N. McKean, Efficiency in Government Through Systems Analysis, Wiley, New York, 1958. 14. E. J. Mishan, Cost-Benefit Analysis, Praeger, New York, 1958. 15. Howard Raiffa, Decision Analysis: Introductory Lectures on Choices Under Uncertainty, AddisonWesley, Reading, Mass., 1968 16. Ralph F. Miles, Jr. (Ed.), Systems Concepts: Lectures on Cotemporary Approaches to Systems, Wiley-Interscience, New York, 1973. 17. Ronald A. Howard, James E. Matheson, and D. Warner North, “The Decision to Seed Hurricanes,” Science 176 (June 16, 1972). 18.The Benefits of Hurricanes, Time, 115 (September 24, 1973). 19. Wayne Boucher (Ed.), The Study of the Future: An Agenda for Research (draft), The Futures Group, Glastonbury, Conn., 1974. 20. Douglas Daetz and Richard H. Pantell (Eds.), EnvironmentalModeling: Analysis and Management, Dowden, Hutchinson and Ross, Stroudsburg, Pa., 1974. 21. Lynn White, Jr., Technology Assessment from the Stance of a Medieval Historian, The American Historical Review, 79 (1) (1974). 22, Willis W. Harman, On Normative Forecasting, in The Study of the Future: An Agenda for Research (draft), The Futures Group, Glastonbury, Conn., 1974. 23. William Arrowsmith, Teaching and the Liberal Arts: Notes Toward an Old Frontier, in The Liberal Arts and Teacher Education (Donald N. Bigelow, Ed.), University of Nebraska Press, Lincoln, Nebraska, 1971. 24. K. Chen and G. J. Zissis, Philosophical and Methodological Approaches to Technology Assessment, ZSTA J. 1 (1) 17-28 (1975). 25. Donald N. Michael, On Learning to Plan-and Planning to Learn, Jossey-Bass Publishers, San Francisco, 1973. 26. Theodore J. Lowi, The End of Liberalism, W. W. Norton & Co., New York, 1969. 27. See [ 11, Arnstein, et al. 28. Alexander N. Christakis, A New Policy Science Paradigm, Futures (December 1973). Received 23 May I9 75; revised 29 May 1975