Interpretive structural modeling—A useful tool for technology assessment?

TECHNOLOGICAL

FORECASTING

AND SOCIAL CHANGE 11,165-185

Interpretive Structural Modeling-A Tool for Technology Assessment?

(1978)

Useful

RICHARD H. WATSON

ABSTRACT Interpretive Structural Modeling (ISM) is a computer based technique for helping small groups develop graphical representations of complex systems. In this paper, the rationale for the use of Interpretive Structural Modeling in activities such as technology assessment is developed and the basic concepts underlying the technique are explored. Several applications of ISM are described and from these experiences some observations are made about the nature and effectiveness of the ISM process and product. ISM was found to provide its users with a systematic and comprehensive method for integrating group judgments in the development of “first-cut” structural models. At the same time, however, the technique was found to be relatively inflexible and may, in some instances, inhibit group processes. Possible directions for the development of less rigid methods are suggested.

The Need for Tools for Structuring Complexity The major issues of the era increasingly seem to occur at the interface between technology and society. In so doing, they invariably cut across the boundaries of multiple social, economic, political, institutional, environmental, and technological systems. These complex issues of technology and society severely tax our planning and problemsolving capabilities. The traditional reductionist models and paradigms of “normal” science are now often perceived as being madequate to deal with such issues. As R. F. Rhyne has noted: The arenas within which partial solutions are illusory have grown remarkably in subject scope, special intent, and time. Subjects recognized as ‘hard’ and ‘soft’ now interweave, neither to be seen as dominant above the other. In almost every step to meet the macro-problem, the decision-maker (whether some executive or the voter who must judge how well he likes the action taken in his name) is forced to act as though he comprehended fields that are so wide he cannot even fairly sample them [I].

Technology assessment (TA) is an emerging mode of policy analysis which is intended to help deal with these complex issues of technology and society. In the words of one practitioner: It may be defined as a class of policy studies examining the fullest range of impacts of the introduction new technology or the expansion of a present technology in new or different ways [2].

of a

TA is described as being comprehensive and holistic. It is asserted that holistic thinking is something we do not know how to do routinely, is not a scientific, engineering, or disciplinary enterprise, but is essentially an art form. However, art form or not, a variety of analytical methods have and are being applied to technology assessment [3-71. RICHARD WATSON is a Research Assistant Professor nology at the University of Washington, Seattle. @Elsevier

North-Holland,

Inc., 1978

with the Program in Social Management

of Tech-

16.5

166

RICHARD

H. WATSON

Structural Modeling and Technology Assessment One class of methods which seems particularly appropriate to TA is structural modeling. Structural models are those for which the modeling process is focused on the tasks of selecting the components of the model and making explicit the interactions between them [8]. This is in contrast to dynamic models where the focus is on the use of various mathematical techniques and simulation languages to forecast system behavior. This distinction is parallel to Kane’s identification of geometric and arithmetic approaches to forecasting [9]. While the former methods are more concerned with system structure and the form of relationships, the latter are coefficient oriented i.e., concerned about precise numerical specification. While it is clear that structural models are necessary precursors to dynamic models, the availability of sophisticated techniques and the orientations of modelers themselves have, to a large extent, resulted in an emphasis on the dynamic or arithmetic phase of modeling. The appropriateness of structural modeling for TA derives from the nature of problems addressed by TA. James Taylor identifies two types of problems: well-defined and ill-defined [lo]. Well-defined problems are those for which there is essentially, an agreement about ends and the general means for achieving them. The problem is typically that of choosing the “best” means among available alternatives. In contrast, ill-defined problems are those for which there is rarely agreement either about ends or means, and which, more often than not, cut across disciplinary boundaries. Technology assessment is commonly agreed to fall in this latter category. Taylor goes on to suggest a strategy for the solution of such ill-defined problems. This strategy involves first a scanning phase in which the potentially relevant components of the problem system are identified and adequately defined. The second phase is a patterning phase in which the relevant patterns of relationships between components are explored and defined. Only after these tasks have been completed can the process of invention and evaluation of alternative solutions be carried out effectively. This process of scanning and patterning is easily recognized as the process of structural modeling. To accomplish these tasks, Taylor postulates the existence of an ideal polymath. This ideal polymath is an individual who has all the skills and knowledge needed to adequately structure the problem system and design and evaluate solutions for it. Rarely, if ever, however, are the complex issues which are the subjects of technology assessment accessible to the skills and perspectives of one individual, one discipline or one segment of society. The only alternative appears to be to attack complex issues in the context of group or team approach. Typically, such groups are designed to encompass the range of disciplines relevant to the issue at hand. Increasingly, however, it is found that such disciplinary expertise alone is insufficient to deal with issues which can impinge heavily and unevenly on many segments of society. Representatives of the various affected groups are therefore seeking a more direct role in the resolution of these issues. Moving the solution of complex problems from an individual context to a group context introduces a whole new set of problems of group functioning. To adequately perform the scanning and patterning functions suggested by Taylor, severe requirements are placed on group communication capabilities. Typically, members of such task groups do not, at least initially, even speak the same language. One discipline’s jargon may be incongruent with another’s and, more generally, the language of the scientist may be incomprehensible to the layman. A more subtle and more serious problem arises from the fact that different disciplines rely on different paradigms and are therefore likely to have different world views. And again, the world views of science in general may not provide a

INTERPRETIVE

STRUCTURAL

MODELING

FOR TA

167

meaningful interpretation of reality for the citizen who is concerned that his or his children’s “ox will be gored”. One veteran of interdisciplinary research has indicated that, as a rule of thumb, up to one year per discipline is required to establish “real” communications on multidisciplinary terms [l I]. Such a time span is clearly inappropriate for problems of TA which invariably have short fuses. Furthermore, since the composition of such groups generally must change to adapt to the requirements of different problems, the practicality (to say nothing of the desirability) of longterm standing multidisciplinary groups is questionable. These characteristics of the problems addressed by technology assessment and the multidisciplinary group context in which TA is practiced dictate several criteria for structural modeling methods if they are to be useful in TA. They should be understandable to a variety of potential users, including disciplinary experts, decision-makers and citizens. They should provide a means of integrating the diverse theoretical, empirical and subjective perceptions of these groups. They should be capable of handling the large number of components and relationship typical of complex systems. They should be relatively easy to use. They should be heuristic both in terms of assessing the adequacy of model formulation and leading to insights about system behavior. And, they should be communicable, both within and without the TA team. There are a large number of structural modeling techniques. A recent survey of structural modeling tools by Lendaris and Wakeland identified over 100 methods [ 121. Other overviews of structural modeling methods are provided by McLean and Kawamura and Christakis [ 13, 141. This paper will focus on just one of these methods, Interpretive Structural Modeling (ISM). Interpretive Structural Modeling Interpretive Structural Modeling is a computer-aided method for developing graphical representations of system composition and structure. ISM had its inception in John Warfield’s perception of the need, when attempting to couple science to policy, for “a set of communication tools which have both a scientific and lay character serving as a linkage mechanism between science and the public, and having meaning for all who are involved” and which, in particular, are capable of communicating a holistic sense of the elements and their relations which define system structure [ 151. Warfield stipulates a set of requirements for these communication tools which include [16]: 1. provision for the inclusion of the scientific elements; 2. means for exhibiting a complex set of relations; 3. means for showing that complex set of relations which permit continuous observation, questioning and modification of the relations; 4. congruence with the originators’ perceptions and analytical processes; 5. ease of learning by public (or, by inference, multidisciplinary) audience. Graphical models or, more specifically, directed graphs (digraphs) appear to satisfy these requirements. In such a representation, the elements or components of a system are represented by the “points” of the graph and the existence of a particular relationship between elements is indicated by the presence of a directed line segment. It is this concept of relatedness in the context of a particular relationship which distinguishes a system from a mere aggregation of components. For example, Abraham Maslow’s list of human needs becomes a system in the context of the relation, “is subordinate to”:

168

RICHARD

Maslow’s Hierarchy Survival

H. WATSON

of Needs

Relation

t Security

T

= is subordmate

to

t Belongingness t Esteem Self-actualization A different type of system is illustrated in Fig. 1. Domestic production and demand for petroleum are related to OPEC policy through the relationship, “affects’ ’ Although the potential for comprehensiveness and comprehensibility in the graphical model of a system is apparent, there remains the problem of the identification of system composition and structure. The problem is one of capturing, integrating, and translating a number of individual, incomplete and possibly conflicting mental models. Presumably, system composition can be determined from the nominations of a sufficiently broad and knowledgeable universe of participants. The problem is by no means trivial but does appear to be tractable. Defining structure, however, requires a determination of the existence or nonexistence of a given relationship between all the element pairs of the system. Taken to the extreme, to define structure for a system of N elements, it would be necessary to examineN(N - 1)/2 element pairs. The existence or nonexistence of directed relationship would be recorded in an N X N interaction matrix. For the simple system of Fig. 1, the corresponding interaction matrix would be as follows (1 in a matrix cell denotes the existence of a directed relationship between the row element and the column element): I I OPEC policy

2

3

4

5

I

I

0

0

I

0

0

I

I

2 Oil supply

1

3 Oil price

I

0

0

I

I

t0

I

0

4 Domestic

demand

5 Domestic

production

0

0

In actual practice, it is doubtful that each possible element pair would be systematically examined. Common sense often allows the a priori elimination of some of the possibilities. However, for large systems, the number of possibilities are large and the risk of overlooking important relationships is always present. The amount of what Warfield terms “intellectual materials handling” involved in developing graphical representations of a complex system constitutes a serious impediment to their use. Recognizing both the potential utility of graphical models and the magnitude of the intellectual material handling problem involved in constructing such models, Warfield developed the mathematical under-pinnings (using matrix algebra and graph theory) for

INTERPRETIVE

STRUCTURAL

MODELING

FOR TA

Domestic

/Domestic

.

Policy ‘------+

= Affects

Fig. 1.

the computer-aided approach to the task called Interpretive Structural Modeling. The associated computer software were developed by the Battelle Columbus Laboratories and have been made available to potential users. The mathematical details of ISM are developed in the Battelle Monograph Shucturing Complex Systems [ 171. There are two concepts which underlie ISM and which are essential to understanding both the ISM process and product. One is the concept of reuchability and the other is the concept of transitive inference. Through the use of these concepts in conjunction with the bookkeeping capabilities of the computer, the ISM system offers a formal approach to structuring complex systems which is claimed to be more efficient and effective than less formal unassisted approaches. REACHABILITY The basic building block of the ISM algorithm is the reachability matrix. Consider the simple system of Fig. 2 and the corresponding direct interaction and reachability matrixes: Whereas the interaction matrix indicates whether or not there is a direct interaction between the row and column elements, the reachability matrix indicates whether or not a column element can be “reached” from a row element along a continuous directed path (e.g., D is reachable from B). While a unique reachability matrix is derivable for each interaction matrix, there are several possible interaction matrices for each reachability matrix. For example, the addition of a direct interaction between B and D would not alter the reachability matrix. As a result, the directed graph (digraph) which is constructed from the reachability matrix is the “minimum edge” representative of the system, i.e., that

Interaction A

D

A

Reachability

B

C

D

0

0

1

A

0

0

B

0

Cl

0

DO

0

A

B

0

C 0

D

0

I

1

t B

A

i”\ B

C

1

Cl

0

DO

0

0

Fig. 2.

1

1 0

RICHARD

I70

Mental model

Reachability

:’f B

Y C

Digraph Constructed from Reachability Matrix

A

B

C

D

B-A-D

A B 1 Cl

1

0 0

t C

I

1 1 1

Dl

IO

D t A A

Matrix

H. WATSON

Fig. 3.

digraph which satisfied the conditions of reachability with the minimum number of interelement links. It should also be noted that the digraph which is constructed from the reachability matrix is a hierarchial structure. As such, there is not the possibility for feedback from upper to lower levels in the structure. For example, consider the addition of a feedback from D to B in Fig. 2 as shown in Fig. 3. TRANSITIVE

INFERENCE

An example of a transitive relationship is one for which if A is related to B and B is related to C, it may be inferred that A is related to C. The property of transitivity allows some of the cells of the reachability matrix to be filled by inference. For instance, in the previous example six cells can be filled by transitive inference: (B,C), (B,D), (C,B), (C,D), (D&l and (D,C). Through the use of reachability and transitive inference the ISM algorithm reduces the number of element pairs which must be examined to define system structure by from 50-80 percent. Fortunately, many important system relationships are transitive. These can be roughly classed into two types: ordering relationships and affective relationships. Ordering relationships are relations which effect a strict hierarchical sequencing of the system elements such as more important than. greater than . more feasible than. The use of the ISM for ordering relations is totally unambiguous. Affective relationships are relations aggravates. . . contributes to. such as: affects. . . influences. . . causes. . . supports. These seem to be the more interesting kinds of relationships for the kinds of problems with which TA is concerned. However, as will be demonstrated, the use of KM with such relationships is not entirely straightforward. The ISM Process The steps in the ISM process are as follows: 1. Defining a set of elements (system components, objectives, problems, etc.) specific to a particular context which compose the system of interest. 2. Defining a contextual relation which describes the interelement relationship to be explored (e.g., causes, affects, supports, aggravates, is more important than and so on).

INTERPRETIVE

STRUCTURAL

MODELING

171

FOR TA

3. Deciding upon a decision rule by which the group will decide its response to the ISM queries (consensus, majority rule). 4. Conducting a computer-aided ISM exercise during which the system structure (pattern of relationships between elements) is systematically explored through a series of computer directed queries (via a CRT or printer terminal) of the form: 1s ELEMENT A RELATED TO ELEMENT B There are two phases of queries. In the first phase, responses. Symbolically, they are as follows:

there are four possible

V A is related to B A B is related to A X A is related to B and B is related to A (feedback) 0 A and B are not related Note that these responses are bidirectional and permit filling two cells in the reachability matrix with each response. Based on the response to this phase, the computer creates a matrix model of several unconnected hierarchical tree structures. The second stage of queries involves simple yes or no (unidirectional) responses and permits the computer to interconnect these tree structures. 5. Constructing the resulting directed graph from the computer generated instructions. 6. Modifying the initial ISM digraph as necessary to achieve a satisfactory representation of the object system. Step 1 is recognized as the scanning function of Taylor’s ill-defined problem strategy while steps 2, 4, and 6 correspond to the patterning functions. The advantages claimed for ISM include not only the numerical efficiency previously discussed but also comprehensiveness and improvements in group process. The advantages claimed are as follows [18]: 1. The process is systematic; the computer is programmed to consider all possible pairwise relations of system elements, either directly from the responses of the participants or by transitive inference. 2. The process is efficient; depending on the context, the use of transitive inference may reduce the number of the required relational queries by from 50-80 percent. 3. No knowledge of the underlying process is required of the participants; they simply must possess enough understanding of the object system to be able to respond to the series of relational queries generated by the computer. 4. It guides and records the results of group deliberations on complex issues in an efficient and systematic manner. 5. It produces a structured model or graphical representation of the original problem situation that can be communicated more effectively to others. 6. It enhances the quality of interdisciplinary and interpersonal communication

172

RICHARD

H. WATSON

within the context of the problem situation by focusing the attention of the participants on one specific question at a time. 7. It encourages issue analysis by allowing participants to explore the adequacy of a proposed list of systems elements or issue statements for illuminating a specified situation. 8. It serves as a learning tool by forcing participants to develop a deeper understanding of the meaning and significance of a specified element list and relation. 9. It permits action or policy analysis by assisting participants in identifying particular areas for policy action which offer advantages or leverage in pursuing specified objectives. Given this set of attributes, ISM would appear to be a valuable addition to technology assessment’s set of tools. There are, however, some significant questions regarding the utility of the ISM process and product. These include the following: 1. Does the ISM system produce an “accurate” or at least satisfactory representation of the object system or do limitations of the ISM algorithm seriously distort the system from that intended by the ISM users? 2. Are there limitations or ideosyncracies of the ISM algorithm which force the users to distort their treatment of the problem situation to fit the limitations of the tool? 3. Is the highly structured ISM process a satisfying vehicle for the exchange of information among members of the user group? 4. Are there other processes which are effective means of producing graphical representations of system structure? Several examples of the use of interpretive structural modeling have appeared in the literature [ 19, 20, 21, 22, 231. However, in the opinion of the author, nowhere have questions such as the above been dealt with adequately. What follows is a recitation of experiences with several applications of ISM and some observations about the ISM process and product drawn from those experiences. It should be noted that this work was done using a particular (and relatively early) version of the ISM software. Development of ISM has subsequently been advanced by several researchers, particularly Professor Raymond Fitz of the University of Dayton [24]. Some of the observations made here are peculiar to the version of ISM used. Where this is the case, it will be so noted. Other observations, however, are more generic in nature and applicable regardless of the software version used. APPLICATIONS

OF AND PERSPECTIVES

ON ISM

A variety of applications of ISM were undertaken, a few of which were experiments aimed specifically at investigating the characteristics of ISM. Most, however, were attempts to use ISM in the context of “real” analyses. In all instances, affective rather than ordering relationships were explored as these were felt to be both more useful in TA application and conceptually more difficult. Four of these applications will be described here. One ISM application involved a graduate seminar in technology assessment at the University of Washington. This group was composed of five students from a variety of

INTERPRETIVE

STRUCTURAL

MODELING

173

FOR TA

disciplines. They were in the process of developing a proposal for an assessment of a state nuclear safety initiative. The group was interested in developing a perception of the energy system which would be affected by passage of the nuclear safety initiative. The element set, consisting of 24 elements, was generated in a brainstorming session lasting approximately an hour. The relationship chosen to be explored was “affects or influences.” The ISM session and construction of the initial digraph required two hours. The products of this effort are shown in Figs. 4(a) and (b) where 4(a) is the initial digraph produced by ISM and 4(b) is the result of subsequent modifications. A second application involved in a group of five professional engineers, all residents of suburban Seattle. The objective of this group was to develop an understanding of the

ELECTRIC

PCMR

SUPPLY

----I

EwLOrnNl

psji+jf

c

I

I IMJJSTRIAL AND CQ’NRCIAL GRWTI-

5

FOSSIL FUEL CAPACITY

4 FWUATICN

GENERATING

7

$ SOLAR AND WIND F04ER GENERATING CAPK1l-f

.--

t

.Y

f-------a-7 ~~~~~R’-,P~~N~i~mu~~~GE

+ B3W4D PCHER

COST OF SOldWIND GENERATED ELECTRICITY

FOR ELECTRICAL

c R&DCtdSOlARpElD WIND P(Y(ER I ENERGY

53iSERVATIoN

PRICE OF ELhXAL

-““_t -1 COST OF NUCLEAR GENERATED ELECTRICITY

NUBER OF F(EM4INING HMROP(Y(ER SITES

A

4 COST OF FOSSIL FUEL GENERATED ELECTRICllY 4

STRENGTH LOBBY

OF “FCMER”

,STF(ENGTH OF M ENv1F73+milPl_ Lmw

v

=

AFFECTS

t PI&K

EN’dKt+ENTbl

SENSITIVtTY

Fig. 4(a). Initial digraph of the northwest electrical energy system.

OR

INFLUENCES

174

RICHARD

_I

H. WATSON

NWa/rorarrI

FOSSIL WPCIN

ML

GENERATING

<-

GEbWATlffi

GWACIN axT OF SCuR PM) WIND GE&FIAlEC ELECTRICIM 1

DEMPW

EhERGY OXSERVATION

FOR ELECTRIC4

t-PRICE

OF ELECTRICAL

CfJST OF NUCLEAR GENERATED ELECTRICITY

KhER

ELEClRlC1l-f

ce

STRWGM LCBBY

OF l3VIRWWTAL

-

Fig. 4(b).

Modified digrapb of the northwest

= AFFECTS

OR

INFLUENCES

electrical energy system.

system of factors or problems contributing to high automotive energy consumption. The resulting model was to be used to identify leverage points in the system as a step toward formulating proposed energy conservation strategies. This group spent one evening (approximately 2 % hours) identifying what they perceived as problems relevant to high automotive energy consumption using a nominal group process [25]. Another evening (approximately 3 hours) was spent structuring this system of problems using the ISM system for the relationship “aggravates or contributes to”. Part of a third evening was spent to discussing and modifying the KM digraph. The initial and modified digraphs developed by this group are shown on Figs. 5(a) and (b), respectively. The participants in a third application were 12 members of environmentally oriented citizens group. As in the previous case, the objective was to develop an understanding of

INTERPRETIVE

STRUCTURAL

MODELING

175

FOR TA

the factors contributing to high automotive energy consumption with the eventual purpose of formulating energy conservation strategies. In this instance, the group was presented with a problem list which was developed by the author primarily from literature sources. The participants spent some time discussing and modifying this problem list. These problems were then related using the EM system for the relation “aggravates or contributes to.” ENERGY

HIGH tv.rmwrIvE CCNSLWTICN

INAOEQUATE

PlBLIC

MISM4NIVjEFLKI

TRJNSPORTA-

OF PWLIC

FEDERJL REWATION AFFECIING PJJTO TECwa.CGY E.G, EMISSIONS STPNDARDS

_

I

I

LCW ALIT0 FUEL EFFlCIEtUZY

INEFFICIEKl USE OF AUTOS (LW OcclJPANcl 1

INSUFFICIENT ORIENTATION TO THE USE OF O3tWIC4TIcNS REIATIVELY FM( WIG

LIX FLNDING TW4SWRTATIcN

GENERALLY F0X OF MITI-FA’4ILY

LENDING

*LIVABILITYn HOUSING

1

1

INmEPWTE D3ELOiwN.r a? EsTRIcrIms m

4 LACK OF DEVELOFWNT OF ENERGY EFFICIENT AJXO PRO+WSION SYSTEb‘S

AND/

+

LACK OF CCWREHENSIM LPMI USE PLAFNING

AND TAX FOLICIES

d

4 w\RKErIffi PPXTICES GEMWILINFLLENCEOF m-r0 INrmTRY 4 Lad bum

AND

= AGGRAVATES CONTRIBUTES

OR TO

FUEL PRICES

Fig. 5(a). Initial digraph of the engineer group’s perceptions of problems related to the energy intensiveness of transportation in Washington State.

t t 4

176

INAIEWTE PlBLlC TIC?4 SERVICE

TWNSWRTA-

-

NEFFICIENT lmoccup/wcy

1

MlOF PIBLIC TRA%SFORTATIcN SYSTEMS Ala l-fxHaLoGY

FflATIVELY FOR PIBLIC

US

RICHARD

5OF AUTOS -

FEDERAL REGULATION AFFECTING AUTO TEWOLOGY E,G, EMlSSlCNS STAWWDS

LW FWDING TRWSPDRTATION

INSUFFICIENT ORIEmATICN TO THE USE Cf CCWWICATICNS

IN4rmu4TE KMLOPWNT OR RESTRICTIONS ON

lxfwd?.lBw

H. WATSON

Am/

IJB

6

t

FLEL EFFICIEBCY

t

lr

I

WTO

LlICKOFtEV ElLOPPEKTW EhERGY EFFI C IENT+vlc PROWLSIW S’fSTENS /

SPRAWL

tt

L

tECENWJ_IZATIcN PNll LACK OF CoORDINATIoN OF RESIDENTIAL AREAS AMZ SU’WRTING ACTIVITIES

IACK OF INI-EGRATION OF WORKING AJdD LIVING AREA? 1

t

c

t

GENEWLLY POOR nLIVABILIpI” OF MJLTI-FAMILY tDUSIt,G

UICK OF OYWWWSIM LPM) USE FUWING -

f IN STATE &WED TO HIG&‘,AYS& RATHER THAN TWNSWRTATIDN

Fig. S(b). siveness

Modified

digrapb

of transportation

DEDICATED STATE FWDIffi HIWAYS ( ICATED GASOLINE TAX ?

of the engineer

in Washington

group’s

CONTRIUTES

FOR

perceptions

= AGGRAVATES

of problems

related

to the energy

OR TO

inten-

State.

For a variety of reasons which will be discussed later, it was not possible to finish the KM session with this group. Two versions of the problem structure were developed by the author based on the group’s majority and minority responses to an initial series of ISM queries (approximately a third of the total queries). These structures (the unmodified initial diagraphs) are shown in Figs. 6(a) and (b).

INTERPRETIVE

STRUCTURAL

MODELING

177

FOR TA

The final application to be reported involved a group of eight students and faculty in a “head to head” comparison of ISM with a less formal unassisted approach in developing structural models for two different though related systems of similar complexity. Given DEGRADU)AIRQU&I~

,

LCMAUTO

HIGHTRMSWRTATION C%wTl,CN

FELECW

WOR

,

ENERGY f

PCCESSIBILITY

LACK OF t43BILIl-YFOR ""-;'I;"‘

INCFfASIffi FREQU3CYAMl OF AUTO TRIPS

LENGM

REGISTRATION FEES Do NOT EMDJRAGE AUTCM3TlM EFFICIENCY

II

HIGH COST OF NEW hJRE EFFICIENT) CARS

I 1

'4

POOR GKlAL1l-YTRANSIT

SERVICE

LENDING POLICIES OF FINANCIAL INSTITUTIONS 0 8

DIFFICULTY OF IMER'+X@L CONhECTIC%

I

1 iu INADEWATE CRITERIA FOR HIGHWAY CONSTRUCTION MCISIONS t 14

INAIY3JATE MSIT

FUNDING

t ~~~FCORGKIbldYOFALTERNATIM INTERCITY TRMSWRTATION I

lACK OF LA&

USE PLANvIffi

s

20 STATUS VALUE OF AUTO CMMSHIP AMI USE f 22 INSTITUTICfdAL STRl_kZTlJR G5AJ03 TO HIGHdAYS i-,._

1 POCITICAL KMER LOBBY

c

=

AGGRAVATES

OF HIMY

OR

CONTRIBUTES

TO

Fig. 6(a). Initial digraph of the environmental group’s perceptions of problems related to the energy intensiveness of transportation in Washington State. “Indiscriminate” use of feedback response.

RICHARD

178 DEGRADED

AIR

RX)R

0wLIl-Y

ACCESSIBILITY t

f HIGH TRMSFfXTATIcN CCMUYPTION

LOW AmC!

FUEL

H. WATSON

IACK OF MJBlLIpl N3N-CRIERS

ENERGY

INCREASING FREQUENCY OF AUTO TRIPS

ECCN(m

AND LENGTH

FOR

INEFFICIENT USE OF (Low CmmNCY)

ALli0.S

t REGISTRATION FEES D3 ENCOMGE AUTC,%.,TIM

NOT EFFICIENCY

P03R

QUALlpl

DIFFICULTY CCfdNECTIWS

IGH COST OF NEW r b0RE EFFICIENT) CARS

TRANSIT

OF

0

SERVICE

lNTERFUD,q

d FtXlR QUALITY 6F ALTEFP\U\TIVE INTERCITY TRJNSWRTATION

t

1 PUBLIC UdFPMILIARIPI WITH TRANSPORTATION ALTERNATIMS

UfwN

SPRAWL

-1

INADEQLLATE

TRANSIT

FMDING

T

LACK

OF iPND

INADEQUATE CRITERIA FOR HIGHWAY CONSTRUCTION DEClSlCNS

“SE

PLANNING i

t

t

/II

LENDING WLICIES INSTITl”!ONS

OF

FINANCIAL

INSTITUTIONAL STRUCTURE GEARU) TO HIGMAYS

-

I HIGMAY

TRUST

FUJD

e POLITICAL LOBBY STATUS VALVE AND USE -

=

AGGRAVATES

OF AUTO

OR

POWER OF

HIGMAY

0M4ERSHIP

CONTRIBUTES

TO

Fig. 6(b). Initial digrapb of the environmental group’s perceptions of problems related to the energy intensiveness of transportation in Washington State. Restricted use of feedback response.

the highly subjective nature of ISM or any competitive process, devising a controlled test was impossible. A total of over 9 hours (72 person hours) were allotted to this experiment. One system chosen for examination was the 1990 Northwest energy system given a moratorium on further nuclear development and continued historical growth in energy demand. The second was the 1990 Northwest energy system given moratorium and the establishment of policies leading to a zero per capita growth in energy demand by 1985. The initial three hour session was devoted to defining the elements of both these systems. The elements defined included various actors in the systems, different technological components of the system, and a variety of environmental, social, economic and institutional and value inputs. Again, a nominal group technique was employed. A second session was devoted to structuring one of the systems (the Historical Growth System) manually, i.e., with blackboard and chalk as the only aid to the process.

INTERPRETIVE

STRUCTURAL

MODELING

FOR TA

179

The approach used was basically reductionist, first segregating the elements into subsystems, then developing the structure within the subsystems, and finally, interrelating the subsystems. A third session was devoted to structuring the remaining system using ISM. In each application extreme care was taken to explain the role and operation of ISM to the participants. It was emphasized that ISM’s role was basically one of bookkeeping, recording the group’s responses and, on the basis of those responses, structuring the system elements according to a set of simple, logically consistent rules. Several observations about the ISM process and product were derived from the experiences gained in these and the other applications. The more important of these are discussed below. I. The Ultimute ISM Products (i.e., the Modified Digraphs) were Generally Judged to be Both Satisfactory und Useful Representations of the Object Systems. The participants generally felt that the final ISM produced digraphs were successful in capturing and communicating a holistic sense of system structure. The models were felt to be useful in opening up new insights on the possible reasons for observed system behavior and in focusing the groups attentions on key areas requiring in depth study and analysis. In one application (not reported here) the ISM model was used as the basis for further structural modeling using KSIM [26]. 2. Definition of the System Elements und Contextual Relation (Steps 1 and 2) is at Least us Important as Any of the Phases of the ISM Process. Ambiguities in these definitions or lack of agreement about the definitions can lead to confusion, frustration and inconsistencies in later phases. This factor was evident in each of the ISM experiments. Unfortunately, many such ambiguities are only recognizable when attempting to deal with interelement relationships during the computer directed querying phase of the ISM process. At that point, however, the inflexibility of the ISM process makes combining, redefining or eliminating existing elements or adding new elements d@cult. The ISM software does permit changing the text of the element definition at any point in the process. However, unless this is done the first time an element is encountered, redefining an element may require starting the process over again because judgments as to how the redefined element relates to other elements may well change. Elements can be combined or eliminated, subject to the same difficulties. Adding an element is not possible, although new elements can be inserted into the ISM produced digraph when modifying. This latter problem is avoided in the newer version of ISM where elements are introduced one at a time. Still, extreme care should be taken in the group process in which system elements are defined. A variety of group processes such as the nominal group technique, brainwriting and others should be examined to determine which are most appropriate to the ISM process. 3. Almost Invariably the Initial Directed Graph Produced Using ISM is Only Partially Satisfactory. The reasons for this are several. One is related to the heuristic nature of the ISM process. The results of inconsistencies or irrationalities in group reasoning are graphically displayed, easily recognized, and easily corrected in the initial ISM digraph. An illustration of this is shown on Figs. 4(a) and (b) where 4(a) is the “before” model, (i.e., the initial ISM digraph and 4(b) is the “after” model, with modifications included. In Fig. 4(a), a variety of impacts are shown as caused by the demand for electrical power. These

180

RICHARD

H. WATSON

impacts are, in turn, shown as affecting the amount of generating capacity in each of several technologies. On inspection of this model, it was decided that logically the impacts were more directly a result of the generating capacity which, in turn, was affected by demand. This portion of the model was therefore modified to reflect this perception. The construction and subsequent modification of the ISM digraph is a feedback learning process and should be counted as a positive feature of ISM. There are also factors related directly to the ISM algorithm which sometimes result in the ISM digraph not reflecting all of the user groups perceptions of system structure, particularly for affective type relationships. For instance, the ISM algorithm produces the minimum edge digraph. As a result, some direct relationships which may be considered very important by the ISM participants are not captured by the ISM algorithm. In terms of reachability, these relationships are redundant. This fact is illustrated in Figs. 5(a) and (b). These digraphs represent the problem system related to automotive enegenergy conservation in urban Washington. In the initial digraph (5a) urban/suburban sprawl was represented as “aggravating” automotive energy consumption only indirectly. Review of this structure resulted in (among many other modifications) the addition of a direct link from urban/suburban sprawl to automotive energy consumption as a result of the group’s perception of sprawl resulting in greater trip lengths. It would obviously be “nice” if ISM were not constrained to the minimum edge algorithm. As it stands, however, the appropriate view of the initial ISM digraph is clearly as a basic skeleton structure which is subsequently to befleshed out. The contribution of ISM in helping to sort through a complex system to define that initial skeleton structure is significant. However, there is some danger that the initial ISM product might lead to premature closure of the structuring process. 4. The ISM Algorithm Cannot Adequately Handle Feedback. By now, most of us have internalized the notion that the first law of systems is that “everything is connected to everything else” and that some of the more important structural characteristics of complex systems are the feedback loops by which this interconnectedness is achieved. However, as previously discussed, the ISM algorithm produces a multilevel hierarchical structure in which it is not possible to have feedback from one level in the hierarchy to lower levels. If in an ISM session the mutual interaction or feedback response is used for those element pairs between which a feedback relationship exists, the effect is to collapse all the elements which are part of that feedback loop into a single level of the structure (forming a cycle in graph theory nomenclature). More importantly, the effect is to remove all but one member of that feedback loop (sometimes referred to as the proxy element) from any further consideration in the ISM process. This results in a severe loss of the quantity of information contained in the resulting graph. None of the detail structure within the feedback loop or linkages between elements outside the loop and most of the elements of the loop are explored. In order to deal effectively with systems which exhibit feedback, it was found to be advantageous to restrict use of the ISM feedback response to those instances where (1) the elements are mutually interrelated, (2) the interactions are of equal strength; and (3) the interactions are simultaneous, i.e., there is no delay between the action of one element on the other and the subsequent feedback. Where these are not met, the direction of the dominant or preceding relation is used to determine the ISM response. An example of the indiscriminant use of the feedback response is shown in Fig. 6(a). In this example, the majority of the participants could not adapt to the need to restrict the use of feedback. As a

INTERPRETIVE

STRUCTURAL

MODELING

FOR TA

181

result, the resulting structure contains two large cycles. (Elements 1, 2,4, 8, 13, 14, 16, and 17 and elements 11, 19, 20, and 22). While these elements are probably all interrelated (albeit only weakly in some cases), the information content of this graph is much less than that of Fig. 6(b) which was developed using the more restricted definition of the feedback response, it is necessary to incorporate feedbacks when modifying the initial ISM digraph. The addition of feedback is illustrated in Figs. 5(a) and (b). Again, as long us the initial ISM digraph is viewed us a skeleton structure for subsequent elaboration, it is possible and relatively easy to incorporute feedback into the final structure. 5. ISM Requires u Single Response und Cannot Easily Incorporate Minority Views. The ISM process requires a single response to the computer query when, in fact, the user group may be severely divided on what the appropriate response should be. Ideally, one would like to be able to develop the system structures corresponding to both the majority and minority views. It is possible to record the minority view, either separately or typed directly on the session record if a printing terminal is being used. However, since the querying sequence is dynamic (which elements pairs are considered depends on the responses to previous queries) it is impossible to merely record votes and rerun the ISM session according to minority views. This is a serious drawback if, as McLean suggests, the emphasis in structural modeling should be on developing multiple alternative system structures [ 131. 6. The ISM Process is Extremely Time Consuming. Regardless of the numerical efficiency of the ISM system, an ISM exercise (i.e., the responses to the computer directed queries) is time intensive. For example, the initial structure of Fig. 5(a) required 2 hours and 32 minutes of intensive interaction with the computer during which time 107 responses were made. The average response time of 1.5 minutes seems to be fairly typical for small groups. The average time per response has been found to be a function of the group size and the loquacity and argumentativeness of the group members. In most instances the groups were composed of from four to seven members. In these cases, the querying and responses could easily be handled in an informal conversational manner. As mentioned, however, one group of 12 was involved. Because of the more cumbersome group size, the ISM queries were displayed to the group via an overhead projector. The group responses were tallied using a Group Dialog System voting machine [27]. Unfortunately, this experiment was not very successful. Some of the user group were apparently put off by all the technology (computer terminal, projector, and Group Dialog System.) Others felt that their input was being unduly manipulated by the ISM process, despite a careful attempt to explain the process and the computer algorithm. Still other members were apparently engaging in gamesmanship, attempting to manipulate the group response. Finally, the formal tallying of each response was time consuming. The net result was a record 4 minutes per response and an aborted session as the hour grew late and the patience of the participants grew short. On the basis of this very limited trial, it would appear that the use of ISM should be restricted to small groups for which informal query-discussion-response procedures can be used successfully. 7. The ISM Process is Highly Structured, Allows Little Freedom to the Participants and can Become Very Tedious. To use the ISM system, it is necessary for the user group to surrender control of its processes to the ISM algorithm. This apparently has both advantages and disadvantages.

182

RICHARD

H. WATSON

The ISM system is a stern task master and does tend to keep the discussion focused via constant querying. Similarly, the ISM process also insures comprehensiveness, at least within the limitations imposed by the minimum edge, multilevel digraph algorithm. _ At the same time, however, the ISM process is rigid and inflexible. In some respects, this is a safety feature, preventing the group from inadvertently skipping over important relationships. In other respects, the necessity to conform to the requirements of the ISM algorithm may largely prevent or at least make it difficult to take advantage of those intuitive leaps, serendipitous occurrences, and synergistic effects which often make group processes productive. Moreover, it was obvious that many participants chaffed under the need to make their thought processes subservient to the data requirements of the system. Whether or not this affected the quality of their participation is not known. Finally, the ISM process can become extremely tedious, particularly for large systems (20 or more elements). Aside from the previously discussed time-intensity of the process, one reason for this is the “first element syndrome’ so named because in the first series of ISM queries, the first element in the element list is tested against every other element in the system. For large systems, the participants soon become sick of that first element. (The newer ISM algorithm avoids this problem.) The other and more significant reason for the tedium of ISM is that in most systems there are many element pairs which are obviously unrelated. Unfortunately, with ISM it is necessary to deal explicitly with many of these pairs, particularly in the last step (interconnection) of the ISM algorithm. This is tedious and aggravating and on more than one occasion prompted partially facetious comments like “can’t the stupid computer figure that out by now?” It is probable that this factor combined with the length of the ISM sessions contributes to a decreasing quality of the thought which decides responses as sessions wear on. 8. The ISM Process and Product are Sensitive to the Ordering of Elements in the Element Set. The elements which are paired in the ISM queries depend on the order of the elements in the element list. A different ordering results in a different set of queries. Faced with different queries, it was observed that there were differences in the structures derived from the same element set with elements in different order. This does not reflect any inconsistencies in the ISM algorithm but does indicate some difference in the subjective decision processes triggered by different queries. This would be particularly troubling if one were concerned with comparing the system perceptions of two or more different groups who may not have their element list ordered identically.

9. The Skill of the “Facilitators’ ’ and the Commitment of the Participants Strongly Affect the Success of an ISM Session. A distinct learning curve phenomena was observed on the part of the session facilitator or moderator. As the facilitator became more skillful at mediating between the participants and the ISM software and hardware, the effectiveness of the ISM process improved. By the same token, there was an observable correlation between the degree to which the participants felt they had a stake in the outcomes and the effectiveness of ISM. Those groups with a commitment to the outcome were more willing to work through whatever difficulties they perceived in the ISM process to get to the product.

INTERPRETIVE

STRUCTURAL

MODELING

FOR TA

183

10. The Question as to Whether or not ISM is Superior to Less Formal Approaches to Structuring Complex Systems is Unresolved. The experiment described in which ISM competed with a less formal approach was inconclusive, primarily as result of deficiencies in the design and execution of the experiment. The fact that these were contrived experiments and not actual applications in which the participants had a stake undoubtedly had detrimental consequences. In neither instance was it possible to complete the structure because of time limitations. In the manual case, the group got as far as defining and structuring the subsystems, and very roughly interrelating the subsystems. No detailed interrelation between elements of different subsystems was accomplished. In the ISM case, the initial structure was developed but not modified. One observation that can be made is that the greater flexibility of the manual approach was valuable. In several instances, ambiguous elements were redefined, redundant elements were eliminated or incorporated into others and missing elements were added. The difficulty in doing this during the ISM session was perceived as a problem. It is also not clear that the quality of information exchange during the ISM process was superior to the manual process. What ISM may gain in terms of comprehensiveness and intensity may be somewhat offset by the evolving, heuristic, visual nature of the manual process. The ability to see the structure evolve, even if grossly, may have some advantages. In short, there is the need for more comparisons of this nature before it can be asserted that the ISM system offers any significant advantages. Conclusions This paper does not question the utility of structural models. In this author’s experience, structural models are a valuable means of developing, recording and communicating perceptions of complexity. Methods which promise to make this process more effective merit careful examination. Interpretive Structural Modeling is such a method. Individuals or groups using ISM gain the virtures of the systematic, comprehensive nature of the ISM process and the intellectual labor saving provided by the algorithm’s computerized bookkeeping. Against these virtues they must balance some liabilities. The less important of these relate to the possible lack of a one-to-one correspondence between the initial structural model produced using ISM and the user’s actual perception of system structure. This problem stems from the fact that ISM uses the reachability matrix to develop the minimum edge, hierarchical representation of system structure. Given that this is recognized, it is relatively easy to incorporate missing relationships during post-ISM modification of the ISM model. More serious problems arise from the basic inflexibility of the ISM process. Among these are the difficulty of adding, deleting, combining or redefining elements in the course of an ISM session, the inability to easily incorporate minority viewpoints, the time intensive and often tedious nature of the process, and the inability to review the structure as it evolves. Some of these problems can be alleviated, if not eliminated, by modifications to the software. One which immediately suggests itself is to modify the algorithm to allow the users to choose the pivot element (the element with which all others are paired) at each step in the initial phases of ISM queries. This would give the users greater control and would probably make the process less time consuming. A second modification would allow the users to view the intermediate structures and modify them as the ISM session

184

RICHARD

H. WATSON

progressed. This would increase the heuristic nature of the process. Both these modifications have apparently been incorporated into the newest version of ISM. This represents a really significant advance. Finally, more experimentation is needed with more flexible and more visual approaches to developing structural models while such approaches have not had the benefit of intensive development, they may however, provide the balance between systematic comprehensiveness and heuristic flexibility which ISM seems to lack at present. References 1. Rhyne, R. F., Communicating Holistic Insights, Fi&fs within Fields within Fields 5 (l), (1972). 2. Coates, Joeseph F., Technology Assessment and the Public Wisdom, Journal of the Washington Academy of Sciences 68 (I), 4(1975). 3. Coatea. Joseph F.. Some Methods and Techniques for Comprehensive Impact Assessment, Technol. Forecost. Sot. Chunge 6, 341-357 (1974). 4. Gordon, Theodore .I., A Discussion of Some Methods of Technology Assessment, The Fufures Group Report No. 138-01-17, Glastonbury, Connecticut, April I, 1974. Approaches to Technology Assess5. Chen, Km and Ztssts, George J., Philosophical and Methodological ment, Environmental Research Institute of Michigan, June 1973. 6. Coates, Joseph F.. The Role of Formal Models in Technology Assessment. Technol. Forecost. Sot,. Change 9, 139-190 (1976). 7. Coates, Joseph F., Technology Assessment-A Tool Kit, CHf?MTECK 372-383 (June 1976). 8. McLean, Mich and Shepard, Paul, The Importance of Model Structure Furures 3 (I), 4 (1976). 9. Kane, Julius and Vertinsky, Alan B., The Arithmetic and Geometry of the Future, Technol. Forecust. Sot. Change 8, 115-130 (1975). Team in Perspectives on Technology As~essmmt. Sherry IO. Taylor, James. Building an Interdisciplinary Arnstein and Alexander Christakis, eds., Science and Technology Publishers, Jerusalem, 1975. II. Mar, Brian W. Where Resources and Environmental Simulation Models are Going Wrong, Dept. of Civil Engineering, University of Washington, 1973, unpublished paper. 12. Lendaris, George G. and Wayne W. Wakeland, Structural Modeling-A Bird’s Eye View, Systems Science Ph.D. Program. Portland State University, (February 1977). Role for Structural Modeling, in Futures Reseurch I Neti 13. McLean, Mick, Getting the Problem Right-A Directions. H. A. Linatone & W. H. C. Simmonda. eds., Addison-Wesley, Reading, Ma, 1977. Kazuhtko and Christakia, Alexander, Methods for Structural Modeling, IEEEKMC fnter14. Kawamura, nutiorml Conference. (November 1976). 15. Warfield, John D.. Assault on Complexity. Battclle Monograph Number 3, Battelle Memorial Institute, Columbus, 1973. p. 13-I. 16. Ibid, p. 13-3. 17. Warlield, John N., Structuring Complex Systems, Battelle Monograph No. 4 Battelle Memorial Institute, Columbus, April 1974. 18. Brill, Evan, et al., The Interpretive Structural Modeling System in Users’ Guide, Battelle Columbus Laboratories, Columbus, Ohio, 1975, Vol. 1, pp. 2-3. 19. Hart, William L. and Malone, David W. Goal Setting for a State Environmental Agency, Proceedings of the 1974 IEEE Conference on Decision and Control, (November 1974). Robert W. and Sage, A. P., Applications of Interpretive Structural Modeling to Higher 20. Hawthorne, Education Program Planning, Socio-Economic Planning Sciences 9, 3 143 (1975). of Interpretive 21. Brand, Dewitt, H., Jr., Irwin, Donna M., and Kawamura, Kazuhiko, Implementation Structural Modeling in a State-Level Planning Context, Seventh Annual Pittsburgh Conference on Modeling and Simulation, (April 1976). 22. Kawamura, Kazuhiko, and Christakis, Alexander N., The Role of Structural Modeling in Technology Assessment, Second International Congress on Technology Assessment, (October 1976). 23. Several applications of ISM by Battelle associates are found in the Battelle Monograph Portraits of Complexity4pplicutions of Systems Methodologies to Problems, Maynard M. Baldwin, ed., Battelle Monograph No. 3. Battelle Memorial Institute, Columbus, 1975. for Designing Dynamic Models 24. Fitz, Raymond and Homback, Daniel, A Participative Methodology Through Structural Models, Seventh Annual Pittsburgh Conference on Modeling and Similation, (April 1976).

INTERPRETIVE

STRUCTURAL

MODELING

25.

FOR TA

185

Delbecq, Andre L., Vande Ven, Andrew A., and Gustafson, David H., Group Techniques for Program Planning--A Guide to Nominal Group and Delphi Processes, Scott Foresman and Company, Glenview, Illinois, 1975, p. 40. 26. Kane, Julius, A Primer for a New Gross-Impact Language-KSIM, Technol. Forecast. Sot. Change, 4, 129-142 (1972). 27. Sheridan, Thomas, Community Dialog Technology, Proceedings of the IEEE, C* (3), 463475 March 1975. Received 6 December

1976; revised 6 May 1977