Challenges and promises of integrating knowledge engineering and qualitative methods

Computers & Geo,lciences Vol. 16. No. 6. pp. 847-856. 1990 Printed in Great Britain. All rights reserved

0098-3004,90 $3.00 + 0.00 Copyright ~ 1990 Pergamon Press pie

CHALLENGES AND PROMISES OF INTEGRATING KNOWLEDGE ENGINEERING AND QUALITATIVE METHODS C. GUSTAV LUNDBERG12 and GUNILLA HOLM~ t School of Business and Administration, Duquesne University, 600 Forbes Avenue, Pittsburgh. PA 15282, U.S.A., 2Swedish School of Economics and Business Administration, Arkadiagatan 22, 00100 Helsinki, Finland and 3Department of Education and Professional Development, Western Michigan University, Kalamazoo, MI 49008, U.S.A. (Received 6 February 1990) Abstract---Our goal is to expose some of the close ties that exist between knowledge engineering (KE) and qualitative methodology (QM). Many key concepts of qualitative research, for example meaning, commonsense, understanding, and everyday life. overlap with central research concerns in artificial intelligence. These shared interests constitute a largely unexplored avenue for interdisciplinary cooperation. We compare and take some steps toward integrating two historicallydiverse methodologies by exploring the commonalities of KE and QM both from a substantive and a methodological/technical perspective. In the second part of this essay, we address knowledge acquisition problems and procedures. Knowledge acquisition within KE has been based primarily on cognitive psychology/science foundations, whereas knowledge acquisition within QM has a broader foundation in phenomenology, symbolic interactionism, and ethnomethodology. Our discussion and examples are interdisciplinary in nature. We do not suggest that there is a clash between the KE and QM frameworks, but rather that the lack of communication potentially may limit each framework's future development. Key Words : Knowledge engineering, Knowledge acquisition, Qualitative research, Ill-structured problems, Problem and model postulation.

INTRODUCTION Our goal with this article is to expose some of the close ties that exist between knowledge engineering (KE) and qualitative methods (QM). It is interesting that so many key concepts of qualitative research, for example meaning, commonsense, understanding, and everyday life, overlap with central research concerns in artificial intelligence--the mother discipline of KE (see, e.g. Bogdan and Biklen, 1982; S. J. Smith, 1988; Giddens, 1984; Pred, 1977, 1981; Chiu, 1989; Kuipers, 1986, 1989; Lenat, Prakash, and Shepherd, 1985). These shared interests constitute an unexplored avenue for interdisciplinary cooperation. Whereas KE's theoretical underpinnings tend to be focused narrowly on cognitive science, QMs have deeper and broader roots in the social sciences. We suggest that many of these underpinnings relate to KE as well. In this paper we compare and take some steps toward integrating two historically diverse methodologies. We discuss the mutual benefits associated with an alliance between KE and QM. First, the commonalities of KE and QM are explored both from a substantive and a methodological/technical perspective. We then outline the stages of KE and QM-based research, comparing and contrasting the

rationale for each framework. The KE and QM research and application efforts are viewed from the perspective of process rationality (Simon, 1986) and problem solving within ill-structured domains. We also address feasibility issues pertaining to each framework, as well as problem and model postulation through QM and KE. In the second part of this paper, we address knowledge acquisition problems and procedures. Knowledge acquisition constitutes a central bottleneck for both knowledge engineers and researchers utilizing qualitative methods. To date, knowledge acquisition within KE has been based primarily on cognitive psychology/science foundations, whereas knowledge acquisition within QM has a broader foundation in phenomenology, symbolic interactionism, and ethnomethodology. Our discussion and examples are interdisciplinary in nature. We do not suggest that there is a clash between the KE and QM frameworks, but rather that the lack of communication potentially may limit each framework's future development. Finally, we are unable to provide a full review of all topics discussed in the article. Readers interested in a more comprehensive view of KE and QM may consult any of the numerous introductory texts. 847

848

C.G. LL,'NDBERGand G. HOLM A knowledge-based model is built around three basic structural components: (1) a heuristic knowledge base, (2) capabilities to perceive the environment (some input device), and (3) an inference engine. The knowledge base expands with the complexity of the task potentially without an accompanying structural metamorphosis. An inference engine is a structure that is designed to manage the system's goals and method (Lundberg, 1989) by selecting and applying knowledge from the knowledge base to the problem at hand (Lundberg and Robinson, 1988). (For further detail consult, e.g. Barr and Feigenbaum, 1981; Duda and Shortliffe, 1983; O'Shea. Self, and Thomas, 1987.) The inference engine, for example, may contain mechanisms for deriving new facts from known facts (Harmon and King, 1985) and for conflict resolution in situations where more than one rule in the knowledge base is a strong firing candidate. On the other hand, a researcher utilizing QMs emphasizes the necessity of holistic analysis: "Focusing on a narrow set of variables necessarily sets up a filtering screen between the researcher and the phenomena he is attempting to comprehend. Such barriers ... inhibit and thwart the observer from a necessary closeness to the data, from an understanding of what is unique as well as what is generalizable from the data, and from perceiving the processes involved in contrast to simply the outcomes" (Rist, 1977, p. 47). Qualitative research may be conducted in natural settings. The data collected have been termed soft, that is, rich in description of people, places, and conversations (Bogdan and Biklen, 1982) and the description "thick" as it attempts to expose "the meaning particular social actions have for the actors whose actions they are" (Geertz, 1973, p. 27). The data are not handled easily by statistical procedures. Researchers conducting qualitative research do not approach the research with specific questions to answer or hypotheses to test (Wets, 1985; Fetterman, 1989; Jorgensen, 1989). Out of many traditions of qualitative research, we quickly review only two--symbolic interactionism and ethnomethodology--in order to clarify the central characteristics of qualitative methods. We focus on these two methodologies because they constitute a theoretical backdrop for an entire family of techniques currently employed in qualitative research.

COMMONALITIES OF KE AND QM Knowledge engineering is the art of bringing the principles of artificial intelligence to help with difficult problems that require an expert's knowledge for their solution: " . . . the art of building computer programs that represent and reason with knowledge of the world" (Feigenbaum, 1977, p. 1016). A KE-project usually evolves through the stages of task identification, need assessment, cost-benefit evaluation, development of contacts and access to expert(s), knowledge extraction, codification of knowledge, testing and evaluation of program, and implementation (Lundberg, 1989). Hayes-Roth's (1984) stage description focuses on the iterative nature of the KE process: (1) knowledge acquisition results in concepts and rules, (2) knowledge system design builds representations for the model's framework and its knowledge, (3) knowledge programming focuses on the knowledge base and the inference engine, whereas (4) knowledge refinement involves consecutive revision of concepts and rules. We see an innate similarity between the described process and that outlined by Miles and Huberman (1988) for the analysis of qualitative data. Miles and Hubcrman's process is depicted in Figure I. The horizontal axis in that diagram expresses time, hence dividing the data analysis process into three components: an anticipatory period, a during-datacollection period, and a post-data-collection period. Miles and ttuberman stress that the four subprocesses overlap for a significant period of time, each suggesting additions and modifications to the system. Collected data or the coding of data, for example, may suggest a preliminary conclusion that in turn can suggest further and fine-tuned data collection and reduction. The Miles and Huberman process is descriptive of applied knowledge acquisition and engineering. In KE, however, this evolution extends its full course into model building. In general terms, a knowledge-based model's data constitute its model. These data may be expressed better in symbolic rather than numerical operations (Duda and Shortliffe, 1983). More specifically, the application domain may play a decisive role in the selection of program structure (Lundberg and Robinson, 1988). These three features constitute the most appealing reason for researchers employing QMs to take some steps toward KE. Data C o l l e c t i o n .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Data R e d u c t i o n .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Data Displays .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Conclusion-Drawing/Verification .

anticip,

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

during

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

post

................................................... >

time

Figure I. Miles and Huberman's process for analysis of qualitative data.

Integrating knowledge engineering and qualitative methods Symbolic interactionism builds on the assumption that human expertise is mediated by interpretation, and that objects, people, situations, and events do not possess their own meaning but rather have meaning conferred on them. The meaning people give to their experience is essential to what the experience is; to how they actively create their world. In Bogdan and Biklen's (1982, p. 33) words: "People act, not on the basis of predetermined responses to predefined objects, but rather as interpreting, defining, symbolic animals whose behavior can only be understood by having the researcher enter into the defining process through such methods as participant observation". People in a given situation may develop common definitions ("shared perspectives") because they regularly interact and share experiences (Bogdan and Biklen, 1982). Ethnomethodology refers to the study of how individuals "create" their daily lives. Researchers in this tradition focus on terms such as "'commonsense understanding", "everyday life", and "practical accomplishment", concerned mainly with microissues such as specifics of conversation and vocabulary and details of action and understanding (Bogdan and Biklen, 1982). Ethnomethodologists (e.g. Garfinkel, 1967) focus on the processes by which individuals manage to produce and sustain a sense of social structure and not on the activity itself (Meltzer, Pctras, and Reynolds, 1975). Similar to KE, cthnomethodology may Ibcus on specitic persons in microsettings. Ramos (1979), however, suggests an implicit macro element because the activities the people create to manage their daily lives have impact also outside the microsetting. Knowledge engineering as well as QM are rooted firmly in process rather than substantive rationality (Simon, 1986): that is both research frameworks are equipped to address explicitly and empirically the dynamics of values and attention crucial for postulations of the l:actual bases for reasoning. They seek to determine the strategies and procedures used in reasoning, so that limited information-processing capabilities can cope with complex realities, and they seek to describe and explain the ways in which nonrational (e.g. emotion:d or motivational) proccsscs influence reasoning. As a result, complex, reallife models may be constructed without the drastic assumptions about both people and their environments that may be associated with models characterized by what Simon (1986) defines as substantive rationality (excmplitied with neoclassical economics). Explorations of the rationale for KE and QM, respectively, reveal that the frameworks have many common features. Each KE or QM project takes place in a rich (Bogdan and Biklen, 1982) and often ill-structured (Simon, 1973) domain. An ill-structured problem lacks a lixcd solution algorithm and maybe even clear goal achievement criteria. The problem's solution may require far more knowledge than the information provided in the problem statement;

849

knowledge that may be used in the important process of imposing a structure on the task. For example, Voss and others (1983) provide a detailed illustration of the process of solving the ill-structured problem of "Soviet agricultural production", whereas HayesRoth and Hayes-Roth (1979) develop a computational model of task scheduling, and G. F. Smith (1988) a heuristic theory of problem solving within management science. Most often ill-structured problems of interest to KE and QM-researchers are highly context sensitive. This is evident in qualitative studies, for example in classroom ethnographies (Holm-Lundberg, 1986) where the environment is characterized by multiple parallel activity processes and perhaps "hidden curricula", or in explorations of the social structure of a lower class slum (Whyte, 1955). The contextuality is less evident in a knowledge-based model consisting of heuristic knowledge. Most knowledgebased models, however, are context and situation specific. In expert commodity trading, for example, no individual het.ristic carries much weight in isolation, and frequently a reading may take on diametrically opposite interpretations as the context changes (Lundberg and Barna, 1987). The drive toward greater context sensitivity was initiated by QM-researchers who were dissatisfied with the level of aggregation in explanation and theory building. For example, some researchers within the sociology of education objected to the study of schools as "black boxes" where little was known about what actually was taking place in the school. Qualitative researchers stress the need to look behind the structural surface for underlying and latent behavioral processes, with an eye on augmenting existing theory (cf. McCall and Simmons, 1969; Bogdan and Taylor, 1975; Apple and Wcis, 1983). Likewise, cognitive scientists and knowledge engineers stress that expert reasoning to a considerable extent is based on large amounts of domain specific knowledge (Lesgold, 1983; Glaser, 1987); knowledge that long was considered trivial and off-limits to scientific study. In addition, many KE applications concern specific problems, for example trouble shooting at Campbell's Soup (Fersko-Weiss, 1985) or managing deep-space probes for NASA (Broad, 1989). Lundberg and Robinson (1988) suggest that the feasibility of KE as a problem-solving framework should be evaluated from three different angles: cognitive, cost-benefit, and theory (re)construction. Traditionally, cognitive and cost-benefit considerations have taken the front seat among knowledge engineers. Among QM researchers, on the other hand, almost all the emphasis is placed on theory (re)construction (e.g. Willis, 1977). Knowledge engineering is most suitable for tasks that are costly, amenable to computerization, where expertise is nonubiquitous, where a timely solution of the problem has high value, and where human reasoning

850

C.G. LUNDBERGand G. HOLM

is likely to be affected by stress or pressure (Freiling and others, 1985; Lundberg and Robinson, 1988; Widman and Loparo, 1989). The models may be single person models, or models of compound and abstract expertise. In many situations there are sound cognitive reasons to stay with single experts. First, it may be difficult to locate several experts on the same level of expertise. Second, every reasoner attends to underlying factors and defines reality somewhat uniquely, and employs a personal reasoning style. Amalgams of knowledge from several reasoning styles may produce a model of little psychological interest, and even substandard reasoning. As an example, the Lundberg and Barna (1987) commodity trading model is based firmly on a particular view of markets--one that leaves room for nonefficient behavior--a particular world-view, and a unique mix of standard trading rules. In areas where the paradigms are more established and the cognitive styles more uniform, say in certain fields of medicine, it makes more sense to collect rules and reasoning strategies from multiple reasoners. A model based on multiple experts may possess superhuman reasoning capabilities because it mixes the machine's strengths (e.g. search, procedural knowledge, and "short-term memory") with human strengths (strategy, metaknowledge, experience, and performance history). Mittal and Dym (1985) address some of the problems and potentials associated with knowledge acquisition from multiple experts, but in most situations KE shuns away from models of interpersonal dependencies that figure so prominently in QM-based research. Qualitative methods are most suitable in domains that are holistic, characterized by significant interpersonal interaction, where the researcher sees a need for model and problem postulation, and where actors are unable or unwilling to verbalize their reasoning processes. We defer the verbalization problems to the knowledge acquisition section of this essay, focusing here on broader issues characterizing the mutual domain. Despite its technical appearance and focus on domain-specific applications, KE has laid the groundwork for "holistic'" modeling. This is the result of the incremental nature of KE-models and the relative lack of restricting assumptions. Both QM and KE shun reductionism, although this feature is seldom stressed in the KE-camp. The reluctance of KE to take this step is related closely to the knowledge engineers' modest view of their ability to contribute to theory. Lundberg (1989) suggests that KE can become an important contributor to theory (re)construction if it takes the route of cognitive science, that is focuses on building domain theories around a multitude of carefully modeled individual processes. Rapidly emerging theories of expertise (e.g. Sternberg, 1985a, 1985b) is a prime example of this theory building process, having evolved through numerous and varied novice-expert studies.

Importantly, even when used for theory construction purposes, KE models tend to be "'shallow" operationalizations of their application domains, because conclusions are drawn directly from observable features of the present situation. In a "'deeper" model an underlying (often covert) mechanism accounts for the observable facts (Kuipers, 1984). Qualitative simulation (within AI). for example, is based on a structural description containing constraints holding among time-varying parameters, and a behavioral description consisting of a finite set of time-points representing the qualitatively distinct states of the system and each parameter value (Kuipers, 1984). Artificial intelligence work on deep models is in its infancy. For QM-researchers, however, models tend to be interpretations tied to theories. These researchers usually stress that the knowledgeacquisition phase should be unaffected by theoretical assumptions, and primarily geared towards generating hypotheses. Model building, in the relatively few instances where QM-researchers take the full step into operationalizations, tends to be associated with attempts to locate a theoretical home for their findings. Well aware that there arc important differences between modeling commonsense reasoning in say medicine and in everyday behavior, we suggest that the divergence in the two frameworks' utilization of underlying theory warrant further investigation. Research based on qualitative methods may be able to show KE the way to a more aggressive theory (re)construction stance, whereas KE may provide the tools for modeling the processes that QM-researchers so successfully unearth. Model and problem postulation figure as a prominent rationale for both KE and QM. Model postulation is the process of specifying the underlying process in situations where only consequences or outcomes are known, ttayes-Roth and Hayes-Roth's (1979) cognitive science model of the planning process derived through observation of subjects scheduling a set of everyday errands, and Philips' (1983) qualitative study of the roots of Native Americans' communication problems in Anglo classrooms, constitute instructive examples of how the underlying processes and problems can be specified. HolmLundberg's (1986) ethnographic work on the implementation of gender policies in urban high schools in Helsinki, constitutes an example of problem and model postulation. In this example, she reveals misconceptions both with respect to the outcome (student and teacher attitudes and actions) and to the model of how attitudes are changed. Farber, Lundberg, and Holm (1987) have built on this example and have managed to cast some QM findings in KE formalizations. KNOWLEDGE ACQUISITION We next turn to a discussion of knowledge acquisition, comparing, and contrasting a considerable set

Integrating knowledge engineering and qualitative methods of methodologies and techniques for acquisition that have been developed by KE and QM researchers. Again, despite some striking similarities in purpose, the two researcher groups rarely draw upon each other's findings. We illustrate the potential power of a cross-over with a simple example. It is of interest both to scientists and practitioners (planners, farmers, etc.) to understand both the physical and social factors that cause desertification and to build KE models of the processes. A knowledge engineer may note it easier to enlist experts on the physical and the generalizable aspects of desertification, than to enroll local experts and agents in attempts to isolate the micro and localized social and economic processes in operation. The latter insights may be obtained best through QMs based on participation in the socioeconomic system--by "living it to learn it" (cf. S. J. Smith, 1988). Knowledge engineering researchers have explored primarily the cognitive aspects of knowledge acquisition and expertise: acquisition is difficult when processes are rehearsed thoroughly (i.e. when heuristics have become automatic), when reasoners use images and analogs, or when they attempt to verbalize past behavior (Ericsson and Simon, 1984). Knowledge engineers have paid little attention to the affective side, for example to pride and jealousy of what one knows (e.g. a master cartographer, a fiction writer, or a cardiologist may not want to get exposed to some of the mundane cognitive reasons for why they know). Also, relatively little KE work has focused on the problems of subcultures, integrated (say multiperson) systems, power relationships, and activities with underspecified or bogus goals (for notable exceptions see Carbonell, 1979, 1980; Banerjee, 1986). Recently, Hoffman (1987) and Lundberg (1989) have explored alternative KE techniques for knowledge acquisition. Hoffman presents five types of procedures: analysis of tasks familiar to the expert, structured and unstructured interviews, limited formation tasks, constrained processing tasks, and the method of "tough cases". However, frequently when real-life (familiar) stimuli are used in KE, the subjects are unable to isolate more than fundamental and general heuristics (Lundberg and Barna, 1987). In such experimental situations, the expert can match the data directly against salient and concrete memory traces, and may be unable to retrieve well-rehearsed (automatically firing) heuristics. Interestingly, QM-developers have addressed several fundamental issues pertaining to KE's knowledge acquisition methodology, notably (un)structured interviews, and methods based on the assumption that deliberate violations of the constraints that are involved in the familiar tasks can be used to expose reasoning (see Hoffman's third and fourth types of methods). Many QMs, similar to their KE counterparts, actively utilize informal field inquiry. Whereas knowledge engineers may take a rather casual stance CAGEO 16 6 ~ H

851

to unstructured interviews and see them as a freefloating gateway to more structured engineering, interpretative researchers make a virtue of nonstandardization (Jorgensen, 1989: Fetterman, 1989). Given that the purpose of QM may be to produce hypotheses rather than to verify them, participant observation and informal interviewing enable the researcher to reformulate the problem as she/he goes along, allows her/him to use situational knowledge to avoid misleading or meaningless questions, and to ease into the field at an appropriate pace (LeCompte and Goetz, 1982a, 1982b; Crain, 1977: Reichardt and Cook, 1979). Proponents of the procedures also stress that field workers can infer motives more validly by contrasting stated ideals with actual behavior, distort difficult-to-quantify variables less than researchers attempting quantification, and have a big advantage over, for example the survey researcher in delicate situations where covert research may be essential (Dean, Eichhorn, and Dean, 1969). Participant observation (also termed fieldwork or naturalistic observations) provides a way to "get close to the data". The researcher must penetrate the world of the actors and be able to see it from their point of view and use their categories because the actor's behavior is based on subjective, particular meanings. Participant observation sees self and society in processual terms. It seeks to provide an analytic description of a complex social organization (see McCall and Simmons, 1979). A participant observer may or may not play an active part in events, or she/he may interview participants in events which may be considered part of the process of observation. The variation in the researcher's level of activity may bewilder knowledge engineers. Knowledge abstraction in KE usually is based on protocol analysis (see Ericsson and Simon, 1984), and hence on passive observation of people in action. We suggest that this difference in outlook constitutes an invitation to mutual expansion and experimentation rather than to polarization. We note convergence in the views of KE and QM on the researcher's domain expertise. For example, both KE and QM researchers may see it as a benefit if the researcher is a domain novice, a status that may help the observer notice taken-for-granted aspects of human action and interaction. Similar to KE, norm or rule violations constitute an integral knowledge acquisition part in both symbolic interactionism and ethnomethodology. Goffman's (1959) dramaturgical approach within symbolic interactionism is based on the view that when human beings interact they try to manage the impression the others receive of them. People do not reflect over the norms that regulate social conduct, they are taken for granted. Therefore researchers may create situations where norms are violated to show that they exist and how they are maintained. S. J. Smith (1988) provides a recent example of the dramaturgical framework in action (although one where

852

C.G. LU.~DSERGand G. HOLM

rule and norm violations do not figure prominently) in a study of the social reality linking race and crime in Birmingham, England. She discusses strategies for gaining "an insider's access to local life" (p. 25), the analogy of depicting social life as drama, and problems related to determining or creating meaning. Smith suggests that the interpretation of social behavior--the interpretative researcher's findings-are dependent on which "plot" or "script" is adopted as a starting point. One plot abstractly links race relations issues with a moral panic over law and order, providing a collection of ideas around which the script appropriate to a particular event or locality is constructed. A script, for example, may relate to the management of danger and uncertainty (S. J. Smith, 1988). Smith's work, along with that of Hare (1985) and Goffman (1974), provide a direct link to AI and KE through concepts such as script, theme, and frame discussed extensively by cognitive science researchers like Minsky (1975), Schank and Abelson (1977), and as applied in AI modeling (Winston and Horn, 1981). Methods of actively interrupting the social conduct of interactants under study are prominent in ethnomethodological studies. Garfinkel's (1967) work represents a branch of this methodology based on attempts to penetrate normal situations of interaction to uncover the taken for granted rules. Examples of Garfinkel's experiments include situations where students were asked to act as borders in their own homes, or to bargain for fixed-value items in a store. In these situations the people in the "actor's" environment have not been able to interpret the situation as a game, an experiment, a deception, or a play because of the everyday nature of the situation, and characteristically they are given no help in constructing a new definition for the situation. Not surprisingly, studies of this type have been criticized as the sociology of instigation (Meltzer, Petras, and Reynolds, 1975). In the method's defense, it may expose taken-for-granted human communication and behavior, and thus provide valuable insights into the basic features of everyday interaction: into how meaning is constructed and social reality created. As an example, consider the potentially revealing situation that may arise if, instead of answering the ubiquitous "How are you?" with an echo or "Fine, thanks", we respond "What do you mean how am I?" (see Garfinkel, 1967). From the perspective of KE, the goal of using techniques based on abstraction and rule/norm violations in knowledge acquisition should be to determine the point where a minimal abstraction in the stimulus or task leads to a significant shift in the type of heuristics-at-use: to a switch from routine decision-making patterns to the use of controlled heuristics (Lundberg, 1989). Too much abstraction may reduce a problem solver's incentive, motivation, or ability to provide a realistic solution (Lundberg and Barna, 1987).

We view the difference in usage of abstraction less as a theoretical limitation than as a possibility to expand the domain for KE. The QM-approach can show the way to modeling situations characterized by interpersonal interaction or rich contexts. For example, the application of QMs may provide insights into the mechanisms that produce a system with high annual interest rates for poor loan-takers in certain developing nations (if interests are collected daily the loan-taker "can afford" interest rates that a richer loan-taker would never consider), nomadic behavior, decisions about the size of the cattle herd or crop shifts, on-site production decisions, decisions about foreign investments or plant location, or the processes of bargaining and bribing. In addition to being rich in formal and informal interaction, many of the given examples are set in socioeconomic or geographically defined subcultures. For example, a group of experts may be seen as a subculture. The way we model any expert is influenced by paradigms, beliefs, meanings, and values that the expert shares with her or his reference group. Modeling a floor trader on the Chicago Mercantile Exchange may not be so different from attempts to understand the behavior of members of other subcultures. Not understanding the context is likely to lead to failure to capture the individual trader's problem-solving process. Similarly, viewing domain experts as a subculture may increase the researcher's understanding of why subjects may not be willing to verbalize their knowledge. This is obvious if the skills are practiced within illegal or morally shady domains, but less intuitive if the expert has organizational or economic reasons to withhold her/his knowledge or desires to protect the mystique of expertise. For example, it is not unusual to hear experts (poets, cartographers, financial advisors) claim that their expertise is an art and beyond both analysis and modeling. Finally, Lundberg (1989) discusses a knowledge acquisition procedure that focuses on (actual or merely suggested) changes in taken-for-granted environments. For example, a direct attempt to ask a master cartographer how she/he decides where to place symbols may result in a response like "1 just know how to do it", or "it is simply usual practise"; responses of little value to a knowledge engineer. If instead the knowledge engineer suggests a change to an already produced pattern on the map, it is likely that the cartographer would be able to give detailed descriptions of why the new alternative would be less satisfactory. Hence, the extraction procedure utilizes an indirect method for distilling the rules and principles of a situation/phenomenon by exploring why its alternatives are less appealing to the subject. We suggest that this minimal-abstraction method may be of interest also to QM researchers. The method is related closely to the practice in KE of comparing the prototype model's solution to a problem to that of an expert, focusing on situations where the two solutions

constitute

models

with

of

Features

knowledge

General

the

fundamental

domain

on

Interest

Focus

Focus

stabibase

< .......

persons

ill-structured

....... >

< .......

and

change

...............

some

view

...............

Emphasis on problem and model postulation and theory construction

domains

................

Interpersonal interaction, sub-cultures, groups, structure, power-relationships

settings

Allows

Holistlc

...................

..................

..................

KA

Use and

based

of frames, scripts, schemata in m o d e l i n g

Use of techniques on abstraction

Use of limited information tasks

< ....... --->

<...

< .......

....... >

<-..

< .......

to d a t a

Acquisition

Closeness

Knowledge

Norm

and

violations

....... > < .......

< .......

....... >

....... >

rule

OM

rich

description

................

in r e s e a r c h e r ' s level

Use of frames, sclpts. p l o t s in d a t a c o l l e c t i o n

Techniques and methods based on Ethnomethodology and Symbolic interactlonism (e.g. d r a m a t u r gical approach)

..........................

initial

Variation activity

Virtue of nonstandardization

B r o a d b a s e f o r K A in psychology, sociology, and anthropology

Hypotheses-free data collection

Thick,

............................

(KA)

Similarity in v i e w o n r e s e a r c h e r ' s domain expertise: virtue of s t a r t i n g K A as n o v i c e

........................

................

Researcher often passive observer

Cognition-based

Hypotheses-free data collection

............................

ZZ

Figure 2. Summary of commonalities and potential benefits from alliance between KE and QM (arguments roughly in order they were discussed in text).

in r i c h

..................

OM

...................

substantive

in m i c r o

on process rather than rationality

specific

<---

....... >

....... >

....... >

<---

and

assumptions

specificity

Unearthing of covert behavioral cognitive processes

few

sensitivity,

Relatively

model

world

I n t e r e s t in m e a n i n g , commonsense, everyday llfe, u n d e r s t a n d i n g

Reasoning

Context

Simulation framework for m o d e l i n g and verification/valldatlon

Some m o d e l a n d problem postulation

...............

.................

...............

Focus on relative l i t y in k n o w l e d g e

Expandabillty

Data

Operational

...................

.................

................

....................

..................

KF.

854

C.G. LUNDBERGand G. HoLM

are different and the expert's justification for his or her decision and why the model's solution is inferior. CONCLUSIONS We have explored some of the common features of interpretative research and KE, focusing on the mutual benefits of a KE-QM alliance. We do not suggest that the processes are interchangeable, nor that they always are compatible, but rather that each, in some instances, can help the other expand and evolve. We pull together some key features of KE and QM in Figure 2. In this graph, we mark common features with a line spanning both the KE and the QM columns. Features prominent only in one framework are listed in the respective column, with the corresponding arrow pointing toward the other research framework to signal the applicability of the feature. The length of the arrow (long or short) signifies the relative importance of the message. In situations where there are significant qualitative differences in a feature shared by KE and QM, we list the feature in both columns. We do not claim that the classification is exhaustive nor beyond debate, but rather intended to summarize our discussion. We argue that the benefits from an alliance between the two methodologies are mutual; that qualitative methods can open previously closed research domains to knowledge engineering, and that knowledge engineering formalizations may provide a precise language for casting models derived through qualitative methods. We argue that knowledge engineers somewhat reluctantly are starting to use their techniques for theory construction, and that this process should be expanded. Researchers utilizing qualitative methods, in turn, discuss the data collection and analysis/theorizing stages of their work in detail, but may tend to leave off model specification before the operationalization phase (e.g. Bogdan and Taylor, 1975; Dobbert, 1982). This circumstance stems from the belief among QM researchers that it is impossible to replicate fully for example a participant observation study because the conditions constantly change. Yet, we claim that knowledge engineering can provide interpretative researchers with some suitable formalizations. An exploration of the research niche that the two methodologies collectively inhabit may help each prevent reinventing wheels, and may lead to further possibilities to distribute knowledge and expertise between people and groups in organizations and geographically. An alliance could lead to expansion of the knowledge bases by making some knowledge engineers more comfortable in their attempts to move toward modeling domains with prominent interaction between individuals or between individuals and structure. In one sense, many KE projects address the latter issue implicitly. Interpretative research could show the way toward making these issues more

explicit. Quantitative methods researchers may pay some attention to the insights that KE has collected about micro processes of a cognitive nature. Finally we argue that KE techniques could provide a simulation framework for qualitative and knowledge-based research. Simulation may be the most natural way to model processes based on qualitative data. These simulations may take the form of laboratory testing of processes that would be costly to implement without simulated experience, sensitivity analysis, or provide a laboratory for novices who wish to understand the modeled process. This also could provide a framework for a novel procedure for testing some qualitative hypotheses. Judging from the voluminous discussion of the reliability, validity, falsifiability, etc. of QM results, this could be a welcome contribution. REFERENCES Apple, M., and Weis, L., 1983, Ideology and practice in schooling: Temple Univ. Press, Philadelphia, 286 p. Banerjee, S., 1986, Reproduction of social structures, an artificial intelligence model: Jour. Conflict Resolution, v. 30, no. 2, p. 221-252. Barr, A., and Feigenbaum, E. A., eds., 1981, The handbook of artificial intelligence, v. i: William Kaufmann Inc., Los Altos, California, 409 p. Bogdan, R. C., and Biklen, S. K., 1982, Quantitative research for education: an introduction to theory and methods: Allyn & Bacon, Boston, 253 p. Bogdan, R. C., and Taylor, S. J., 1975, Introduction to qualitative research methods: a phenomenological approach to the social sciences:John Wiley& Sons, New York, 266 p. Broad, W. J., 1989, 'Smart' machines ready to assume many NASA duties: The New York Times, 6 March, p. I and I 1. Carbonell, J. G., 1979, The counterplanning process: a model of decision-making in adverse situations: Dept. Computer Science, Carnegie-Mellon University, Pittsburgh, 40 p. Carbonell, J. G., 1980, The politics project: subjective reasoning in a multi-actor planning domain: Computer Science Research Review, 1979-1980, Carnegie-Mellon University, Pittsburgh, 13 p. Chiu, C., 1989, Constructing qualitative domain maps from quantitative simulation models, in Widman, L. E., Loparo, K. A., and Nielsen, N. R., eds., Artificial intelligence, simulation, and modeling: John Wiley & Sons, New York, p. 275-299. Crain, R. L., 1977, Racial tensions in high school: pushing the survey method closer to reality: Anthropology and Education Quart., v. 8, no. 2, p. 142-154. Dean, J. P., Eichhorn, R. L., and Dean, L. R., 1969, Limitations and advantages of unstructured methods, in McCall, G. J., and Simmons, J. L., eds., Issues in participant observation: a text and reader: AddisonWesley, Reading, Massachusetts, p. 19-24. Dobbert, M. L., 1982, Ethnographic research: theory and application for modern schools and societies: Praeger, New York, 391 p. Duda, R. O., and Shortliffe, E. H., 1983, Expert systems research: Science, v. 220, no. 4594, p. 261-268. Ericsson, K. A., and Simon, H. A., 1984, Protocol analysis: verbal reports as data: The MIT Press, Cambridge, Massachusetts, 426 p. Farber, P., Lundberg, C. G., and Holm, G., 1987, Heuristic knowledge and the cultivation of practical judgment:

Integrating knowledge engineering and qualitative methods Dept. Education and Professional Development, Western Michigan Univ., Kalamazoo, Michigan, 41 p. Feigenbaum, E. A., 1977, The art of artificial intelligence: themes and case studies of knowledge engineering: IJCAI, v. 2, p. 1014-I029. Fersko-Weiss, H., 1985, Expert systems: decision making power: Personal Computing, v. 9, no. 11, p. 97-105. Fetterman, D. M., 1989, Ethnography. Step by step: Sage Publ., New York, 156 p. Freiling, M., Alexander, J., Messick, S., Rehfuss, S., and Schulman, S., 1985, Starting a knowledge engineering project: a step by step approach: AI Magazine, v. 6, no. 3, p. 150-163. Garfinkel, H., 1967, Studies in ethnomethodology: PrenticeHall, Englewood Cliffs, New Jersey, 288 p. Geertz, C., 1973, Thick description: toward an interpretive theory of culture, in Geertz, C., ed. The interpretation of cultures: selected essays: Basic Books, New York, p. 3-30. Giddens, A., 1984. The constitution of society--outline of the theory of structuration: Univ. California Press, Berkeley, California, 402 p. Glaser, R.. 1987, Thoughts on expertise, in Schooler C., and Schaie, W., eds., Cognitive functioning and social structure over the life course: Ablex Publ. Co.. Norwood, New Jersey, p. 81-94. Goffman, E., 1959, The presentation of self in everyday life: Doubleday. New York, 255 p. Goffman, E., 1974, Frame analysis: an essay on the organization of experience: Harper & Row, New York, 586 p. tlare, A. P., 1985, Social interaction as drama: Sage Publ., Beverly tlills, California, 183 p. Ilarmon, P., and King, D., 1985, Expert systems: John Wiley & Sons, New York, 283 p. Hayes-Roth, B., and Hayes-Roth, F., 1979, A cognitive model of planning: Cognitive Science, v. 3, no.4, p. 275-310. Itayes-Roth, F., 1984, The knowledge-based expert system: a tutorial: Computer, v. 17, no. 9, p. 11-28. tloffman, R. R., 1987, The problem of extracting the knowledge of experts from the perspective of experimental psychology: AI Magazine, v. 8, no. 2, p. 53-67. tlolm-Lundberg, G., 1986, Gender and vocational guidance in Finland: policy, practice and student perceptions: unpubl, doctoral dissertation, SUNY at Buffalo, Buffalo, 346 p. Jorgensen, D. L., 1989, Participant observation: a methodology for human studies: Sage Publ., New York, 133 p. Kuipers, B., 1984, Commonsense reasoning about causality: deriving behavior from structure: Artificial Intelligence, v. 24, no. I-3, p. 169-203. Kuipers, B., 1986, Qualitative simulation: Artificial Intelligence, v. 29, no. 3, p. 289-338. Kuipers, B., 1989, Qualitative reasoning with causal models in diagnosis of complex systems, in Widman, L. E., Loparo, K. A., and Nielsen, N. R., eds., Artificial intelligence, simulation, and modeling: John Wiley & Sons, New York, p. 257-274. LeCompte, M. D., and Goetz, J. P., 1982a, Problems of reliability and validity in ethnographic research: Review Educational Research, v. 52, no. I, p. 31-60. LeCompte, M. D., and Goetz, J. P., 1982b, Ethnographic data collection in evaluation research: Educational Evaluation and Policy Analysis, v. 4, no. 3, p. 387-400. Lenat, D., Prakash, M., and Shepherd. M., 1985. CYC: using common sense to overcome brittleness and knowledge acquisition bottlenecks: AI Magazine, v. 6, no. 4, p. 65-85. Lesgold, A. M., 1983, Acquiring expertise: Tech. Rept. No. PDS-5 (ONR), Learning Research and Development Center, Univ. Pittsburgh. Pittsburgh, 51 p.

Lundberg,C. G.,

855

1989, Knowledge acquisition and expertise evaluation: Professional Geographer, v. 41, no. 3, p. 272-283. Lundberg, C. G., and Bama, A. C., 1987, A knowledge based model of commodity trading expertise: The International Jour. Modeling and Simulation, v. 7, no. 4, p. 173-178. Lundberg, C. G., and Robinson, V. B., 1988, Computer programs that know: a tutorial: Computers, Environment and Urban Systems, v. 12, no. l, p. 49-71. McCall, G. J., and Simmons, J. L., 1969, The nature of participant observation, in McCall, G. J., and Simmons, J. L., eds., Issues in participant observation: a text and reader: Addison-Wesley, Reading, Massachusetts, p. 1-5. Meltzer, B. N., Petras, J. W., and Reynolds, L. T., 1975, Symbolic interactionism: genesis, varieties and criticism: Routledge & Kegan Paul, Boston, 144 p. Miles, M. B., and Huberman, A. M., 1988, Drawing valid meaning from qualitative data: toward a shared craft, in Fetterman, D. M., ed., Qualitative approaches to evaluation in education--the silent revolution: Praeger, New York, p. 222-244. Minsky, M., 1975, A framework for representing knowledge, in Winston, P. H., ed., The psychology of computer vision: McGraw-Hill Book Co., New York, p. 211-277. Mittal. S., and Dym, C. L., 1985, Knowledge acquisition from multiple experts: AI Magazine, v. 6, no. I, p. 32-36. O'Shea, T., Self, J., and Thomas, G., eds., 1987, Intelligent knowledge-based systems: an introduction: llarper & Row, London, 231 p. Philips, S. A., 1983, The invisible culture. Communication in classroom and community on the Warm Springs Indian Reservation: Longman, New York. 147 p. Pred, A., 1977, The choreography of existence: comments on H/igerstrand's time-geography and its usefulness: Economic Geography, v. 53, no. 2, p. 207-221. Pred, A., 1981, Social reproduction and the time-geography of everyday life: Geografiska Annaler, v. 63B, no. 1. p. 5-22. Ramos, R., 1979, Movidas: the methodological and theoretical relevance of interactional strategies, in Denzin, N. K., ed., Studies in symbolic interaction, v. 2: JAI Press, Greenwich, Connecticut, p. 141-165. Reichardt, C. F.. and Cook, T. D., 1979, Beyond qualitative versus quantitative methods, in Cook, T. D., and Reichardt, C. S., eds., Qualitative and quantitative methods in evaluation research: Sage Publ., Beverly Hills, California, p. 7-32. Rist, R. C., 1977, On the relations among educational research paradigms: from disdain to detente: Anthropology and Education Quart., v. 8, no. 2, p. 42-49. Schank, R. C., and Abelson, R. P., 1977, Scripts, plans. goals and understanding: Lawrence Erlbaum Assoc.. Hillsdale, New Jersey, 248 p. Simon, H. A., 1973, The structure of ill-structured problems: Artificial Intelligence, v. 4, no. 3-4, p. 181 -.201. Simon. H. A., 1986, Rationality in psychology and economics: Jour. Business, v. 59, no. 4, pt. 2, p. $209-$224. Smith, G. F., 1988, Towards a heuristic theory of problem structuring: Management Science, v. 34, no. 12, p. 1489-1506. Smith, S. J., 1988, Constructing local knowledge: the analysis of self in everyday life, m Eyles, J., and Smith, D. M., eds., Qualitative methods in human geography: Polity Press, Cambridge, p. 17-38. Sternberg, R. J., ed., 1985a, Human abilities: an information-processing approach: W. H. Freeman and Company, New York, 259 p. Sternberg, R. J., 1985b, Human intelligence: the model is the message: Science, v. 230, no. 4730, p. I I 1 I-I 118. Voss, J. F., Greene, T. R., Post, T. A., and Penner, B. C., 1983, Problem-solving skill in the social sciences, m

as6

c. G.

LDIDaERG and G. Hout

Bower, G. H., cd.. The psychology of learning and motivation. in Research Theory, v. Academic Press, York, p. Weis. L., Between two black students an urban college: Routledge Kegan Paul, 220 p. L. E., Loparo, K. 1989, Artificial gence, simulation, modeling: a survey, in L. E., K. A., Nielsen, N. eds.,

Artificial simulation. and John Wiley Sons, New p. I-44. P.. 1977. to labour: working class get working jobs: Columbia Press, New 226 p. P. H., Horn, B. P., 1981, AddisonWesley, Massachusetts, 430 Whytc, W. 1955, Street society: The Chicago Press, 366 p.