- Email: [email protected]
A theoretical basis for expert systems
Artificial Intelligence in Economics and Management L.F. Pau (Editor) © Elsevier Science Publishers B.V. (North-Holland), 1986
11
A THEORETICAL BASIS FOR EXPERT SYSTEMS S¥STEMS H. Keith Hall, James C. Moore and Andrew B. Whinston Krannert Graduate School of Management Purdue University West Lafayette, Indiana 47907 U.S.A.
In recent years, a great deal of research has been devoted to the expert system approach to computer-based problem solving. solving.ll Since there are so many different facets to the problem of developing expert systems (including the language for representing expert knowledge, the ability to provide users with an explanation system, the system's acquisition of expert knowledge, etc.) it is perhaps not too surprising that the decision-theoretic basis for expert system models, has received relatively little attention. In this paper we take a step toward remedying this deficiency def~ciency by extending the framework of classical decision theory to provide a theoretical basis for an "ideal" expert system. In view of the fact that there may be many difficulties associated with finding an optimal solution ("ideal expert system") in applications, we also consider the development of a "satisficing" solution which we believe has many advantages over traditional production-rule based approaches to expert systems. 1.
INTRODUCTION
Decision theory has typically characterized "best" decisions based upon the underlying assumption of a rational, utility maximizing decision-maker. He is seen to make inferences about an uncertain domain after obtaining information about critical parameters that affect the consequences of his decision. Inferences are based upon a set of beliefs about the likelihood of the environment, as embodied in a subjective probability distribution. The acquisition of infomation may be any activity such as experiments, tests, questions, etc., as long as it generally reduces uncertainty by altering these beliefs. If this framework is extended to incorporate the sequential gathering of (costly) infomation from different sources prior to making a final decision, then such a process may involve the formulation of an a priori strategy for collecting information and making a decision, as is necessary in an expert system. The formulation of what decision theory would describe as an optimal strategy in such a model would therefore correspond to the construction of an "ideal" expert system that computes only "best" solutions. In the next section we construct such a model; an extension of that presented in Marschak and Radner (1972). We have omitted all of the proofs in what follows, but they may be found in Moore, Hall, and Whinston (1985). 2.
AN "IDEAL" EXPERT SYSTEM
We suppose that a state of the environment can be specified by the values of ~£ unobservable parameters, 8e = (e1, (8 1 , ... ..• ,8£), ,e~), and m observable parameters, (z1, ..• z == (zl, •.• ,zm)' Thus the state space is given by ••• x Zm x 01 x ••• x X = Zl xx •••
0~
=Z x
0,
where we suppose each 8 i and each Zj is a finite set. 2
For example, in the case of
H K. Hall et al. H.
12
diagnosis. the value of 6i 8i might correspond to the presence (6i (8i := 1) I) or medical diagnosis, (81 := 0) a) of the i1h i~h disease, disease. while the values of the Zj Zj 's would represent absence (61 symptoms and personal characteristics of a patient. set. D D {dl, {dl •...• }. of of final decisions availWe suppose that there is a finite set, ... ,ddpp }' w(8.d). where able to the decision-maker; 33 who receives a gross payoff w(e,d), w: 8 8 xx DD +~ R. Thus the gross payoff is aa function of the unobservable parameters, parameters. (81." .•,6£), 8£). and the final decision taken, taken. while (as will be be specified in more e8 := (61,'" onl y to Z. InforInfordetail shortly) obtainable information is directly pertinent only Z can, can. however, however. be used to make inferences about 8 on the basis of of aa mation on Z function. ~: ~: Z xx 8 8 +~ [0,1], [a.I]. which defines aa probability known probability density function, measure by
=
~(Y): n(Y)
~E
~(z,6) ~(z.8)
Z x 8. for Y c Z
(z. 8)EY (z,6)EY A = {O,l, {a.I •...• n} be the set of initial initial actions (or experiments) available to Let A: ... ,n} the decision maker where aa := a represents no information collection. Associated A is a finite set of information signals, signals. Yaa ,• which we can write as: with each aa E A
°
{1.2 • ... . ..•,n(a)}; n(a)}; Yaa := {1,2, Ya.na: Z +~ Yaa . 44 We then define for and a single-valued function mapping Z onto Ya,na: a := O,l, a.I •...• 1.2 •...• n(a) a ... ,nn and yy 1,2, ... ,n(a) -I -1 M := {z {z E Zln (z) := y} M (y) na (y) ay a {Mal •...• Man(a)} 0.1 •...• n. ss We shall call any non-empty non-empty parand Ma {Mal,' .. ,Man(a)} for aa 0,1, ... ,n. Z. B B~ ~ Z, Z. an information structure on B. tition of a subset of Z, I. Let BC ZZ be non-empty, non-empty. and let a EE A. Definition 1. a. l(B,a), l(B.a). as: structure induced on by a,
B
We define the information
l(B. a) := {BnM {BIlMa1 1 • .BIlM a 2.··· • .BIlMan ()}\{r;lJL l(B,a) ,··· ,B/lMan(a)}\{(jJ}. a ,B/lM a2 a Bc ZZ be non-empty, non-empty. let B: B = {Bl," {BI." .,Bk} .• Bk} be an information Definition 2. Let BC B. and l~t let a: B B +~ AA (we shall refer to such aa function as an action structure on B, Q). The refinement of BB by a, a. R(B,a), R(B.a), is defined by: function ££ on ~). k R(B,a) := U U l[B.,a(B.)]. R(B,a) jj=1 :l JJ JJ Bc Z Z be non-empty, non-empty. and let Bl Definition 3. Let BC B2 be information structures BI and B2 B2 ), ), (or that Bl on B. We shall say that Bl B2 (or BI is aa refinement of B BI is as fine as B2 Hf: and write Bl BI ~> - BB2 , iff: c B". S". E B B22): ) : B' ~ (V B' E BI)(~ E B 1 )(:;:[ B" E Remark . If B B is non-empty subset of Z, B B is an information structure on B, and a Remark. B, then R(B,a) R(B,a) is an information structure on B B and aa is an action function on B, B. refinement of B. Assumption: The decision maker can take up to r information-gathering actions, A (and its it s where 1I << rr << n. Since we include the null information action in A associated cost will be assumed to be zero), we can, without loss of generality, assume that the decision maker takes exactly rr information-gathering actions. Definition 4.
acr satisfying:
A feasible strategy, a, cr, is aa sequence of r A ,a 1l ),(B ), (B 22 ,a .a22),· ),···, (Br,a ), (Brr ++1l ,8» «B 1I ,a .. ,(B r ,a rr ),(B
+ 1I pairs:
A Theoretical Basis for Expert Systems
1.
B1
{z} {Z}
2. Z.
a.
at: A for t == 1,2, ... .•. ,r. a : Bt ++A t t
b.
Bt + 1
3.
13
R(Bt,at ) for t = l,. ... ,.r.
0: Br+l 0: Br+l ++ D.
We shall denote the set of all feasible strategies for a particular decision problem V by "Z(V).,,6 A sequence of action functions can be seen to create, a priori, a sequence of partitions on Z, each a refinement of the previous partition in the sequence. The element of a given partition in the sequence in which the "true" state of the environment falls will determine the exact sequence of signals that the decision maker will receive from the sequence of action functions generating the partition. If a sequence of action functions, its associated sequence of partitions, and a final decision function defined on the last partition in the sequence are chosen a priori, then a feasible strategy is formed. The decision maker in effect generates the information structure from which he will make his final decision.
10 s:
2 zZ Example Let Z = {(zl,z2) {(zl'zZ) E R R210 Ma = = {Ma 1,M l,Ma 2} for a == 1,2,3, where
Let r where
< 2 for i = 1,2},7 A = {O,l,2,3}, {0,1,2,3}, and
MU Mll
I} M12 {z E zlo Zlo < z1 zl < 1}
{z E zI1 < zl z1 < 2}
M21
{z E zlo 1 } M22 < 1} Zlo < z2 <
{z E Zll ZI1 < z2 < 2}
M31
{z E Z IZ2 - zl {z z1 > O} M32
{z E Z a}. IZ2 - zl Z~2 z1 < o}.
2 and consider the feasible information-gathering strategies a and a * B1
B1*
{Z}, a 1 (Z)
B2
B2*
*
{M ll ,M 12 }, {MU L
*
a* 1 (Z)
a 2 (M (MU) 11 )
2 ,a2 (M 12 ) 2,a
3,
a* (M ll ) 2Z(MU)
*2 (M 12 ) 3,a 3,a/M
2.
1,
In decision-tree format, these two strategies can be represented as
a:
14
H K. Hall et al. li.K
a *:
((
Oc--=-a-=--+
Thuss the final information structures yielded Thu yi eld ed by the two strategies are B3
{M ll nM nM 2l ,M 12nM nM3l ,M I PM 32 }, {M Mll n M22 ,M 21 ,,M ll nM 31 ,M12nM32},
and
quite different, despitee the fac factt th that which the rreader eade r will note are quit e differ ent , despit at the two ac tions. strategies make use of the same set of initial actions. o
Definition 5. Let a er = «Bl' «B 1 ,al)'"'' (Br, a r ), (B r + l ,0» (( l) , ... ,(Br'((r),(Br+l,o» be a feasible strategy and aq(B) q E{2, ... ,r+l}. For each B EB q , we define aq( B) as the sequence (of length q-l) actions er along th the that Thus: of ac tions taken by the strategy a e path th at yyields ield s B. Thu s: wheree "a(t,B)" denotes de notes the action taken at step aq(B) ==
c[a(t,B) J. C(B) == t~l t ~lc[a(t,B)J.
(1)
Consequently the expected informati inf ormational s trategy a er is given by: onal cost of strategy
f(er) rea)
= =
L: L
1T(B)c(B). rr (B)c(B).
(2)
BEBr+l
er Given a feasible stragety, a "rl(er)," thatt is by "n(a) ," tha (er ) == L L: 0rl (a)
L: L
= =
( a ,B,o), thee eexpected (a,B, o), we denote th xpect ed ggross ros s payoff for er a
L: q,(z,8)w[8,o(B) ~ (z,8) w[8 , 0 (B)J. L J.
(3)
8EG BEB zEB 8EG We can now more formally state the goal of our decision problem, as fo follows. llows . Objective: have:
er * Choose a
=
=
* 0* )EL(V) ,Br+1, )EL:(V) such that for all aer (a * ,Br+l,o
rl(er *) - r(er r(er). n(a) rea*)) > 0rl(er) (a) - rea).
= <((,B,o> EL(V), EL: (V) ,
=
we (*)
15
A Theoretical Basis for Expert Systems Definition 6. If E non-empty. we define the potential gross ~ * B ~ Z is non-empty, associated with~, V(B), and the conditionally optimal decision set for !. ~, D (B), with!. V(B). ~L ~( z,8IB) w (e,d) V(B) = Max ~L ~ (z,8JB) (8.d) 8Ee dED zEB 8EG
D* (B) = = {d E DI DJ ~L ~(z.8JB)w(8,d) ~L ~(z,8IB)w(8,d) zEB 8Ee
VCB) }. v(B)
The following is then easily proved. Proposition 1. If 0a = «Bl,al), «B1,U1) ••..• (Br.ur).CBr+1.0» ... ,(B r ,ar ),(B r +l,O» E ~L (V) is optimal for V, L 1T(B) V(B). then for each B E B O(B) E D*(B). Furthermore, Furthermore. ~0 (o) (0) = ~ 1T(B)V(B). Br+l1 we have o(B) r+ BEBr+l We define the finest information structure obtainable from~, from!.
Definition 7. by:
n n-l n-l n-1 n-1 Ma~ 2.···' a= Maa~N onM { a= n IM n1 IM nnMn 2"'" n1 IM ,nMn,n (). () 1Ma~ aN"o ' a= a~ a , n-2 n nnN 0'···' ()}\{0}. MaN onMn-, Man () nQ1 IM ,nM 1 22n ,. ... , a= nQ1IM aaN aa= a } \ {<;Zl}.
We sshall hall say that a strategy Br+l == BA BA.. Lemma 1.
If
sA, sA.
0
0
«B1,U1) • ... . .• ,(B r ,a ,ur),(Br+1,0» = «Bl,al), r ),(B r +l,O»
= «Bl,al), ••. ,(Br,ar),(Br+l'O»
yields
is feasible, then
sA
~
sA
if
Br + 1 .
Because of Lemma 1 the following is essentially a special case of the theorem on p. 54 of Marschak and Radner (1972).
Proposition Propositi on 2.
Suppose a 0 = «Bl'&I), «B1,a1) •...• (Br.ar),(Br+1.~» Br+1 = = ... ,(Br,&r),(Br+l, ~ » is such that Br+l 0 = (B ,a ), ... ,(B ,a ),(Br+l'0» E ~ (V), (B1'u1)' . • .• (Br'ur).(Br+1.0» L r r 1 1
D* (i;lr+l) • Then for any ;S E D*(Br+l)'
we have
L
0 (0) < 0 (0)
~
sA
1T(B) V(B). 1T(B)V(B).
BEBr+1 BEBr+l While. by Proposition 2. payoff. it While, 2, a strategy of the form 0 maximizes the gross payoff, may not maximize the net payoff as in our objective equation (*) above. That is, the expected cost of such a strategy may be sufficiently higher than some alternative feasible strategy, 0*, 0*. as to more than offset the (possibly) higher expected gross payoff from Also. Also, since our decision problem is constrained to to have r < n information actions, there may be no feasible strategies which info rmation structure structur e given giv en A. Nonetheless, Nonethe les s . Proposition 2 result in the finest information whi ch will be used in Section 3 to develop a clas classs of "satisficing strategies" which approximate an optimal solution.
o.cr .
3.
APPLICATION APPL ICAT ION TO EXPERT SYSTEMS
There are three types of expert knowledge that must be acquired to implement our ideal expert system. First is the knowledge of the problem domain that is necesparameters. identify a set of possible decisary to identify a set of critical parameters, sions (problem solutions), and calculate the a priori likelihoods of each of the es of the environment. Second is the knowledge of the informationvarious stat states acquisition abilities of the system or system user which, which. combined with knowdomain. is necessary to identify the set of information-gathering ledge of the domain, actions and measurable parameters, and to calculate the conditional probabilities of the measurable parameters (z). (z), given the state of the environment (8). And lastly. knowledge of the desirability of the outcomes (and therefore the correctlastly, ness) of the possible final decisions is needed neede d to specify a payoff function to guide the decision process.
H K. Hall et al.
16
The ease with which the environmental parameters may be identified will vary with different problem domains. Generally, however, one would expect that expert expe rt knowledge that may easily be represented in production rules form will also have a fairly developed set of critical parameters already identified by expert sources. Knowledge of the system or system user's abilities to collect information on the decision problem under study, together with knowledge of the domain, allows the identification of the se sett of available information-gathering actions (A) and the rs (2 set of observable paramete parameters (Z = 21 Z1 x ..• •.• x2 xZm)' m). In most applications a practical method of constructing the density function, ~, ~ , would be to construct the marginal density function, f(8), g(zI8), and fee), develop the conditional probability function, g(zle), then obtain the joint density function, ~, by calculating ~(8,z) g(zI8)f(8) ~(e, z) = = g(zle)f(e)
(8,z)E e8 x 2. Z. for (e,z)E
Generally we wou ld expect that information on both the margi nal density function, would marginal f(8), and the conditional density function, g( g(zI8), fee), zl e) , would be available in published sources; however, in applications both functions would need to be checked in consultation with an expert. The last type of knowled knowledge ge needed to implement our model as an expert system is knowledge of the desirability of the outcomes of the different decisi decisions. ons. Although the identification of the set of critical and measurable measur ab le environmental parameters, set of possible decisions, and set of informational actions already involves some of this knowledge,B knowledge,S we require a much more precise measure of the function.. The importance of desirability of the outcomes to estimate a payoff function using knowledge of desirability of the possible outcomes of the decision to guide the solution process comes from the recognition that collecting col lec ting information is costly and this must be weighed against its potential benefit benefitss (in possibly making a better decision).9 However, this may be the most difficult knowledge to acquire, in the detail necessary to fully specify our model. For this reason, we close this section by discussing a class of strategies which do not require the specification of a payoff function, yet still produce good ssolutions. oluti ons. To obtain our satisficing strategy, we first (as discussed above), obtain a specification of the states of both the unobservable and the observable environZ; the probability density function, ~; the sset ment, 0e and 2; et of possible final decisions, decisions , D; the set of information-gathering actions, A; and the partition of 2Z and cost associated with action a, Ma and c(a), respectively. Next, using the partitions of 2, Z, Ma, for finest f or all a E A, the fine s t information structure obtainable calculated. expertt source the best from A, BA, must be calcu lated. lloo We then elicit from an exper decisions,, d E D for each element of BA; thus obtaining a decision strategy decisions (BA,8*), where 6*: BA ~ D. If this choice of decision is assumed to be guided by (BA,6*), maximizing the expected value of an underlying gross payoff function, then we have obtained an element of the conditional optimal decision set for each B E BA. That is, 6*(B) ED*(B) for all B E BA. 6*(B), grouped Third, those elements of sA with the same best decision, 6*(B ), may be group ed to gether get her to create the coarsest partition of 2Z possible that may result in the same expected gross payoff as BA. Formally, this partition, BD, is defined by BD = {Bdld {Bd ld E D*}. l l 6- (d) = U Band B(d) = {B E sAI6(B) where Bd BEB(d) function on BD may be defined such that 6D(B) = 6*(B)
d}.
Since a new decision
B
where E BA, B E BD, and BC B; by Proposition 2 above, any feasible strategy that yields BD (i.e., R(Br,a r ) ~ BD) maximizes the expected gross payoff. Lastly, we need to find those strategies, if any, that yield BD. Letting ~D LD C ~(V) L(V) denote this set of strategies, we then find that strategy, aD, such that
Theoretical Basis Basis for for Expert Expert Systems Systems AA Theoretical
(V 0a EE LD): LD): reo) rea) __ (V
LL
n(B)C(B) >> ~(B)C(B)
BEBr+l BEBr+l Since rr may may be be less less than than n, n, LD LD may may be be empty. empty. Since may be be done. do ne. may
--
LL
17 17
n(B)C(B) __ r(oD). reaD). ~(B)C(B)
D D BEBr+11 BEBr+
If this this is is the the case case the the following following If
First, follow follow the the above above steps steps for for rr == nn and and find find some some First, ll «Bi,at), .•• ... ,(~,a~)'(~+l'O+» ,(~, a~)'(~+l' O+» such s uc h that that 11 == «sr,at),
0+ 0+
*
(V 0a E E LD+): LD+): reo) rea):::~r(o rea *).). (V th n-IAt the the r-r!~ step step of of this this strategy, st rategy, for for each each BB EE 0r+l 81+1 one one of of the the following f ollowing two two At things are are true. true. a) a) B B~ ~ Bd Bd for for some some dd EE D*. D*. Since Since we we may may define define 8++(B) O++(B) == d, d, no no things further information information is is required r equired for for this thi s subset subset of of Z. Z. b) b) There There exists exists no no dd EE D* D* further for which which BC B ~ Bd' Bd. Here Here we we must must consult consult again again with with our our expert expert source source to to find find out out for whether, having having arrived arrived at at B, B, it it would would be be feasible feasible to to conduct conduct further further informationinformationwhether, ga therin g activities; activities; or or whether whether aa final final decision decision should should be be made made at a t that that point, point, gathering a nd if if so, so, which which one. one. and
Proceeding in in this this way, way, we we will will obtain obtain aa new new (generally (generally truncated) truncated) strategy strategy Proceeding ++ ++ ++ ++ ),(B++ ++ ++ ++++ ,0++ ++ ++ ++ ++ ++ a = «B ,a ++,a ++),(B » o =«B 11 ,a 11 ),(B ++, a ++),(B ++ ,8 » +1 rr rr rr +1 ++ n,B++ ++ = B++ , and a++ ++ a+ + ++ ++++ ,, aand ++ which will + will satisfy: satisfy: rr << rr = B Btt = a t for for which S n,B r++ = B nd B Btt , and a tt + r 11 rr 11 t 1, 2 , ... ... ,r. ,r. The The strategy strategy 0++ a++ will will be be "good" "good" in in the the sense sense that that (a) (a) it it is is concontt == 1,2, sistent with with the the expert's expert's preferences preferences (judgments) (judgments) and and (b) (b) it it should should come come close close to to sistent the goal goal of of maximizing maximizing expected expected gross gro ss payoff payo ff (in (in terms terms of of the the expert' expert's implicit the s imp licit gross payoff payoff function) function) at at minimum minimum expected expected cost. cost. gross
H K. Hall et al. H.
18
Footnotes
lI See , for example, Davis, Buchanan, and Shortliffe (1977), Davis, et al. (1981), McDermott and Steel (1981), and Holsapple, Shen, and Whinston (1982), respectively. Also, see Bonczek, Holsapple, and Whinston (1981) for an exploration of management applications of expert systems. 2We do not exclude the possibility that we may have 0. = Z. for some i and j. 1
J
3 In our medical example, the elements e l ements of D would correspond to possible treatments.
4Thus information is noiseless with respect to Z; that is, for each z 4Thus a single signal receivable from z if experiment a is performed. 5
For a == O,n(O) == 1, and MO
{Mal}' where Mal = Z.
E Z there is
Note that an information
action will generally not be noiseless with respect to 8 and that a set of signals, Ya is related to 8 by a conditional probability density function h : a
Y x 8 a a
+
[0,1] where
h (yle) (y [8) a
=
); zEM
4J(z,e) 4>(z,8)
for
eEG, and y E Ya . 8E0,
ay
6we 6We shall often find it convenient to regard a feasible strategy, 0= 0 = «B 1 ,a l , a l ), ... ,(B r ,a r ),(B r + 1l ,6» as being composed of two parts: i. ii.
the information-gathering strategy:
a
= «B 1l ,a 1l ), ... ,(B r
,a r
»,
the decision strategy, (B r + l1 ,,6). 6 ).
7Strictly speaking, of couse, this example is not legitimat legitimate, e , since Z is not finite. It is, however, very convenient. 8The importance of identifying these environmental parameters lies in their effect on the outcome of the decision and in their effect on the desirability of the outcome. Therefore, if expert sources hhave ave already identified the parameters, then they will have done so with the set of ppossible oss ible decisions, the potential outcomes o utcomes of the decisions, and the desirability of these outcomes at least implicitly in mind.
9 When a human expert goes through throu gh the process of collecting information he/she implicitly implicitl y makes ma kes judgments about the desirability of the possible outcomes. judgments These jud gmen ts weigh the likelihood of the existence of particular states of the environment by their potential effect on o n the goodness of the decision. 10Note lONote that by Definition 7 this may be achieved by calculating the final information structure of any non-sequential strategy using all a E A. 11 since I1 Note that this strategy always exists sin ce a simple non-sequential strategy involving all a E A will generate strategy in );D+. );D+ .
A Theoretical Basis [or for Expert Systems
19
References Bonczek, R. H., C. W. Holsapple and A. B. Whinston (1981). Decision Support Systems, Academic Press, 1981.
Foundations of
Buchanan, B. G. and R. O. Duda (1982). "Principles of Rule-Based Expert Systems," to appear in Advances in Computers, (M. Yovits, ed.), Vol. 22, August 1985. Davis, R., B. G. Buchanan, and E. H. Shortliffe (1977). "Production Rules as a Representation of a Knowledge-Based Consultant Program," Artificial Intelligence, 8, 15-45. Davis, R. and J. J. King (1984). "The Origin of Rule-Based Systems in AI," in Rule-Based Expert Systems, (G. B. Buchanan and E. H. Shortliffe, eds.), Addison-Wes~
Davis, R. et al., (1981). "The Dipmeter Advisor: Interpretation of Geological Signals," Proc. Seventh Int'l. Joint Conf. Artificial Intelligence, August. DeGroot, H. M. (1970).
Optimal Statistical Decisions, McGraw-Hill, Inc.
Gordon, J. and E. H. Shortliffe (1984). "The Dempster-Shafer Theory of Evidence," Gardon, Chapter 13 in Rule-Based Expert Systems, (B. B. Buchanan and E. H. Shortliffe, eds.), pp. 272-292. Holsapple, C. W., S. Shen and A. B. Whinston (1982). "A Consulting System for Data Base Design," International Journal Information Systems, Vol. 7, no. 3 March, 281-296. Ishizuka, M., K. S. Fu, and J. T. P. Yao (1982). "Inference Procedures Under Uncertainty for the Problem-Reduction Method," Information Sciences, 28, 179-206. Marschak, J. and R. Radner (1972). Harschak, Press.
Economic Theory
2i £f
Teams, Yale University
McDermott, J. and B. Steele (1981). "Extending a Knowledge-Based System to Deal Coni. Artificial with Ad Hoc Constraints," Proc. Seventh Int'!. rnt'!. Joint ConL Intelligence, 824-828. Moore, J. C., H. K. Hall, and A. B. Whinston (1985). Hoore, Expert Systems." Nau, D. S. s. (1983).
"Theoretical Foundations of
"Expert Computer Systems," IEEE, February, 63-85.
Shortliffe, E. H. and B. G. Buchanan (1975). "A Model of Inexact Reasoning in Shartliffe, Medicine," Mathematical Biosciences, 23, 351-379. Hedicine," Shortli£fe, Shortli~fe, E. H., R. Davis, S. G. Axline, B. G. Buchanan, C. C. Green and S. N. Cohen (1975). "Computer-Based Consultations in Clinical Therapeutics: Explanation and Rule Acquisition Capabilities of the MYCIN System," Computers and Biomedical Research, 8, 303-320. Suwa, M., A. C. Scott, and E. H. Shortliffe (1984). "Completeness and Consistency in a Rule-Based System," Chapter 8 in Rule-Based Expert Systems, (B. B. Buchanan and E. H. Shortliffe eds.), pp. 159-170.
11
A THEORETICAL BASIS FOR EXPERT SYSTEMS S¥STEMS H. Keith Hall, James C. Moore and Andrew B. Whinston Krannert Graduate School of Management Purdue University West Lafayette, Indiana 47907 U.S.A.
In recent years, a great deal of research has been devoted to the expert system approach to computer-based problem solving. solving.ll Since there are so many different facets to the problem of developing expert systems (including the language for representing expert knowledge, the ability to provide users with an explanation system, the system's acquisition of expert knowledge, etc.) it is perhaps not too surprising that the decision-theoretic basis for expert system models, has received relatively little attention. In this paper we take a step toward remedying this deficiency def~ciency by extending the framework of classical decision theory to provide a theoretical basis for an "ideal" expert system. In view of the fact that there may be many difficulties associated with finding an optimal solution ("ideal expert system") in applications, we also consider the development of a "satisficing" solution which we believe has many advantages over traditional production-rule based approaches to expert systems. 1.
INTRODUCTION
Decision theory has typically characterized "best" decisions based upon the underlying assumption of a rational, utility maximizing decision-maker. He is seen to make inferences about an uncertain domain after obtaining information about critical parameters that affect the consequences of his decision. Inferences are based upon a set of beliefs about the likelihood of the environment, as embodied in a subjective probability distribution. The acquisition of infomation may be any activity such as experiments, tests, questions, etc., as long as it generally reduces uncertainty by altering these beliefs. If this framework is extended to incorporate the sequential gathering of (costly) infomation from different sources prior to making a final decision, then such a process may involve the formulation of an a priori strategy for collecting information and making a decision, as is necessary in an expert system. The formulation of what decision theory would describe as an optimal strategy in such a model would therefore correspond to the construction of an "ideal" expert system that computes only "best" solutions. In the next section we construct such a model; an extension of that presented in Marschak and Radner (1972). We have omitted all of the proofs in what follows, but they may be found in Moore, Hall, and Whinston (1985). 2.
AN "IDEAL" EXPERT SYSTEM
We suppose that a state of the environment can be specified by the values of ~£ unobservable parameters, 8e = (e1, (8 1 , ... ..• ,8£), ,e~), and m observable parameters, (z1, ..• z == (zl, •.• ,zm)' Thus the state space is given by ••• x Zm x 01 x ••• x X = Zl xx •••
0~
=Z x
0,
where we suppose each 8 i and each Zj is a finite set. 2
For example, in the case of
H K. Hall et al. H.
12
diagnosis. the value of 6i 8i might correspond to the presence (6i (8i := 1) I) or medical diagnosis, (81 := 0) a) of the i1h i~h disease, disease. while the values of the Zj Zj 's would represent absence (61 symptoms and personal characteristics of a patient. set. D D {dl, {dl •...• }. of of final decisions availWe suppose that there is a finite set, ... ,ddpp }' w(8.d). where able to the decision-maker; 33 who receives a gross payoff w(e,d), w: 8 8 xx DD +~ R. Thus the gross payoff is aa function of the unobservable parameters, parameters. (81." .•,6£), 8£). and the final decision taken, taken. while (as will be be specified in more e8 := (61,'" onl y to Z. InforInfordetail shortly) obtainable information is directly pertinent only Z can, can. however, however. be used to make inferences about 8 on the basis of of aa mation on Z function. ~: ~: Z xx 8 8 +~ [0,1], [a.I]. which defines aa probability known probability density function, measure by
=
~(Y): n(Y)
~E
~(z,6) ~(z.8)
Z x 8. for Y c Z
(z. 8)EY (z,6)EY A = {O,l, {a.I •...• n} be the set of initial initial actions (or experiments) available to Let A: ... ,n} the decision maker where aa := a represents no information collection. Associated A is a finite set of information signals, signals. Yaa ,• which we can write as: with each aa E A
°
{1.2 • ... . ..•,n(a)}; n(a)}; Yaa := {1,2, Ya.na: Z +~ Yaa . 44 We then define for and a single-valued function mapping Z onto Ya,na: a := O,l, a.I •...• 1.2 •...• n(a) a ... ,nn and yy 1,2, ... ,n(a) -I -1 M := {z {z E Zln (z) := y} M (y) na (y) ay a {Mal •...• Man(a)} 0.1 •...• n. ss We shall call any non-empty non-empty parand Ma {Mal,' .. ,Man(a)} for aa 0,1, ... ,n. Z. B B~ ~ Z, Z. an information structure on B. tition of a subset of Z, I. Let BC ZZ be non-empty, non-empty. and let a EE A. Definition 1. a. l(B,a), l(B.a). as: structure induced on by a,
B
We define the information
l(B. a) := {BnM {BIlMa1 1 • .BIlM a 2.··· • .BIlMan ()}\{r;lJL l(B,a) ,··· ,B/lMan(a)}\{(jJ}. a ,B/lM a2 a Bc ZZ be non-empty, non-empty. let B: B = {Bl," {BI." .,Bk} .• Bk} be an information Definition 2. Let BC B. and l~t let a: B B +~ AA (we shall refer to such aa function as an action structure on B, Q). The refinement of BB by a, a. R(B,a), R(B.a), is defined by: function ££ on ~). k R(B,a) := U U l[B.,a(B.)]. R(B,a) jj=1 :l JJ JJ Bc Z Z be non-empty, non-empty. and let Bl Definition 3. Let BC B2 be information structures BI and B2 B2 ), ), (or that Bl on B. We shall say that Bl B2 (or BI is aa refinement of B BI is as fine as B2 Hf: and write Bl BI ~> - BB2 , iff: c B". S". E B B22): ) : B' ~ (V B' E BI)(~ E B 1 )(:;:[ B" E Remark . If B B is non-empty subset of Z, B B is an information structure on B, and a Remark. B, then R(B,a) R(B,a) is an information structure on B B and aa is an action function on B, B. refinement of B. Assumption: The decision maker can take up to r information-gathering actions, A (and its it s where 1I << rr << n. Since we include the null information action in A associated cost will be assumed to be zero), we can, without loss of generality, assume that the decision maker takes exactly rr information-gathering actions. Definition 4.
acr satisfying:
A feasible strategy, a, cr, is aa sequence of r A ,a 1l ),(B ), (B 22 ,a .a22),· ),···, (Br,a ), (Brr ++1l ,8» «B 1I ,a .. ,(B r ,a rr ),(B
+ 1I pairs:
A Theoretical Basis for Expert Systems
1.
B1
{z} {Z}
2. Z.
a.
at: A for t == 1,2, ... .•. ,r. a : Bt ++A t t
b.
Bt + 1
3.
13
R(Bt,at ) for t = l,. ... ,.r.
0: Br+l 0: Br+l ++ D.
We shall denote the set of all feasible strategies for a particular decision problem V by "Z(V).,,6 A sequence of action functions can be seen to create, a priori, a sequence of partitions on Z, each a refinement of the previous partition in the sequence. The element of a given partition in the sequence in which the "true" state of the environment falls will determine the exact sequence of signals that the decision maker will receive from the sequence of action functions generating the partition. If a sequence of action functions, its associated sequence of partitions, and a final decision function defined on the last partition in the sequence are chosen a priori, then a feasible strategy is formed. The decision maker in effect generates the information structure from which he will make his final decision.
10 s:
2 zZ Example Let Z = {(zl,z2) {(zl'zZ) E R R210 Ma = = {Ma 1,M l,Ma 2} for a == 1,2,3, where
Let r where
< 2 for i = 1,2},7 A = {O,l,2,3}, {0,1,2,3}, and
MU Mll
I} M12 {z E zlo Zlo < z1 zl < 1}
{z E zI1 < zl z1 < 2}
M21
{z E zlo 1 } M22 < 1} Zlo < z2 <
{z E Zll ZI1 < z2 < 2}
M31
{z E Z IZ2 - zl {z z1 > O} M32
{z E Z a}. IZ2 - zl Z~2 z1 < o}.
2 and consider the feasible information-gathering strategies a and a * B1
B1*
{Z}, a 1 (Z)
B2
B2*
*
{M ll ,M 12 }, {MU L
*
a* 1 (Z)
a 2 (M (MU) 11 )
2 ,a2 (M 12 ) 2,a
3,
a* (M ll ) 2Z(MU)
*2 (M 12 ) 3,a 3,a/M
2.
1,
In decision-tree format, these two strategies can be represented as
a:
14
H K. Hall et al. li.K
a *:
((
Oc--=-a-=--+
Thuss the final information structures yielded Thu yi eld ed by the two strategies are B3
{M ll nM nM 2l ,M 12nM nM3l ,M I PM 32 }, {M Mll n M22 ,M 21 ,,M ll nM 31 ,M12nM32},
and
quite different, despitee the fac factt th that which the rreader eade r will note are quit e differ ent , despit at the two ac tions. strategies make use of the same set of initial actions. o
Definition 5. Let a er = «Bl' «B 1 ,al)'"'' (Br, a r ), (B r + l ,0» (( l) , ... ,(Br'((r),(Br+l,o» be a feasible strategy and aq(B) q E{2, ... ,r+l}. For each B EB q , we define aq( B) as the sequence (of length q-l) actions er along th the that Thus: of ac tions taken by the strategy a e path th at yyields ield s B. Thu s: wheree "a(t,B)" denotes de notes the action taken at step aq(B) ==
c[a(t,B) J. C(B) == t~l t ~lc[a(t,B)J.
(1)
Consequently the expected informati inf ormational s trategy a er is given by: onal cost of strategy
f(er) rea)
= =
L: L
1T(B)c(B). rr (B)c(B).
(2)
BEBr+l
er Given a feasible stragety, a "rl(er)," thatt is by "n(a) ," tha (er ) == L L: 0rl (a)
L: L
= =
( a ,B,o), thee eexpected (a,B, o), we denote th xpect ed ggross ros s payoff for er a
L: q,(z,8)w[8,o(B) ~ (z,8) w[8 , 0 (B)J. L J.
(3)
8EG BEB zEB 8EG We can now more formally state the goal of our decision problem, as fo follows. llows . Objective: have:
er * Choose a
=
=
* 0* )EL(V) ,Br+1, )EL:(V) such that for all aer (a * ,Br+l,o
rl(er *) - r(er r(er). n(a) rea*)) > 0rl(er) (a) - rea).
= <((,B,o>
=
we (*)
15
A Theoretical Basis for Expert Systems Definition 6. If E non-empty. we define the potential gross ~ * B ~ Z is non-empty, associated with~, V(B), and the conditionally optimal decision set for !. ~, D (B), with!. V(B). ~L ~( z,8IB) w (e,d) V(B) = Max ~L ~ (z,8JB) (8.d) 8Ee dED zEB 8EG
D* (B) = = {d E DI DJ ~L ~(z.8JB)w(8,d) ~L ~(z,8IB)w(8,d) zEB 8Ee
VCB) }. v(B)
The following is then easily proved. Proposition 1. If 0a = «Bl,al), «B1,U1) ••..• (Br.ur).CBr+1.0» ... ,(B r ,ar ),(B r +l,O» E ~L (V) is optimal for V, L 1T(B) V(B). then for each B E B O(B) E D*(B). Furthermore, Furthermore. ~0 (o) (0) = ~ 1T(B)V(B). Br+l1 we have o(B) r+ BEBr+l We define the finest information structure obtainable from~, from!.
Definition 7. by:
n n-l n-l n-1 n-1 Ma~ 2.···' a= Maa~N onM { a= n IM n1 IM nnMn 2"'" n1 IM ,nMn,n (). () 1Ma~ aN"o ' a= a~ a , n-2 n nnN 0'···' ()}\{0}. MaN onMn-, Man () nQ1 IM ,nM 1 22n ,. ... , a= nQ1IM aaN aa= a } \ {<;Zl}.
We sshall hall say that a strategy Br+l == BA BA.. Lemma 1.
If
sA, sA.
0
0
«B1,U1) • ... . .• ,(B r ,a ,ur),(Br+1,0» = «Bl,al), r ),(B r +l,O»
= «Bl,al), ••. ,(Br,ar),(Br+l'O»
yields
is feasible, then
sA
~
sA
if
Br + 1 .
Because of Lemma 1 the following is essentially a special case of the theorem on p. 54 of Marschak and Radner (1972).
Proposition Propositi on 2.
Suppose a 0 = «Bl'&I), «B1,a1) •...• (Br.ar),(Br+1.~» Br+1 = = ... ,(Br,&r),(Br+l, ~ » is such that Br+l 0 = (B ,a ), ... ,(B ,a ),(Br+l'0» E ~ (V), (B1'u1)' . • .• (Br'ur).(Br+1.0» L r r 1 1
D* (i;lr+l) • Then for any ;S E D*(Br+l)'
we have
L
0 (0) < 0 (0)
~
sA
1T(B) V(B). 1T(B)V(B).
BEBr+1 BEBr+l While. by Proposition 2. payoff. it While, 2, a strategy of the form 0 maximizes the gross payoff, may not maximize the net payoff as in our objective equation (*) above. That is, the expected cost of such a strategy may be sufficiently higher than some alternative feasible strategy, 0*, 0*. as to more than offset the (possibly) higher expected gross payoff from Also. Also, since our decision problem is constrained to to have r < n information actions, there may be no feasible strategies which info rmation structure structur e given giv en A. Nonetheless, Nonethe les s . Proposition 2 result in the finest information whi ch will be used in Section 3 to develop a clas classs of "satisficing strategies" which approximate an optimal solution.
o.cr .
3.
APPLICATION APPL ICAT ION TO EXPERT SYSTEMS
There are three types of expert knowledge that must be acquired to implement our ideal expert system. First is the knowledge of the problem domain that is necesparameters. identify a set of possible decisary to identify a set of critical parameters, sions (problem solutions), and calculate the a priori likelihoods of each of the es of the environment. Second is the knowledge of the informationvarious stat states acquisition abilities of the system or system user which, which. combined with knowdomain. is necessary to identify the set of information-gathering ledge of the domain, actions and measurable parameters, and to calculate the conditional probabilities of the measurable parameters (z). (z), given the state of the environment (8). And lastly. knowledge of the desirability of the outcomes (and therefore the correctlastly, ness) of the possible final decisions is needed neede d to specify a payoff function to guide the decision process.
H K. Hall et al.
16
The ease with which the environmental parameters may be identified will vary with different problem domains. Generally, however, one would expect that expert expe rt knowledge that may easily be represented in production rules form will also have a fairly developed set of critical parameters already identified by expert sources. Knowledge of the system or system user's abilities to collect information on the decision problem under study, together with knowledge of the domain, allows the identification of the se sett of available information-gathering actions (A) and the rs (2 set of observable paramete parameters (Z = 21 Z1 x ..• •.• x2 xZm)' m). In most applications a practical method of constructing the density function, ~, ~ , would be to construct the marginal density function, f(8), g(zI8), and fee), develop the conditional probability function, g(zle), then obtain the joint density function, ~, by calculating ~(8,z) g(zI8)f(8) ~(e, z) = = g(zle)f(e)
(8,z)E e8 x 2. Z. for (e,z)E
Generally we wou ld expect that information on both the margi nal density function, would marginal f(8), and the conditional density function, g( g(zI8), fee), zl e) , would be available in published sources; however, in applications both functions would need to be checked in consultation with an expert. The last type of knowled knowledge ge needed to implement our model as an expert system is knowledge of the desirability of the outcomes of the different decisi decisions. ons. Although the identification of the set of critical and measurable measur ab le environmental parameters, set of possible decisions, and set of informational actions already involves some of this knowledge,B knowledge,S we require a much more precise measure of the function.. The importance of desirability of the outcomes to estimate a payoff function using knowledge of desirability of the possible outcomes of the decision to guide the solution process comes from the recognition that collecting col lec ting information is costly and this must be weighed against its potential benefit benefitss (in possibly making a better decision).9 However, this may be the most difficult knowledge to acquire, in the detail necessary to fully specify our model. For this reason, we close this section by discussing a class of strategies which do not require the specification of a payoff function, yet still produce good ssolutions. oluti ons. To obtain our satisficing strategy, we first (as discussed above), obtain a specification of the states of both the unobservable and the observable environZ; the probability density function, ~; the sset ment, 0e and 2; et of possible final decisions, decisions , D; the set of information-gathering actions, A; and the partition of 2Z and cost associated with action a, Ma and c(a), respectively. Next, using the partitions of 2, Z, Ma, for finest f or all a E A, the fine s t information structure obtainable calculated. expertt source the best from A, BA, must be calcu lated. lloo We then elicit from an exper decisions,, d E D for each element of BA; thus obtaining a decision strategy decisions (BA,8*), where 6*: BA ~ D. If this choice of decision is assumed to be guided by (BA,6*), maximizing the expected value of an underlying gross payoff function, then we have obtained an element of the conditional optimal decision set for each B E BA. That is, 6*(B) ED*(B) for all B E BA. 6*(B), grouped Third, those elements of sA with the same best decision, 6*(B ), may be group ed to gether get her to create the coarsest partition of 2Z possible that may result in the same expected gross payoff as BA. Formally, this partition, BD, is defined by BD = {Bdld {Bd ld E D*}. l l 6- (d) = U Band B(d) = {B E sAI6(B) where Bd BEB(d) function on BD may be defined such that 6D(B) = 6*(B)
d}.
Since a new decision
B
where E BA, B E BD, and BC B; by Proposition 2 above, any feasible strategy that yields BD (i.e., R(Br,a r ) ~ BD) maximizes the expected gross payoff. Lastly, we need to find those strategies, if any, that yield BD. Letting ~D LD C ~(V) L(V) denote this set of strategies, we then find that strategy, aD, such that
Theoretical Basis Basis for for Expert Expert Systems Systems AA Theoretical
(V 0a EE LD): LD): reo) rea) __ (V
LL
n(B)C(B) >> ~(B)C(B)
BEBr+l BEBr+l Since rr may may be be less less than than n, n, LD LD may may be be empty. empty. Since may be be done. do ne. may
--
LL
17 17
n(B)C(B) __ r(oD). reaD). ~(B)C(B)
D D BEBr+11 BEBr+
If this this is is the the case case the the following following If
First, follow follow the the above above steps steps for for rr == nn and and find find some some First, ll «Bi,at), .•• ... ,(~,a~)'(~+l'O+» ,(~, a~)'(~+l' O+» such s uc h that that 11 == «sr,at),
0+ 0+
*
(V 0a E E LD+): LD+): reo) rea):::~r(o rea *).). (V th n-IAt the the r-r!~ step step of of this this strategy, st rategy, for for each each BB EE 0r+l 81+1 one one of of the the following f ollowing two two At things are are true. true. a) a) B B~ ~ Bd Bd for for some some dd EE D*. D*. Since Since we we may may define define 8++(B) O++(B) == d, d, no no things further information information is is required r equired for for this thi s subset subset of of Z. Z. b) b) There There exists exists no no dd EE D* D* further for which which BC B ~ Bd' Bd. Here Here we we must must consult consult again again with with our our expert expert source source to to find find out out for whether, having having arrived arrived at at B, B, it it would would be be feasible feasible to to conduct conduct further further informationinformationwhether, ga therin g activities; activities; or or whether whether aa final final decision decision should should be be made made at a t that that point, point, gathering a nd if if so, so, which which one. one. and
Proceeding in in this this way, way, we we will will obtain obtain aa new new (generally (generally truncated) truncated) strategy strategy Proceeding ++ ++ ++ ++ ),(B++ ++ ++ ++++ ,0++ ++ ++ ++ ++ ++ a = «B ,a ++,a ++),(B » o =«B 11 ,a 11 ),(B ++, a ++),(B ++ ,8 » +1 rr rr rr +1 ++ n,B++ ++ = B++ , and a++ ++ a+ + ++ ++++ ,, aand ++ which will + will satisfy: satisfy: rr << rr = B Btt = a t for for which S n,B r++ = B nd B Btt , and a tt + r 11 rr 11 t 1, 2 , ... ... ,r. ,r. The The strategy strategy 0++ a++ will will be be "good" "good" in in the the sense sense that that (a) (a) it it is is concontt == 1,2, sistent with with the the expert's expert's preferences preferences (judgments) (judgments) and and (b) (b) it it should should come come close close to to sistent the goal goal of of maximizing maximizing expected expected gross gro ss payoff payo ff (in (in terms terms of of the the expert' expert's implicit the s imp licit gross payoff payoff function) function) at at minimum minimum expected expected cost. cost. gross
H K. Hall et al. H.
18
Footnotes
lI See , for example, Davis, Buchanan, and Shortliffe (1977), Davis, et al. (1981), McDermott and Steel (1981), and Holsapple, Shen, and Whinston (1982), respectively. Also, see Bonczek, Holsapple, and Whinston (1981) for an exploration of management applications of expert systems. 2We do not exclude the possibility that we may have 0. = Z. for some i and j. 1
J
3 In our medical example, the elements e l ements of D would correspond to possible treatments.
4Thus information is noiseless with respect to Z; that is, for each z 4Thus a single signal receivable from z if experiment a is performed. 5
For a == O,n(O) == 1, and MO
{Mal}' where Mal = Z.
E Z there is
Note that an information
action will generally not be noiseless with respect to 8 and that a set of signals, Ya is related to 8 by a conditional probability density function h : a
Y x 8 a a
+
[0,1] where
h (yle) (y [8) a
=
); zEM
4J(z,e) 4>(z,8)
for
eEG, and y E Ya . 8E0,
ay
6we 6We shall often find it convenient to regard a feasible strategy, 0= 0 = «B 1 ,a l , a l ), ... ,(B r ,a r ),(B r + 1l ,6» as being composed of two parts: i. ii.
the information-gathering strategy:
a
= «B 1l ,a 1l ), ... ,(B r
,a r
»,
the decision strategy, (B r + l1 ,,6). 6 ).
7Strictly speaking, of couse, this example is not legitimat legitimate, e , since Z is not finite. It is, however, very convenient. 8The importance of identifying these environmental parameters lies in their effect on the outcome of the decision and in their effect on the desirability of the outcome. Therefore, if expert sources hhave ave already identified the parameters, then they will have done so with the set of ppossible oss ible decisions, the potential outcomes o utcomes of the decisions, and the desirability of these outcomes at least implicitly in mind.
9 When a human expert goes through throu gh the process of collecting information he/she implicitly implicitl y makes ma kes judgments about the desirability of the possible outcomes. judgments These jud gmen ts weigh the likelihood of the existence of particular states of the environment by their potential effect on o n the goodness of the decision. 10Note lONote that by Definition 7 this may be achieved by calculating the final information structure of any non-sequential strategy using all a E A. 11 since I1 Note that this strategy always exists sin ce a simple non-sequential strategy involving all a E A will generate strategy in );D+. );D+ .
A Theoretical Basis [or for Expert Systems
19
References Bonczek, R. H., C. W. Holsapple and A. B. Whinston (1981). Decision Support Systems, Academic Press, 1981.
Foundations of
Buchanan, B. G. and R. O. Duda (1982). "Principles of Rule-Based Expert Systems," to appear in Advances in Computers, (M. Yovits, ed.), Vol. 22, August 1985. Davis, R., B. G. Buchanan, and E. H. Shortliffe (1977). "Production Rules as a Representation of a Knowledge-Based Consultant Program," Artificial Intelligence, 8, 15-45. Davis, R. and J. J. King (1984). "The Origin of Rule-Based Systems in AI," in Rule-Based Expert Systems, (G. B. Buchanan and E. H. Shortliffe, eds.), Addison-Wes~
Davis, R. et al., (1981). "The Dipmeter Advisor: Interpretation of Geological Signals," Proc. Seventh Int'l. Joint Conf. Artificial Intelligence, August. DeGroot, H. M. (1970).
Optimal Statistical Decisions, McGraw-Hill, Inc.
Gordon, J. and E. H. Shortliffe (1984). "The Dempster-Shafer Theory of Evidence," Gardon, Chapter 13 in Rule-Based Expert Systems, (B. B. Buchanan and E. H. Shortliffe, eds.), pp. 272-292. Holsapple, C. W., S. Shen and A. B. Whinston (1982). "A Consulting System for Data Base Design," International Journal Information Systems, Vol. 7, no. 3 March, 281-296. Ishizuka, M., K. S. Fu, and J. T. P. Yao (1982). "Inference Procedures Under Uncertainty for the Problem-Reduction Method," Information Sciences, 28, 179-206. Marschak, J. and R. Radner (1972). Harschak, Press.
Economic Theory
2i £f
Teams, Yale University
McDermott, J. and B. Steele (1981). "Extending a Knowledge-Based System to Deal Coni. Artificial with Ad Hoc Constraints," Proc. Seventh Int'!. rnt'!. Joint ConL Intelligence, 824-828. Moore, J. C., H. K. Hall, and A. B. Whinston (1985). Hoore, Expert Systems." Nau, D. S. s. (1983).
"Theoretical Foundations of
"Expert Computer Systems," IEEE, February, 63-85.
Shortliffe, E. H. and B. G. Buchanan (1975). "A Model of Inexact Reasoning in Shartliffe, Medicine," Mathematical Biosciences, 23, 351-379. Hedicine," Shortli£fe, Shortli~fe, E. H., R. Davis, S. G. Axline, B. G. Buchanan, C. C. Green and S. N. Cohen (1975). "Computer-Based Consultations in Clinical Therapeutics: Explanation and Rule Acquisition Capabilities of the MYCIN System," Computers and Biomedical Research, 8, 303-320. Suwa, M., A. C. Scott, and E. H. Shortliffe (1984). "Completeness and Consistency in a Rule-Based System," Chapter 8 in Rule-Based Expert Systems, (B. B. Buchanan and E. H. Shortliffe eds.), pp. 159-170.