- Email: [email protected]
Judea pearl: Probabilistic reasoning in intelligent systems
Decision Support Systems 8 (1992) 73-75 North-Holland
73
Judea Pearl: Probabilistic Reasoning in Intelligent Systems Morgan Kaufrnann Publ. Inc., San Mateo, California, 1988,
Review
by T. V~tmos
Hungarian Academy of Sciences, Victor Hugo U. 18-22, 1 1 3 3 Budapest, Hungary
This is a fundamental book which should be read by everybody who is more or less actively engaged in knowledge-based systems. The most popular and commercial forms of applications are expert systems which mainly collect some rules of procedures and experience in a certain practical field, They can be very useful in those cases where the rules and conditions are well-defined, the problem complexity exceeds the limits of what is conceivable by men yet remains within the workable limits of computation. The problem starts really with the handling of uncertainty which is a longevous, possibly eternal conundrum of human thinking, our knowledge about the knowledge - epistemology, The most relevant controversy of this century's physics was the question whether uncertainty is a real way of existence in the Nature, or only the expression of our limited knowledge, if this limit is an eternal natural barrier for us or a limit of our present models, experimental devices. The same holds for any complex problem of life, biology, psycholoy, economy, social relations, By examining these problems in depth the different nature of uncertainties was identified, Classical probability deals with cases where we have a general model of the mechanism of the phenomena but the individual events are random, due to very small, individually not detectable interactions. Following the lessons of physics modern probability theory accepts the interpretation of the physical reality of uncertainty but the original model did not change to provide an answer to expectation about a great number of events based on a similar experience under the same circumstances.
A different kind of uncertainty is the lack of information on a certain event. The two are several times confused. Some phenomena are theoretically probabilistic, some others are phenomenologically the same as e.g. chaotic behavior due to nonlinearities. A different, somehow dual kind of uncertainty exists if the data on the event are certain but not their metric in a certain decision space. Data on uncertain phenomena can be different in nature, i.e. in classical probability they are obtained exclusively by reliable statistics but this reliability and obtainability is also questioned in most real-life problems. Therefore all kinds of other definite or indefinite resources are handled in a similar way as statistical data, e.g. subjective information, beliefs etc. Not only the data themselves, their acquisiton, their classification are uncertain but so is also the reasoning process itself, even if based on certain information. Exceptions, contradictions, paradoxes, lack of completeness, uncertainty on the relations, weaknesses of causal and teleological thinking, all add to this complex problem. The uncertainty of data and the uncertainty of logic are closely connected, mostly via semantic, i.e. the contextual value of information. All these problems were partly discovered by philosophers of past centuries but brought into the focus of research in mathematics, philosophy, psychology and philosophy only in the recent half century. This trend was accelerated by the advance of computer science and intelligent systems but no consolidation could be reached. The variety of different approaches, tenants of different methods are separated into sects, they prefer different meetings and periodicals, well demonstrating the views of Paul K. Feyerabend on the social phenomena of science, the dependence of accepted or refused thruth concerning the habits and fashions of the scientific society. A few people try to integrate this controversial
0167-9236/92/$05.00 © 1992 - Elsevier Science Publishers B.V. All rights reserved
74 but valuable mass of knowledge. The understanding of the different natures of uncertainty, the need for different treatments and the acknowledgement of the nonexistence of any final thruth is one condition for such an attempt. Another condition is a highly professional plethora of the treatment combined with a broad, well-digested knowledge of the existing efforts. These are the virtues of Judea Pearl's activity and its summation in this book. Two further preliminary comments are needed, The first refers to our remark on values in spite of apparent controversies. Most of these different approaches contributed in one way or another to the inexhaustible problem, helped the uncommitted observer see the different sides of the same secret, as art discloses the different layers of the human p s y c h e - remember the statement of Wittgenstein: "epistemology is the philosophy of psychology." And we have to add that uncertainty is the very nature of Uncertainty. The other comment refers to Pearl's own contribution and is related to his biases. These biases are most decent ones: in a successful attempt of providing a workable and theoretically firmly founded, partly new framework, he does not sweep aside the validity of other views and as far as possible tries to make understandable why they are also feasible and how they can be integrated or realistically compared with his preference. According to my knowledge no other textbook on the same problem comprised such a broad perspective with such a professional responsibility as Pearl did now. A further virtue of the book is its readability, As he.points out in his preface, the book can serve different people on different levels of background, It contains well-separable descriptive parts of epistemological explanations, easily understandable examples and well-founded mathematical treatments. The chapters build a progressive structure but most of them can be read separately especially by those who are somehow familiar with the subject and would like to obtain further information on some details. By marking chapters with three different types of stars the reader is gently directed among basic, technical, substantive and more advanced readings, The book starts with the basic epistemological problems of uncertainty, their relation to logic, language and different views on probability
(Chapter 1). The next chapter is devoted to special aspects of the Bayesian inference, this method is the basis of Pearl's all subsequent developments. Taking the Three Prisoners Paradox as a paradigm which is consequently treated through the whole book, he demonstrates the problem of information updating and consequence propagation, the difference between a real chance and a vague information (Chapter 2). This leads to a topological representation of consecutive events and information items, a structural description of informational relevance and causation. A thorough treatment of these graphoids is given, the difference from the ordinary graphs and an axiomatic foundation of graph isomorphisms which is an important component of structural modelling. The simpler Markovian networks are replaced by Bayesian ones which can represent induced and nontransitive or weakly transitive dependecies as well and thus can serve as computational models of a knowledge base containting different types of uncertainty (Chapter 3).The main contribution of Pearl and his group is detailed in the following part with a meticulous discussion of the essential computational problem of uncertainty propagation. We are looking for consequences of uncertain events, their parallel and serial interaction as it is done in medical systems estimating the effects of anamnestic data, symptoms, feelings of the patient, lab measurements, radiology pictures, the procedure of diagnosis, selection of treatment, feedback loops of further examinations, etc. Thus the representation of any kind of uncertainty is less dubious, from the practical point of view, i.e. reasoning and drawing consequences are treated within the context of the propagation issue. In this part several special tree structures are discussed, the difficulties of loops included and feasible applications for certain queries are developed (Chapter 4). Analogue to the propagation updating task is the revision of beliefs, interpretation of faults, finding second-best interpretations, choosing among more or less acceptable explanations (Chapter 5). These results lead us to an obvious application in decision support systems, the graph of a decision problem - taking into consideration chances, reliability and cost of information, risks - bears a structural similarity to Bayesian networks, and renders all propagation-updating-revision mecha-
75 nisms applicable. This hierarchical view of uncertainties enables the designer to follow the effects of confidence. The lucid treatment of goals, subgoals, short- and long-range strategies is an excellent presentation of the decision support system discipline's basic problems (Chapter 6). Statistics and probability work with distributions, probability densities and confidence intervals but most of the usual uncertainty methods in knowledge-based applications neglect these because of the further uncertainty of these estimates and first of all because of the additional computational complexity. The use of Bayesian networks can help intermediate solutions, in this chapter the author investigates the contextual, epistemic meanings of taxonomic hierarchies, confidence intervals, contingencies and related issues (Chapter 7). An exquisitely concluding chapter takes the dual of all the previous tasks as a natural idea: until this point the model was created first in a network form and then the computation used the graph model. The sequence can be reversed: computation on data can yield the network, the structure of the model as a guideline for learning relations from observation. Opposing the challenge of triumphing of this virtual omnipotence the author discusses the theoretical and practical difficulties of this reversion and the epistemological problems of causation or correlation, a problem which was first treated by David Hume, in the 18th century (Chapter 8)! The last part of the book is the most interesting one for people involved in recent problems of artificial intelligence. It discusses the pros and cons, similarities and deviations of different popular non-Bayesian or quasi-Bayesian uncertainty methods, as the Dempster-Shafer model, the Truth Maintenance Systems, the probabilistic logic of
Nilsson and relations to nonmonotonic logic as default logic, circumscription. We can enjoy a new scope of these methods, an insight into deep relations of uncertainty and logic, an attractive unification of those with many deep considerations on epistemic meaning. Pearl uses the customary paradigms of the Three Prisoners, Tweety the Penguin, the Yale Shooting Problem, the NixonDiamond but we receive a new insight, more general than usual and more critical in content even towards the author's recommendations (Chapter 9-10). What can be added to these virtues? The figures are clear and instructive, the short sections give much assistance in following the main lines. The bibliographical-historical remarks are knowledgeable and extremely correct. The only minor remark refers to those simple, conventional examples which are the merits of the book from the point of view of the clear presentation of the ideas and of the easy comparisons with the different other views using the same paradigms. These are instructive but not representative, i.e. they - being simple - do not indicate the basic computational problems of complexity. Pearl's earlier book on Heuristics was excellent just in this respect: it provided calculations and statistics on large-scale heuristic methods. We hope that this will be the next contribution, computational evaluation of the uncertainty approaches. Maybe some profound savants of one or other method or discipline can find errors, although the best people of the intellectual silicon valley helped the revision of the book as this very distinguished roll is listed on Page IX of the Preface - but the achievement will be lasting and stimulating. I conclude with the beginning of this review: it is a must to read!
73
Judea Pearl: Probabilistic Reasoning in Intelligent Systems Morgan Kaufrnann Publ. Inc., San Mateo, California, 1988,
Review
by T. V~tmos
Hungarian Academy of Sciences, Victor Hugo U. 18-22, 1 1 3 3 Budapest, Hungary
This is a fundamental book which should be read by everybody who is more or less actively engaged in knowledge-based systems. The most popular and commercial forms of applications are expert systems which mainly collect some rules of procedures and experience in a certain practical field, They can be very useful in those cases where the rules and conditions are well-defined, the problem complexity exceeds the limits of what is conceivable by men yet remains within the workable limits of computation. The problem starts really with the handling of uncertainty which is a longevous, possibly eternal conundrum of human thinking, our knowledge about the knowledge - epistemology, The most relevant controversy of this century's physics was the question whether uncertainty is a real way of existence in the Nature, or only the expression of our limited knowledge, if this limit is an eternal natural barrier for us or a limit of our present models, experimental devices. The same holds for any complex problem of life, biology, psycholoy, economy, social relations, By examining these problems in depth the different nature of uncertainties was identified, Classical probability deals with cases where we have a general model of the mechanism of the phenomena but the individual events are random, due to very small, individually not detectable interactions. Following the lessons of physics modern probability theory accepts the interpretation of the physical reality of uncertainty but the original model did not change to provide an answer to expectation about a great number of events based on a similar experience under the same circumstances.
A different kind of uncertainty is the lack of information on a certain event. The two are several times confused. Some phenomena are theoretically probabilistic, some others are phenomenologically the same as e.g. chaotic behavior due to nonlinearities. A different, somehow dual kind of uncertainty exists if the data on the event are certain but not their metric in a certain decision space. Data on uncertain phenomena can be different in nature, i.e. in classical probability they are obtained exclusively by reliable statistics but this reliability and obtainability is also questioned in most real-life problems. Therefore all kinds of other definite or indefinite resources are handled in a similar way as statistical data, e.g. subjective information, beliefs etc. Not only the data themselves, their acquisiton, their classification are uncertain but so is also the reasoning process itself, even if based on certain information. Exceptions, contradictions, paradoxes, lack of completeness, uncertainty on the relations, weaknesses of causal and teleological thinking, all add to this complex problem. The uncertainty of data and the uncertainty of logic are closely connected, mostly via semantic, i.e. the contextual value of information. All these problems were partly discovered by philosophers of past centuries but brought into the focus of research in mathematics, philosophy, psychology and philosophy only in the recent half century. This trend was accelerated by the advance of computer science and intelligent systems but no consolidation could be reached. The variety of different approaches, tenants of different methods are separated into sects, they prefer different meetings and periodicals, well demonstrating the views of Paul K. Feyerabend on the social phenomena of science, the dependence of accepted or refused thruth concerning the habits and fashions of the scientific society. A few people try to integrate this controversial
0167-9236/92/$05.00 © 1992 - Elsevier Science Publishers B.V. All rights reserved
74 but valuable mass of knowledge. The understanding of the different natures of uncertainty, the need for different treatments and the acknowledgement of the nonexistence of any final thruth is one condition for such an attempt. Another condition is a highly professional plethora of the treatment combined with a broad, well-digested knowledge of the existing efforts. These are the virtues of Judea Pearl's activity and its summation in this book. Two further preliminary comments are needed, The first refers to our remark on values in spite of apparent controversies. Most of these different approaches contributed in one way or another to the inexhaustible problem, helped the uncommitted observer see the different sides of the same secret, as art discloses the different layers of the human p s y c h e - remember the statement of Wittgenstein: "epistemology is the philosophy of psychology." And we have to add that uncertainty is the very nature of Uncertainty. The other comment refers to Pearl's own contribution and is related to his biases. These biases are most decent ones: in a successful attempt of providing a workable and theoretically firmly founded, partly new framework, he does not sweep aside the validity of other views and as far as possible tries to make understandable why they are also feasible and how they can be integrated or realistically compared with his preference. According to my knowledge no other textbook on the same problem comprised such a broad perspective with such a professional responsibility as Pearl did now. A further virtue of the book is its readability, As he.points out in his preface, the book can serve different people on different levels of background, It contains well-separable descriptive parts of epistemological explanations, easily understandable examples and well-founded mathematical treatments. The chapters build a progressive structure but most of them can be read separately especially by those who are somehow familiar with the subject and would like to obtain further information on some details. By marking chapters with three different types of stars the reader is gently directed among basic, technical, substantive and more advanced readings, The book starts with the basic epistemological problems of uncertainty, their relation to logic, language and different views on probability
(Chapter 1). The next chapter is devoted to special aspects of the Bayesian inference, this method is the basis of Pearl's all subsequent developments. Taking the Three Prisoners Paradox as a paradigm which is consequently treated through the whole book, he demonstrates the problem of information updating and consequence propagation, the difference between a real chance and a vague information (Chapter 2). This leads to a topological representation of consecutive events and information items, a structural description of informational relevance and causation. A thorough treatment of these graphoids is given, the difference from the ordinary graphs and an axiomatic foundation of graph isomorphisms which is an important component of structural modelling. The simpler Markovian networks are replaced by Bayesian ones which can represent induced and nontransitive or weakly transitive dependecies as well and thus can serve as computational models of a knowledge base containting different types of uncertainty (Chapter 3).The main contribution of Pearl and his group is detailed in the following part with a meticulous discussion of the essential computational problem of uncertainty propagation. We are looking for consequences of uncertain events, their parallel and serial interaction as it is done in medical systems estimating the effects of anamnestic data, symptoms, feelings of the patient, lab measurements, radiology pictures, the procedure of diagnosis, selection of treatment, feedback loops of further examinations, etc. Thus the representation of any kind of uncertainty is less dubious, from the practical point of view, i.e. reasoning and drawing consequences are treated within the context of the propagation issue. In this part several special tree structures are discussed, the difficulties of loops included and feasible applications for certain queries are developed (Chapter 4). Analogue to the propagation updating task is the revision of beliefs, interpretation of faults, finding second-best interpretations, choosing among more or less acceptable explanations (Chapter 5). These results lead us to an obvious application in decision support systems, the graph of a decision problem - taking into consideration chances, reliability and cost of information, risks - bears a structural similarity to Bayesian networks, and renders all propagation-updating-revision mecha-
75 nisms applicable. This hierarchical view of uncertainties enables the designer to follow the effects of confidence. The lucid treatment of goals, subgoals, short- and long-range strategies is an excellent presentation of the decision support system discipline's basic problems (Chapter 6). Statistics and probability work with distributions, probability densities and confidence intervals but most of the usual uncertainty methods in knowledge-based applications neglect these because of the further uncertainty of these estimates and first of all because of the additional computational complexity. The use of Bayesian networks can help intermediate solutions, in this chapter the author investigates the contextual, epistemic meanings of taxonomic hierarchies, confidence intervals, contingencies and related issues (Chapter 7). An exquisitely concluding chapter takes the dual of all the previous tasks as a natural idea: until this point the model was created first in a network form and then the computation used the graph model. The sequence can be reversed: computation on data can yield the network, the structure of the model as a guideline for learning relations from observation. Opposing the challenge of triumphing of this virtual omnipotence the author discusses the theoretical and practical difficulties of this reversion and the epistemological problems of causation or correlation, a problem which was first treated by David Hume, in the 18th century (Chapter 8)! The last part of the book is the most interesting one for people involved in recent problems of artificial intelligence. It discusses the pros and cons, similarities and deviations of different popular non-Bayesian or quasi-Bayesian uncertainty methods, as the Dempster-Shafer model, the Truth Maintenance Systems, the probabilistic logic of
Nilsson and relations to nonmonotonic logic as default logic, circumscription. We can enjoy a new scope of these methods, an insight into deep relations of uncertainty and logic, an attractive unification of those with many deep considerations on epistemic meaning. Pearl uses the customary paradigms of the Three Prisoners, Tweety the Penguin, the Yale Shooting Problem, the NixonDiamond but we receive a new insight, more general than usual and more critical in content even towards the author's recommendations (Chapter 9-10). What can be added to these virtues? The figures are clear and instructive, the short sections give much assistance in following the main lines. The bibliographical-historical remarks are knowledgeable and extremely correct. The only minor remark refers to those simple, conventional examples which are the merits of the book from the point of view of the clear presentation of the ideas and of the easy comparisons with the different other views using the same paradigms. These are instructive but not representative, i.e. they - being simple - do not indicate the basic computational problems of complexity. Pearl's earlier book on Heuristics was excellent just in this respect: it provided calculations and statistics on large-scale heuristic methods. We hope that this will be the next contribution, computational evaluation of the uncertainty approaches. Maybe some profound savants of one or other method or discipline can find errors, although the best people of the intellectual silicon valley helped the revision of the book as this very distinguished roll is listed on Page IX of the Preface - but the achievement will be lasting and stimulating. I conclude with the beginning of this review: it is a must to read!