- Email: [email protected]
The synergism of expert system and neural network technologies
Expert Systems With Applications, Vol. 2, pp. I-2, 1991 Printed in the USA.
0957-.-4174/91 $3.00 + .00 © 1991 Pergamon Press pie
EDITORIAL
The Synergism of Expert System and Neural Network Technologies LARRY R. MEDSKER The American University, Washington, DC T H E C U R R E N T DEVELOPMENTS i n a r t i f i c i a l n e u r a l
network technology present yet another opportunity to develop hybrid systems for effective real-world problem solving. Some advocates try to solve all problems with a new technology. A more reasonable and practical approach is to address complex problems by applying the most appropriate set of tools available. If barriers are overcome, artificial neural network (ANN) systems may represent half of the AI market by the end of this decade; however, certain symbolic as well as traditional numerical processing will still best be done by non-ANN systems or components. The multidisciplinary ANN and AI, each of which address human intelligence functions, has a natural synergism so that a fruitful approach is the integration of technologies for solving real problems. A goal of this special issue of Expert Systems with Applications: An International Journal is to encourage sharing of information and to foster discussions about the synergism of expert systems and neural network technologies. Expert systems and artificial neural networks represent complementary approaches: the logical, cognitive, and mechanical nature of expert systems approaches and the numeric, associative, self-organizing, and biological nature of neural networks. Thus, expert systems are especially good for closed systems for which inputs are literal and precise, leading to logical outputs. The value of ANN technology includes its usefulness for pattern recognition, learning, classification, generalization and abstraction, and the interpretation of incomplete and noisy inputs. A natural overlap with traditional AI applications is thus in the area of pattern recognition for character, speech, and visual recognition. Systems that learn are more natural interfaces to the real world than systems that must be programmed, and speed considerations point to the need to take advantage of parallel implementations.
Requests for reprints should be sent to Larry. R. Medsker, Department of Computer Science and Information Systems, The American University. Washington, DC 20016.
Attempts to combine these two technologies are just beginning to emerge. For example, Gallant (1988) has presented connectionist networks for use as expert systems, and Dietz, Kiech, and Ali (1989) have created a neural-network-based expert system for jet and rocket engine diagnostics. Other examples of applications have been developed by Yoon and Peterson (1988) and Li and Wee (1988). Recent results by Benachenhou, Cader, Szu et al. (1990) show promising results for hybrid approaches in which expert systems, for which complete sets of rules are only partially known, create rule-based inputs to neural networks. In that research, the neural network further reduces the solution space by forming clusters of possible solutions around those seeded from the expert system. Several areas involving expert systems and neural networks are thus interesting: • expert system elements implemented with neural networks; that is, use of neural networks in situations where expert systems have previously been used; • traditional areas of AI such as speech and visual recognition tasks that can be done as well or better with neural networks; • integrated intelligent systems--hybrid systems with expert system and neural network components; • novel applications that become possible because of the combination of technologies; • interface issues--improved user-machine interactions, as well as the use of expert systems to train neural networks; and • knowledge acquisition and knowledge engineering problems that can be addressed using neural network technology; for example, identification of implicit knowledge by neural networks to supplement explicit rule-based knowledge. Two articles in this issue address the integration of neural networks with various capabilities of expert systems and other decision-related computational systems. "Toward Conneetionist Production Systems" by Bhogal, Seviora, and Elmasry describes the connectionist architecture, CAPS, which translates production systems written in OPS5 into neural network implemen-
2
tations. Thus, fully trained neural networks can then perform as rule-based symbolic reasoning systems. This translation facility increases the feasibility of hardware implementation. The second article, "Integration of Adaptive Machine Learning and Knowledge-Based Systems for Routing and Scheduling Applications" by Kadaba, Nygard, and JueU describes XROUTE, an experimental, exploratory framework for combining neural networks with capabilities for mathematical modeling, knowledge-based systems, and genetic algorithms. They give an example of the application to the NP-complete vehicle routing problem. Pham and Degoulet describe their new approach in "Macro-connectionist Organization System for Artificial Intelligence Computation (MOSAIC)," which they have devised to integrate different kinds of cognitive functions. MOSAIC allows the management of complex, structured knowledge by means of numerical connectionist networks. Several inference strategies can be used to acquire knowledge explicitly or by fast, unsupervised learning. This work focuses on an intermediate neuronal level of functional organization to integrate expert system features, and the authors illustrate their approach with a medical expert system application. Two other articles describe the application of neural networks in traditional AI areas. "Rule-Based Training of Neural Networks" by Kwasny and Faisal presents the results of applying research and experience from natural language-processing to address issues of maintenance of rule-based systems. They use a connectionist deterministic parser system trained from rules and examples to accommodate changes in knowledge bases. "A Neural Network Based Learning System for Speech Processing" by Palakal and Zoran
L . R . Medsker
describes systems for recognizing patterns with significant variability by learning speech signal properties from speech images. Two articles present approaches to assisting neural network and expert system application developers. In "Expert Systems for Guiding Backpropagation Training of Layered Perceptrons," Steib and Weidman discuss the use of expert systems for training neural networks. "Knowledge Processing Using Fuzzy Cognitive Maps" by Taber presents a technique for modeling changes in dynamic, complex systems and for assisting knowledge engineers in capturing knowledge. This dynamic network approach to information processing allows alteration of causal connections via weight change laws during learning phases. These articles represent a range of areas in which the combination of artificial neural networks and expert systems may be useful. We hope that these examples will increase discussions about the uses of expert systems with artificial neural networks and generate new ideas for their synergism. REFERENCES Benachenhou, D., Cader, M., Szu. H., Medsker, L., Wittwer, C., & Garlin~ D. ( 1990, June). Neural networks for computing invariant clustering of a large open set of DNA-PCR primers generated by a feature-knowledge based system. Proceedings of the HCNNm 90, San Diego, CA. Dietz, W.E., Kiech, E.L., & All, M. (1989). Jet and rocket engine fault diagnosis in real time. Journal of Neural Network Computing,
I (I), 5-18. Gallant, S.L (1988). Connectionist expert systems. Communications of the ACM, 31(2), 152-169. Li, D. & Wee, W. (1988). Integration of neural network and expert system for pattern recognition. Neural Networks, l(sup. I), 32. Yoon, Y. & Peterson, L. (1988). DESKNET: The dermatology expert system with knowledge-based network. Neural Networks, l(sup. I), 477.
0957-.-4174/91 $3.00 + .00 © 1991 Pergamon Press pie
EDITORIAL
The Synergism of Expert System and Neural Network Technologies LARRY R. MEDSKER The American University, Washington, DC T H E C U R R E N T DEVELOPMENTS i n a r t i f i c i a l n e u r a l
network technology present yet another opportunity to develop hybrid systems for effective real-world problem solving. Some advocates try to solve all problems with a new technology. A more reasonable and practical approach is to address complex problems by applying the most appropriate set of tools available. If barriers are overcome, artificial neural network (ANN) systems may represent half of the AI market by the end of this decade; however, certain symbolic as well as traditional numerical processing will still best be done by non-ANN systems or components. The multidisciplinary ANN and AI, each of which address human intelligence functions, has a natural synergism so that a fruitful approach is the integration of technologies for solving real problems. A goal of this special issue of Expert Systems with Applications: An International Journal is to encourage sharing of information and to foster discussions about the synergism of expert systems and neural network technologies. Expert systems and artificial neural networks represent complementary approaches: the logical, cognitive, and mechanical nature of expert systems approaches and the numeric, associative, self-organizing, and biological nature of neural networks. Thus, expert systems are especially good for closed systems for which inputs are literal and precise, leading to logical outputs. The value of ANN technology includes its usefulness for pattern recognition, learning, classification, generalization and abstraction, and the interpretation of incomplete and noisy inputs. A natural overlap with traditional AI applications is thus in the area of pattern recognition for character, speech, and visual recognition. Systems that learn are more natural interfaces to the real world than systems that must be programmed, and speed considerations point to the need to take advantage of parallel implementations.
Requests for reprints should be sent to Larry. R. Medsker, Department of Computer Science and Information Systems, The American University. Washington, DC 20016.
Attempts to combine these two technologies are just beginning to emerge. For example, Gallant (1988) has presented connectionist networks for use as expert systems, and Dietz, Kiech, and Ali (1989) have created a neural-network-based expert system for jet and rocket engine diagnostics. Other examples of applications have been developed by Yoon and Peterson (1988) and Li and Wee (1988). Recent results by Benachenhou, Cader, Szu et al. (1990) show promising results for hybrid approaches in which expert systems, for which complete sets of rules are only partially known, create rule-based inputs to neural networks. In that research, the neural network further reduces the solution space by forming clusters of possible solutions around those seeded from the expert system. Several areas involving expert systems and neural networks are thus interesting: • expert system elements implemented with neural networks; that is, use of neural networks in situations where expert systems have previously been used; • traditional areas of AI such as speech and visual recognition tasks that can be done as well or better with neural networks; • integrated intelligent systems--hybrid systems with expert system and neural network components; • novel applications that become possible because of the combination of technologies; • interface issues--improved user-machine interactions, as well as the use of expert systems to train neural networks; and • knowledge acquisition and knowledge engineering problems that can be addressed using neural network technology; for example, identification of implicit knowledge by neural networks to supplement explicit rule-based knowledge. Two articles in this issue address the integration of neural networks with various capabilities of expert systems and other decision-related computational systems. "Toward Conneetionist Production Systems" by Bhogal, Seviora, and Elmasry describes the connectionist architecture, CAPS, which translates production systems written in OPS5 into neural network implemen-
2
tations. Thus, fully trained neural networks can then perform as rule-based symbolic reasoning systems. This translation facility increases the feasibility of hardware implementation. The second article, "Integration of Adaptive Machine Learning and Knowledge-Based Systems for Routing and Scheduling Applications" by Kadaba, Nygard, and JueU describes XROUTE, an experimental, exploratory framework for combining neural networks with capabilities for mathematical modeling, knowledge-based systems, and genetic algorithms. They give an example of the application to the NP-complete vehicle routing problem. Pham and Degoulet describe their new approach in "Macro-connectionist Organization System for Artificial Intelligence Computation (MOSAIC)," which they have devised to integrate different kinds of cognitive functions. MOSAIC allows the management of complex, structured knowledge by means of numerical connectionist networks. Several inference strategies can be used to acquire knowledge explicitly or by fast, unsupervised learning. This work focuses on an intermediate neuronal level of functional organization to integrate expert system features, and the authors illustrate their approach with a medical expert system application. Two other articles describe the application of neural networks in traditional AI areas. "Rule-Based Training of Neural Networks" by Kwasny and Faisal presents the results of applying research and experience from natural language-processing to address issues of maintenance of rule-based systems. They use a connectionist deterministic parser system trained from rules and examples to accommodate changes in knowledge bases. "A Neural Network Based Learning System for Speech Processing" by Palakal and Zoran
L . R . Medsker
describes systems for recognizing patterns with significant variability by learning speech signal properties from speech images. Two articles present approaches to assisting neural network and expert system application developers. In "Expert Systems for Guiding Backpropagation Training of Layered Perceptrons," Steib and Weidman discuss the use of expert systems for training neural networks. "Knowledge Processing Using Fuzzy Cognitive Maps" by Taber presents a technique for modeling changes in dynamic, complex systems and for assisting knowledge engineers in capturing knowledge. This dynamic network approach to information processing allows alteration of causal connections via weight change laws during learning phases. These articles represent a range of areas in which the combination of artificial neural networks and expert systems may be useful. We hope that these examples will increase discussions about the uses of expert systems with artificial neural networks and generate new ideas for their synergism. REFERENCES Benachenhou, D., Cader, M., Szu. H., Medsker, L., Wittwer, C., & Garlin~ D. ( 1990, June). Neural networks for computing invariant clustering of a large open set of DNA-PCR primers generated by a feature-knowledge based system. Proceedings of the HCNNm 90, San Diego, CA. Dietz, W.E., Kiech, E.L., & All, M. (1989). Jet and rocket engine fault diagnosis in real time. Journal of Neural Network Computing,
I (I), 5-18. Gallant, S.L (1988). Connectionist expert systems. Communications of the ACM, 31(2), 152-169. Li, D. & Wee, W. (1988). Integration of neural network and expert system for pattern recognition. Neural Networks, l(sup. I), 32. Yoon, Y. & Peterson, L. (1988). DESKNET: The dermatology expert system with knowledge-based network. Neural Networks, l(sup. I), 477.