A hybrid expert system for equipment failure analysis

Expert Systems with Applications 28 (2005) 615–622 www.elsevier.com/locate/eswa

A hybrid expert system for equipment failure analysis Hei-Chia Wanga,*, Huei-Sen Wangb a Institute of Information Management, National Cheng Kung University, 1st University Road, Tainan 701, Taiwan Department of Industrial Engineering and Management, Diwan Collage of Management, Madou, Tainan 721, Taiwan

b

Abstract This paper outlines the development of a web-based expert system, equipment failure analysis expert system (EFAES), for the largest steel company in Taiwan. The EFAES inference engine employs both case-based reasoning (CBR) and rule-based reasoning (RBR) to generate a hybrid recommendation list for cross validation. Moreover, this inference engine was designed to support a hierarchical multi-attribute structure. Unlike the traditional ‘flat’ attribute structure, this hierarchical multi-attribute structure allows experts to weigh the attributes dynamically. Two two-dimensional matrixes, multi-attribute analysis (MAA) and subattributes matrix (SAM), are used to store the attributes’ weight values. Normalized relative spending (NRS) is adapted to normalize the weight values for the inference engine. The system recommends both cases and rules, which can give more information in recognizing the failure types. According to our experimental results, applying our proposed method in an inference engine to analyze failure can result in better quality recommendations. q 2005 Elsevier Ltd. All rights reserved. Keywords: EFAES; Multi-attribute analysis; Case-based reasoning; Rule-based reasoning; Normalized relative spending

1. Introduction ‘Equipment Failure’ can be defined as ‘an equipment fails to perform up to an operation standard’. Most failures are minor and tolerable. However, some failures may lead to environmental pollution, catastrophic damage of properties and loss of human lives. Hence, the causes of the equipment failures have been considerably concerned. In order to work on the failure analysis, a corrosion-testing center was established in 1979 in the China Steel Cooperation, the largest steel corporation in Taiwan. From the experiences gained in the past 25 years in the area of failure analysis, much valuable knowledge has been built up by their domain specialists and kept in their minds. However, the analysis decision-making process has operated in the company for years and raises some problems. Most failure analysis is a very complex subject which involves many fields of study. It is not easy to be sorted out by a single expert without teamwork involving. However, * Corresponding author. Tel.: C886 6 2757575/53724; fax: C886 6 2362162. E-mail addresses: [email protected] (H.-C. Wang), huei@ mail.dwu.edu.tw (H.-S. Wang). 0957-4174/$ - see front matter q 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2004.12.042

the expert in the company may move to a different department within the company or even to a different company, or retire, and this means the expertise may be lost. Moreover, for a new employee to become an expert in failure analysis, it normally takes years of training followed by years of practical experience. Therefore, the China Steel Corporation considers using the modern information technology, expert system, to help them to deal with the above problems. Techniques used in constructing a diagnosis expert system include fault tree analysis (Ulerich & Powers, 1988), neural networks (Gaudier, Maida, Malstorm, & Sasken, 2002; Sorsa, Koivo, & Koivisto, 1991), neuralfuzzy (Ayoubi & Isermann, 1997), case-based reasoning (CBR; Bradley, 1994; Hsu, Chiu, & Hsu, 2004; Humphreys, McIvor, & Chan, 2003; Liao, Zhang, & Mount, 2000; Park & Han, 2002; Watson, 1999), and rule-based reasoning (RBR; Lau, IP, & Chan, 2002; Xia & Rao, 1999). In these methods, CBR is one of the most popular methods because each company can easily collect its own experiences as cases and store them into a case-based knowledge base. Large companies, especially have many cases, of failure or success. The basic principle of CBR is ‘similar problems lead to similar outcomes’ (Kwon, 2003). Besides CBR, an

616

H.-C. Wang, H.-S. Wang / Expert Systems with Applications 28 (2005) 615–622

we did not find any expert system shell that could support hierarchical structure (Fig. 1a) because most researchers use a flat-attribute structure for the inference engine. Notwithstanding these difficulties, we proposed an inference technology by two two-dimensional matrixes, multi-attribute analysis (MAA) and subattribute matrix (SAM), with NRS (Baragoin et al., 2001) to infer the situation mentioned above. The proposed system has the following objectives: Fig. 1. The difference between a hierarchical attribute and a flat attribute structure.

RBR system can be effective, because most rules are from cases that experts go through. Our proposed system involves use of RBR and CBR to construct a hybrid expert system for failure diagnosis because many researchers (e.g. Liao, 2004; Montani et al., 2003) suggest using a hybrid method for recommendations. For an expert system, the most import component is the inference engine. A number of issues were considered when designing the inference engine of EFAES. Failure is made onerous by the existence of many decision attributes, and these attributes have different importance to different failure types. Many researchers (Maximilien & Singh, 2002; Wang & Cercone, 2002) believe that different results should involve the application of different weights in inference. How to give different weights to different types of failure is one of the challenges. Another issue is that the attribute structure should not be ‘flat’ (Fig. 1b) because attributes learned from experts or a knowledge engineer could be highly dependent. However,

1. Store the experts’ knowledge for reusing in case-base and rule-base formats 2. Develop a framework of failure analysis that supports attribute relations constructed in a hierarchical structure that allows experts to assign weight dynamically 3. Apply the framework in the largest steel company in Taiwan to illustrate how a hybrid expert system can assist in the failure analysis process

2. Overview of the system The section presents the architecture of the proposed system, equipment failure analysis expert system (EFAES) (as shown in Fig. 2) and describes its workflow. The primary components involved in EFAES are the MAA matrix, subattributes matrix (SAM), knowledge engineer interface, knowledge base, inference engine, and user interface. SAM. Knowledge engineers and experts first define the structure of this matrix for collecting knowledge. Table 1 is an example of the hierarchical attributes matrix. This matrix

Fig. 2. The architecture of EFAES.

H.-C. Wang, H.-S. Wang / Expert Systems with Applications 28 (2005) 615–622 Table 1 The subattributes matrix Hierarchical attribute A

A1(Wa1)

B

A2(Wa2) A3(Wa3) B1(Wb1) B2(Wb2) C1(Wc1)

C

A11(Wa11) A12(Wa12) A21(Wa21)

Table 2 MAA matrix: failure types and attributes Attributes Xj (jZ1,.,n) Failure types Fi (iZ1,.,n)

X1

X2

.

Xn

F1

x11

x12

.

x1n

F2

x21

x22

.

x2n

«

«

«

Fm

xm1

xm2

« . xmn

617

is constructed as top-level attributes (e.g. A, B, C in Table 1) and their subattributes (e.g. A11, A12, A21, A3, B1, B2, C1 in Table 1). This matrix keeps the relations among attributes and weight values of subattributes. The weight values of top-level attributes for different types of failure are stored in the MAA matrix. MAA matrix. This matrix contains the failure types in columns and top-level attributes in rows. An MAA problem may be conveniently analyzed in matrix format (Hwang & Yoon, 1981). In this paper, we set failure types as Fi (iZ 1,.,n) and the attributes as Aj (jZ1,.,n). The score of similarity with Fi can be analyzed by the row vector {xi1,xi2,.,xim}. The column vector {x1j,x2j,.,xnj} represents the attribute Xj’s weight according to values of different failure statuses (see Table 2). In designing the MAA matrix, two works are assigned to the other participators, knowledge engineer and experts, in the proposed expert system. The knowledge engineers first define the criteria—the possible types of failure (Fi)

Fig. 3. An example of a knowledge engineer interface.

Fig. 4. Case-based knowledge base input interface.

618

H.-C. Wang, H.-S. Wang / Expert Systems with Applications 28 (2005) 615–622

Fig. 5. Rule-based knowledge base input interface.

and effect attributes (Xj). Experts then weigh the importance of each attribute (xij), respectively. Knowledge engineer interface. This interface will allow the knowledge engineer to edit the knowledge base and attribute matrixes. Fig. 3 is an example of a user interface

that allows the knowledge engineer to assign the source of cases or rules. Knowledge bases. The knowledge bases in this system are both case- and rule-based. The case-based knowledge base stores the cases from the literature, company, and network publication. Fig. 4 shows a case input interface. The rule-based knowledge base store the rules induced from the domain experts. Fig. 5 is a sample of a rule editor interface. User interface. This interface allows users to input their queries. Fig. 6 shows an example of a query user interface. At the conclusion of the query, rules and cases inferred from the inference engine are recommended for users. Inference engine. The inference engine is the key module in this system. When users input the information, the inference engine generates a list of possible failure types. Fig. 7 shows the output of a sample query.

Fig. 6. An example of a user interface.

Fig. 7. Example of a recommendation list.

H.-C. Wang, H.-S. Wang / Expert Systems with Applications 28 (2005) 615–622

619

Fig. 8. A sample inference recommendation.

The output could present completely and partially matched rules, and similar cases. Both partially matched rules and similar cases are ranked according to similarity scores. The details of similar cases and rules can be linked. Fig. 8 shows an example of the explanation of a failure case and its advice.

3. Inference mechanism The key work for a reasoning system is how to retrieve the best answers for users (Yonggang, Yinghong, & Xueyu, 2001). In the proposed expert system, the main work is to design a suitable inference engine to retrieve answers for the largest steel company in Taiwan. The key tasks underlying the inference engine are to: (1) minimize users’ input (knowledge engineers define all possible values for all attributes); (2) use the MAA matrix to find a suitable weight set; (3) support a hierarchical attributes matrix; (4) normalize the weights of target cases and rules; and (5) rank scores and top-level-scored failure cases and rules. When the system is queried, it checks both the case and rule knowledge bases for similarity assessment. The similarity assessment should be bases on the closest matched ranking by ‘similarity metrics’ (Pal & Palmer, 2000). 3.1. MAA matrix In the proposed system, knowledge engineers and experts concluded that different attributes should have different weight values to different types of failure. This argument was found in the other researches (Maximilien & Singh, 2002; Wang & Cercone, 2002) too. In order to

fit in with the requirement, we use an MAA matrix to store the relations among the top-level attributes and all possible types of failure. An MAA example is shown in Table 3. In this example, when the inference engine compares query conditions with case 1, the engine will use the weight {WA1, WB1, WC1} for the attributes {A, B, C}. If the inference engine compares the query with case 2, the engine will use the weight {WA2, WB2, WC2} for attributes {A, B, C}. 3.2. Hierarchical attributes definition The proposed system aims to be able to support a hierarchical attribute structure. In a hierarchical attribute set, top-level attributes could dynamically include as many related subattributes as the knowledge engineer needs. The hierarchical attribute can group related subattributes for easier maintenance. Moreover, unlike the traditional methods that use a flat attribute structure, the proposed method takes attribute-dependences into consideration. This method can solve the problem caused by the dependence between attributes. Table 4 continues the example in Table 3 and shows an example of a hierarchical attribute set where W is the weight value of each attribute. Table 3 An example of a MAA matrix Failure type Case 1: Failure type 1 Case 2: Failure type 2 Case 3: Failure type 3

Attribute A

B

C

WA1 WA2 WA3

WB1 WB2 WB3

WC1 WC2 WC3

620

H.-C. Wang, H.-S. Wang / Expert Systems with Applications 28 (2005) 615–622

Table 4 Example of a hierarchical attribute

to recommend the top-level items. The similarity is scaled by

Hierarchical attribute table A(WaZWAi) (iZ1,.,3)

A1(Wa1)

B(WbZWBi)

A2(Wa2) A3(Wa3) B1(Wb1) B2(Wb2) C1(Wc1)

C(WcZWCi)

A11(Wa11) A12(Wa12) A21(Wa21)

SimilarityðT; SÞ Z

n X

f ðTi ; Si Þwi

iZ1

where

In this example, top-level attribute A has three subattributes (dependent attributes) {A1, A2, A3}, attribute B has two subattributes {B1, B2}, and attribute C has one subattribute C1. The second-level attribute A1 has its own subattribute, A11 and A12, and so on. The weight value of each attribute or subattribute is presented as Wi, where iZ {a11, a12, a21, a3, b1, b2, c1} and perform a subattributes matrix (SAM). The top-level attribute values {WAi, WBi, WCi}, where iZ1,.,3, are from the MAA matrix with the case we compare.

wi is weight of attribute f(Ti,Si) is the similarity for an attribute between these two targets (Ti,Si) iZ1,.,n. The similarity function is applied to both rule- and casebased knowledge bases. After the comparisons are done, three output frames, which are completely matched rules, partially matched rules, and similar cases, are ranked and presented according to the similarity scores. (An example can be seen from Fig. 7.)

3.3. Normalization 4. Evaluations The work in the normalization step is to normalize the attribute weight values. We adapted the normalized relative spending process (Baragoin et al., 2001) to normalize the value of each level from top the bottom. The top-level weight values from the MAA matrix are assigned. The values used depend on the failure type the case is compared with. The subattribute weight is normalized from SAM. Table 5 extends the weight values from Table 4 to present an example after normalization.

The quality of an expert system rests on the quality of its recommendations (Yuan & Tsao, 2003). This section shows the quality of one recommendation of the proposed system. In our study, we use precision, recall, and F-measure to measure the effectiveness of a given recommendation approach. These three measures are widely accepted in information retrieval and recommended system research (Billsus & Pazzani, 1998; Sarwar, Karypis, Konstan, & Reidl, 2000).

3.4. Top-N score ranking 4.1. Data collection The inference engine aims to generate recommendations for users. Whatever methods are used, it requires a combination of searching and matching. This engine finds the potentially matching answers, and judges from the potential usefulness of the answers. The judgment is done by similarity analysis. In the proposed system, we adapt the k nearest neighbor (KNN) and scale the scores

In this evaluation, we collected cases and rules for two knowledge bases. A total of 63 rules are defined by experts for a rule-based knowledge base, and 200 cases from the Corrosion Atlas (During, 1997) and the China Steel Cooperation for the case-based knowledge based are used for this evaluation.

Table 5 An example of hierarchical multi-attributes after normalization Hierarchical attribute table AðWa0 Z Wa =ðWa C Wb C Wc ÞÞ

   P A1 Wa10 Z Wa 0  Wa1 = 3iZ1 Wai

BðWb0 Z Wb =ðWa C Wb C Wc ÞÞ

   P A2 Wa20 Z Wa 0  Wa2 = 3iZ1 Wai  0   P A3 Wa3 Z Wa 0  Wa3 = 3iZ1 Wai  0   P B1 Wb1 Z Wb 0  Wb1 = 2iZ1 Wbi  0   P B2 Wb2 Z Wb 0  Wb2 = 2iZ1 Wbi C1(Wc 0 1ZWc 0 )

CðWc0 Z Wc =ðWa C Wb C Wc ÞÞ

 0   P A11 Wa11 Z Wa 0  Wa11 = 2iZ1 Wa1i  0   P A12 Wa12 Z Wa 0  Wa12 = 2iZ1 Wa1i A21(Wa 0 21ZWa2 0 )

H.-C. Wang, H.-S. Wang / Expert Systems with Applications 28 (2005) 615–622

621

Table 6 Recall measurements from four different users

EFAES Flat structure with assigned weight Flat structure w/o assigned weight

Expert 1

Expert 2

Knowledge engineer 1

Knowledge engineer 2

Average

0.67 0.33

0.80 0.40

0.60 0.60

0.57 0.43

0.66 0.44

0.33

0.20

0.20

0.29

0.26

Expert 1

Expert 2

Knowledge engineer 1

Knowledge engineer 2

Average

0.2 0.1

0.4 0.2

0.3 0.3

0.4 0.3

0.33 0.23

0.1

0.1

0.1

0.2

0.13

Table 7 Precision measurements from four different users

EFAES Flat structure with assigned weight Flat structure w/o assigned weight

4.2. Comparisons We experimented with the following three approaches that use different weights and architecture. – MAA matrix weight for first level and SAM to normalize the weights for the rest of the levels – Flat weight structure and weights assigned by experts – Flat weights with the average weight values We set the threshold to 10 to check the first 10 recommendations for four different users (expert 1 and 2, and knowledge engineer 1 and 2). The input parameters are from evaluators’ experiences not in knowledge base. The results are shown in the tables below. Tables 6–8 show the recall, precision, and F-measure values of three different methods from four different users, respectively. The results show that our system, EFAES, can recommend better answers than the other two methods.

5. Conclusions and discussion In this paper, we present a web-based system, EFAES, designed for the largest steel company in Taiwan. Our system uses both RBR and CBR for material failure type diagnostics. We aimed to alleviate the attribute-dependence

problem with traditional flat attribute structure that might affect the inference result. An inference engine is designed to support the hierarchical attribute structure. This inference engine employed the MAA matrix for the first-level attributes and failure types, and the rest of the attributes are kept in SAM. These matrices allow experts to assign different weights for types of failure. We used NRS to normalize the weight values for inference. The EFAES inference method is compared with the two other methods and found that EFAES can achieve better recommendation quality. Although this EFAES system shows good evaluation results in terms of good quality of recommendations, some results are not adequate for experts. First, both cases and rules are not enough. Moreover, experts mentioned that some important attributes in the cases of the Corrosion Atlas are not shown. Furthermore, some tests recall values are not high enough. The first two problems take time to collect more cases and rules from other resources by knowledge engineers. The latest one could be solved when experts revise all cases and adjust the weight properly. This project is still ongoing. Regarding future work, we are going to not only collect more experts’ knowledge but also learn rules from the cases by data mining. Moreover, linguistic technologies, such as ontology, could be used to avoid term abuse during user queries.

Table 8 F-measure measurements from four different users

EFAES Flat structure with assigned weight Flat structure w/o assigned weight

Expert 1

Expert 2

Knowledge engineer 1

Knowledge engineer 2

Average

0.31 0.15

0.53 0.27

0.40 0.40

0.47 0.35

0.43 0.29

0.15

0.13

0.13

0.24

0.16

622

H.-C. Wang, H.-S. Wang / Expert Systems with Applications 28 (2005) 615–622

Acknowledgements This work was supported and funded by the China Steel Corporation. We thank the experts of China Steel Corporation for their invaluable assistance in assigning weights for all attributes, defining rules, and stimulating discussion on all aspects of this work.

References Ayoubi, M., & Isermann, R. (1997). Neuro-fuzzy system for diagnosis. Fuzzy Sets and Systems, 89, 289–307. Baragoin, C., Andersen, C. M., Bayerl, S., Bent, G., Lee, J., & Schommer, C. (2001). Mining your own business in retail. IBM redbooks pp. 54–55. Billsus, D., & Pazzani, M. J. (1998). Learning collaborative information filters. In Proceedings of the 15th International Conference on Machine Learning (pp. 46–54). Bradley, P. A. (1994). Case-based reasoning: business applications. Knowledge Engineering Systems, 37(3). During, E. D. (1997). Corrosion atlas. 3rd ed. New York: Elsevier. Gaudier, F., Maida, J., Malstrom, M., & Sasken, H. (2002). A neuralnetwork expert system in dermatopathology. American Journal of Clinical Pathology, 118(4), 631. Hsu, C. I., Chiu, C. C., & Hsu, P. L. (2004). Predicting information systems outsourcing success using a hierarchical design of case-based reasoning. Expert Systems with Application, 26(3), 435–441. Humphreys, P., McIvor, R., & Chan, F. (2003). Using case-based reasoning to evaluate supplier environmental management performance. Expert Systems with Application, 25(2), 141–153. Hwang, C.L., & Yoon, K. (1981). Multiple attribute decision making— Methods and applications. Berlin: Springer. Kwon, O. B. (2003). Meta web service: building web-based open decision support system based on web services. Expert Systems with Applications, 24(4), 375–389. Lau, H. C. W., IP, R. W. L., & Chan, F. T. S. (2002). An intelligent information infrastructure to support knowledge discovery. Expert Systems with Applications, 22(1), 1–10.

Liao, T. W. (2004). An investigation of a hybrid CBR method for failure mechanisms identification. Engineering Applications of Artificial Intelligence, 17(1), 123–134. Liao, T. W., Zhang, Z. M., & Mount, C. R. (2000). A case-based reasoning system for identifying failure mechanisms. Engineering Applications of Artificial Intelligence, 13, 199–213. Maximilien, E. M., & Singh, M. P. (2002). Conceptual model of web service reputation. ACM SIGMOD Record, 31(4), 36–41. Montani, S., Magni, P., Bellazzi, R., Larizza, C., Roudsari, A. V., & Carson, E. R. (2003). Integrating model-based decision support in a multi-modal reasoning system for managing type 1 diabetic patients. Artifical Intelligence in Medicine, 29(1–2), 131–151. Pal, K., & Palmer, O. (2000). A decision-support system for business acquisitions. Decision Support System, 27, 411–429. Park, C. S., & Han, I. (2002). A case-based reasoning with the feature weights derived by analytic hierarchy process for bankruptcy prediction. Expert Systems with Application, 23(3), 255–264. Sarwar, B., Karypis, G., Konstan, J., & Reidl, J. (2000). Analysis of recommendation algorithms for e-commerce. In Proceedings of the ACM Conference on Electronic Commerce (pp. 158–167). New York: ACM. Sorsa, T., Koivo, H. N., & Koivisto, H. (1991). Neural networks in process fault diagnosis. IEEE Transaction on Sysemt Man and Cybernetics, 21(4), 815–825. Ulerich, N. H., & Powers, G. J. (1988). On-line hazard aversion and fault diagnosis in chemical processes: the digraph and fault-tree method. IEEE Transaction on Reliability, 37, 171–177. Wang, Y. W., & Cercone, N. (2002). Fast searches in a recommendation session. Mathematical and Computer Modeling, 36(11–13), 1265–1274. Watson, I. (1999). Case-based reasoning is a methodology not a technology. Knowledge-Based Systems, 12, 303–308. Xia, Q., & Rao, M. (1999). Knowledge architecture and system design for intelligent operation support systems. Expert Systems with Applications, 17(2), 115–127. Yonggang, L., Yinghong, P., & Xueyu, R. (2001). Applying case-based reasoning to cold forging process planning. Journal of Materials processing Technology, 112, 12–16. Yuan, S. T., & Tsao, Y. W. (2003). A recommendation mechanism for contextualized mobile advertising. Expert Systems with Applications, 24(4), 399–414.