Cell assemblies for diagnostic problem-solving

ARTICLE IN PRESS

Neurocomputing 69 (2006) 810–824 www.elsevier.com/locate/neucom

Cell assemblies for diagnostic problem-solving Andreas Wichert MITI Research Group, Technical University Munich, D-81675 Munich, Germany Received 12 March 2004; received in revised form 14 April 2005; accepted 15 April 2005 Available online 5 October 2005 Communicated by G. Palm

Abstract We describe a neuronal model for diagnostic problem-solving. This model which is inspired by cell assemblies gives some hints on how diagnostic problem-solving might actually be performed by the human brain. The diagnostic process is described by a deduction system that performs an abductive inference. The abductive inference itself is described by the verbal category theory. A mapping of a diagnostic problem into a diagnostic system represented by an associative memory with feedback connections is presented. The associative memory with feedback connections offers a self-contained architecture for the administration and representation of manifestations and disorders. This can be implemented efficiently on a serial computer, requiring low memory space and low computational costs. Because of these advantages, this model was chosen for the implementation of a real embedded diagnostic system for a wire bonder machine. The knowledge base of this system is composed of 350 rules, which are stored in 11 modules. These modules model the error behaviour of the microcontroller based units of the machine and are arranged in a taxonomy which corresponds to the hierarchical chains that describe the relationship between disorders and manifestations. r 2005 Elsevier B.V. All rights reserved. Keywords: Abductive inference; Associative memory; Cell assemblies; Expert system; Neural networks; Verbal category theory

1. Introduction The neural assembly theory was introduced by Hebb [9]. He proposed a connection between the structures found in the nervous system and those involved in high-level cognition such as diagnostic problem solving. An assembly of neurons acts as a closed system, and can therefore represent a complex object. Activation of some neurons of the assembly leads to the activation of the entire assembly, so that manipulations on the representation of a complex object are performed [10,23]. The pump of thoughts model [5,21,7] is a theoretical assembly model that explains how thoughts represented by assemblies can be propagated and changed by the brain. The process of human problem solving is described by this model as the transformation of thoughts through a sequence of assemblies [6,21]. Thoughts can be represented by verbal categories. Verbal categories are based on discrete features [29,17] and allow the representation of Corresponding author. Tel.: +49 4140 7389.

E-mail address: [email protected]. 0925-2312/$ - see front matter r 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.neucom.2005.04.008

disorders for diagnostic problem solving. Disorders can be described by a set of discrete features that correspond to errors, manifestations or symptoms. Problem solving by cell assemblies can be modelled by an associative memory with feedback connections [21]. During the diagnostic problem solving, the cause of the occurring manifestations is determined. The cause corresponds to a disorder that should be determined. Generally, a disorder (illness) causes a set of manifestations (errors, symptoms), and the latter affect a disorder. Furthermore, there can be hierarchical chains; a manifestation can also be a disorder that causes other manifestations. The relationship between the disorders and manifestations can be modelled with the aid of sound logical rules. We know that when a disorder is present, certain manifestations occur. For example, when the battery of a car is dead, then the motor does not start, the lights do not turn on and the radio does not work. This rule is sound, which means that when a disorder is present then the manifestations must also be present. This is evident because the manifestations are caused by the disorder. A diagnostic system, however, must deduce the disorder from the present

ARTICLE IN PRESS A. Wichert / Neurocomputing 69 (2006) 810–824

manifestations. This procedure corresponds to the abductive inference in which an hypothesis about the presence of a disorder is made [13,25]. A quality of this kind of rule of inference is that it is unsound, which means that the hypothesis is not necessarily true. The hypothesis about the disorder is the concluding part of the abductive rule; the premise assertions of the abductive rule are described by a combination of manifestations. A mapping of such a diagnostic problem into a diagnostic system represented by an associative memory with feedback connections is presented. The associative memory with feedback connections offers a self-contained architecture for the administration and representation of manifestations and disorders. The manifestations correspond to one layer and the disorders to another one, and the hierarchical chains of disorders and manifestations are represented by cycles. This model representing a diagnose system can be implemented efficiently on a serial computer, requiring low memory space and low computational costs. Because of these advantages, this model was chosen for the implementation of a real embedded diagnostic system for a wire bonder machine. 2. Inference model Diagnostic problems and the diagnose process can be described by diagnostic deduction systems which are a subgroup of production systems [32]. Production systems, as well as diagnostic deduction systems, consist of a set of rules also called the long-term memory and working memory [18]. In diagnostic deduction systems the premise of a rule specifies combinations of assertions, by which a new assertion of the conclusion is directly deduced in an abductive manner. This new assertion is added to the working memory. The working memory contains a description of the state in a diagnose process. This description is represented by manifestations and by the deduced disorders. The working memory, also called short-term memory, is initialized with the initial state description, which corresponds to the initially appearing manifestations before the diagnose process. Deduction systems do not need strategies for conflict resolution because every rule presumably produces reasonable assertions and there is no harm in firing all triggered rules. Deduction systems may chain rules together in a forward direction, from assertions to conclusions, or backward from hypotheses to premises. Backward chaining is used when an observer is asked about the presence of certain manifestations during the diagnose process. If the diagnose is performed on a set of present manifestations, as in our case, forward chaining is used to prevent wasting time pursuing hypotheses which are not specified by present manifestations. 2.1. Symbolic rules The knowledge which describes the relationship between the disorders and manifestations is represented logically in

811

the form A ) B ^ C ^ D or G ) F _ H _ I. The disorder A causes the manifestations B, C and D. The disorder G causes F or H or I manifestations. The rules are represented as





disordera ) manifestation1 ^ manifestation2 ^   ^ manifestationa , disorder0 ) manifestation1 _ manifestation2 _   _ manifestationn .

If disordera appears, then it causes manifestation1 and manifestation2 and .. and manifestationn . Disorder0 cause manifestation1 or manifestation2 or .. or manifestationn . During the abductive inference the manifestations specify the assertions by which the disorder is deduced. A knowledge base is a set of such rules. As an example we present a compendium of six rules concerning problems with the oil of a car diagnose system: 1. cable of the oil pressure lamp is loose ) oil lamp lights during driving round a bend _ oil lamp lights during braking; 2. oil lamp lights during driving round a bend ) driving round a bend ^ problems with oil; 3. oil lamp lights during braking ) braking ^ problems with oil; 4. oil pressure too low ) oil lamp goes out after some time ^ problems with oil ^ during idling; 5. oil level too low ) problems with oil ^ during idling; 6. problems with oil ) oil lamp lights up. For clarity we can replace the names of disorders and manifestations by symbols, each symbol representing a name: 1. 2. 3. 4. 5. 6.

A ) B _ C, B ) D ^ F, C ) E ^ F, G ) H ^ F ^ I, J ) F ^ I, F ) K.

A particular manifestation can also be a disorder that causes other manifestations. Because of this we introduce the following notation. A, B, C, D, E, F, G, H, I, J and K are called features. A, B, C, G, J and F, the features on the left side of the rules, form the set of disorders. The intersection of the set of disorders and manifestations forms the set of intermediate features: in our example B, C, F. The root disorders are represented as the set of disorders without the intermediate features: in our example A, G, J. The leaf manifestations are represented as the manifestations without the intermediate features: in our example D, E, H, I, K.

ARTICLE IN PRESS A. Wichert / Neurocomputing 69 (2006) 810–824

812

Expressing the logical relationship defined by these rules requires an extension to the basic graph model known as AND/OR graph. AND/OR graphs are an important tool for describing the search space generated by expert systems and logical theorem provers [16]. We can represent the set of rules by a directed AND/OR graph. In Fig. 1 we see the representation of the six rules. 2.2. Abductive inference Suppose the manifestations ‘oil lamp lights up’ and ‘braking are present’. How can we perform the inference? The compendium of six rules concerning problems with oil of a car represents the long-term memory. Working memory is initialized with the initial state descriptions ‘oil lamp lights up’ and ‘braking’, which correspond to the initially appearing manifestations before the diagnose process, represented in our symbol notation by K and E. The conclusion is now directly deduced in an abductive manner. The hypothesis about the disorder is the concluding part of the abductive rule; the premise assertions of the abductive rule are described by a combination of manifestations 1. hypothesis F is valid, because K is present, and K is caused by F, F ) K. The short-term memory is now =K; E; F =.

2. hypothesis C is valid, because F and E are present, and F and E are caused by C, C ) E ^ F . The short-term memory is now =K; E; F ; C=. 3. hypothesis A is valid, because C is present, and C is caused by A, A ) B _ C. The short-term memory is now =K; E; F ; C; A=. Answer: The root disorder A is most probably present; it means that the response ‘the cable of oil pressure lamp is loose’ is most probably present. 2.3. Verbal categories The abductive inference can be characterized by the verbal category theory. In this context a category corresponds to a disorder set. The set of manifestations that are caused by disorders is called the disorder set. A present manifestation set is judged as being caused by a disorder, to the extent that its elements are predicted by the disorder set [20]. The similarity of a disorder set Dis and of a manifestation set manif is given by the following formula, which is inspired by the contrast model of Tversky [29,26,19]: SimcðDis; manif Þ ¼

jDis \ manif j jðDis  ðDis \ manif ÞÞj  , jDisj jDisj

SimcðDis; manif Þ ¼

2  jDis \ manif j  1 2 ½1; 1, jDisj

jDisj is the number of manifestations in the disorder set Dis. The hypothesis about the disorder of an abductive ‘and’ rule is valid if SimcðDis; manif Þ ¼ 1, and of an ‘or’ rule if SimcðDis; manif Þ4  1.

A

2.4. Binary vector representation

B

C

D

E

G

H

J

F

I

K Fig. 1. Representation of six rules by a directed AND/OR graph. The ‘and’ rules are indicated by an arc between the links connecting the nodes indicating the manifestations.

Binary vectors can represent features. A ‘one’ represents a feature at the corresponding position of a binary vector; its absence is denoted by a ‘zero’. The feature set A, B, C, D, E, F, G, H, I, J, K is represented by a binary vector of dimension 11. The presence of features C and E is represented by the binary vector ½0 0 1 0 1 0 0 0 0 0 0. In binary vector notation the disorder set is represented by ~. the vector d~ and the manifestation set by the vector m Xn 2 ~m ~Þ ¼ Pn Simcðd;  d m  1. j¼1 j j j¼1 d j In the majority of cases most elements of the binary vector are zero. On a serial computer a pointer representation can save memory space. In the pointer format only the positions of the vector components unequal to zero are represented. The binary vector ½0 0 1 0 1 0 0 0 0 0 0 is represented as the pointer vector ð3 5Þ. 3. Lernmatrix and cell assemblies The Lernmatrix [27,28,11,15,12,22] is composed of a cluster of units. Each unit represents a simple model of a

ARTICLE IN PRESS A. Wichert / Neurocomputing 69 (2006) 810–824

real biological neuron. The Lernmatrix was introduced by Steinbuch, whose goal was to produce a network that could use a binary version of Hebb learning to form associations between pairs of binary vectors. Each unit is composed of weights, which correspond to the synapses in the real neuron. They are described by wij in Fig. 2. T, is the threshold of the unit. We call the Lernmatrix simply ‘associative memory’ if no confusion with other models is possible [2,3]. The biological and mathematical aspects of the Lernmatrix were studied by Palm [21,22,8,30]. It was shown that Donald Hebb’s hypothesis of cell assemblies as a biological model of internal representation of events and situations in the cerebral cortex corresponds to the formal Lernmatrix model. The patterns that are stored in the Lernmatrix are represented by binary vectors. The presence of a feature is indicated by a ‘one’ component of the vector, its absence through a ‘zero’ component of the vector. A pair of these vectors is associated and the process of storing these associations in the associative memory is called learning. The first vector ~ x is called the question vector and the second ~ y, the answer vector. After learning, the question vector is presented to the associative memory and the answer vector is determined. The associative memory represents the long-term memory of our diagnostic system in which the rules are stored. In the initialization phase of the associative memory, no information is stored. Because the information is represented in the weights, they are all initially set to zero. In the learning phase, binary vector pairs are associated. In the first vector ~ x we store the disorder set of manifestations which are caused by it; the disorder itself is indicated by the second vector ~ y. For example, the rule B ) D ^ F is represented in the vectors x ¼ ½0 0 0 1 0 1 0 0 0 0 0 and

x1 x2

xn

w11

w21

w31

wm1

w12

w22

w23

wm2

w1n

w2n

w3n

wmn

T

T

T

T

y1

y2

y3

ym

Fig. 2. The Lernmatrix is composed of a set of units which represent a simple model of a real biological neuron. The unit is composed of weights that correspond to the synapses and dendrites in the real neuron. They are described by wij in this figure. T is the threshold of the unit.

813

y ¼ ½0 1 0 0 0 0 0 0 0 0 0. The learning rule is wnew ¼1 ij

if yi  xj ¼ 1;

wnew ¼ wold ij ij

else:

In Fig. 3 we see the associative memory that is a part of the architecture of the associative diagnostic system after the learning of the six rules. The ‘or’ rules are indicated by a one in the threshold. For example, the ‘or’ rule A ) B _ C is represented by the first unit, and the rule B ) D ^ F is represented by the second unit (see Fig. 3). After learning, the disorders can be determined by the abductive inference with the aid of associative memory. The present manifestations represented by the question vector ~ x are presented to the associative memory and the disorders are identified by the retrieval rule which determines the answer vector ~ y with the aid of the following retrieval rule: yi ¼ mðzi Þ with zi ¼ Pn

2

j¼1 wij



n X

wij xj  1

j¼1

and

mðzi Þ ¼

(

8 > <1

if

> :

else:

0

zi 4  1 and T i ¼ 1; zi ¼ 1 and T i ¼ 0;

To deal with uncertain knowledge the retrieval rule and the representation has to be extended as described in [31]. The architecture of our diagnostic system is composed of associative memory with feedback connections. The shortterm memory, which is initialized with the initial state description, is represented by the row buffer on the left side of the associative memory. The manifestations of the shortterm memory are presented to the associative memory, which determines the disorders by using the retrieval rule to perform an abductive inference. The determined disorders are transported from the buffer below the units (column buffer) via the feedback connections to the shortterm memory. The short-term memory is updated and the procedure is repeated until the short-term memory does not change, i.e. the number of features in it does not grow. 3.1. Inference with cell assemblies We demonstrate this procedure with the following example. The manifestations ‘oil lamp lights up’ and ‘braking’ are present (represented in our symbol notation by K and E). The short-term memory is initialized with =K; E=, see Fig. 4(i). In the first inference step, F is deduced; the short-term memory is now =K; E; F =, see Fig. 4(ii). The features of the short-term memory are presented to the associative memory and, in the following

ARTICLE IN PRESS 814

A. Wichert / Neurocomputing 69 (2006) 810–824 K

11

J

10

I

9

H

8

G

7

F

6

E

5

D

4

C

3

B

2

A

1

1

2

3

4

5

6

7

8

9

10 11

A

B

C

D

E

F

G

H

I

J

K

Fig. 3. The architecture of our diagnostic system is composed of associative memory with feedback connections [22,24]. The rule A ) B _ C is represented by the first unit; that it is an ‘or’ rule is indicated by a one in its threshold (represented in this figure by a black dot). The associative memory forms the long-term memory; the short-term memory that is initialized with the initial state description is represented by the row buffer on the left side of the associative memory. The manifestations of the short-term memory are presented to the associative memory, which determines the disorders by using the retrieval rule to perform an abductive inference. The determined disorders are transported from the buffer below the units (column buffer) via the feedback connections to the short-term memory.

inference, C and F are deduced, see Fig. 5(iii). The shortterm memory is then updated to =K; E; F ; C=. In the next inference steps, C, F and A are deduced. A is deduced, because A ) B _ C is an ‘or’ rule as indicated by the threshold value of the corresponding unit, see Fig. 5(iv). The inference procedure is completed because no new features are determined in the following inference step. The results of our diagnostic systems are hypotheses represented by the features that are not manifestations in the performed diagnose process. If there is more than one determined hypothesis, then they are presented to the user in a fixed arrangement, which reflects their plausibility. In order to determine the arrangement, we use the following rule. The hypotheses with less inference steps are the most probable. In our example the disorder A is most probably present. This means that ‘the cable of the oil pressure lamp is loose’ is most probably present. This is because A is not a manifestation in the performed diagnose process. Suppose the short-term memory is initialized only with =E=. The answer to the diagnose process would be =E= as well, because =E= is not a manifestation in the performed diagnose process.

3.2. Taxonomic knowledge organization For clarity, rules should be arranged in groups [1,14] that define taxonomy. In our diagnose system the components of a modelled machine are reflected by this organization. A module represents each group, each feature is indicated by a name in the context of a module and each module is represented by an associative memory. A disorder that is defined in a certain module can cause manifestations in other modules. This relation corresponds to connections between modules. As an example we present a compendium of 16 rules which are arranged in five modules and their representation by a graph, see Fig. 6. module 1 1. 2. 3. 4. 5. 6.

A)C^D C)E D)E E)F^G I)G^H G ) E ^ module 2½A

ARTICLE IN PRESS A. Wichert / Neurocomputing 69 (2006) 810–824

K

1

11

K

1

11

J

0

10

J

0

10

I

0

9

I

0

9

H

0

8

H

0

8

G

0

7

G

0

7

F

0

6

F

0

6

E

1

5

E

1

5

D

0

4

D

0

4

C

0

3

C

0

3

B

0

2

B

0

2

A

0

1

A

0

1

(i)

1

2

3

4

5

6

7

8

9

0 A

0 B

0 C

0 D

0 E

0 F

0 G

0 H

0 I

10 11 0 J

0 K

(ii)

815

1

2

3

4

5

6

7

8

9

10 11

0 A

0 B

0 C

0 D

0 E

1 F

0 G

0 H

0 I

0 J

0 K

Fig. 4. (i) The manifestations ‘oil lamp lights up’ and ‘braking’ are represented in our symbol notation by K and E, respectively. The short-term memory is initialized with =K; E=. (ii) In the first inference step, F is deduced. The activation of the units is indicated by the buffer below the units and should not be confused with the ‘or’ rule indication of a one in the threshold (represented by a black dot).

K

1

11

K

1

11

J

0

10

J

0

10

I

0

9

I

0

9

H

0

8

H

0

8

G

0

7

G

0

7

F

1

6

F

1

6

E

1

5

E

1

5

D

0

4

D

0

4

C

0

3

C

1

3

B

0

2

B

0

2

A

0

1

A

0

1

(iii)

1

2

3

4

5

6

7

8

9 10 11

0 A

0 B

1 C

0 D

0 E

1 F

0 G

0 H

0 I

0 J

0 K

(iv)

1

2

3

4

5

6

7

8

9

10 11

1 A

0 B

1 C

0 D

0 E

1 F

0 G

0 H

0 I

0 J

0 K

Fig. 5. (iii) The short-term memory is now =K; E; F =. The features of the short-term memory are presented to the associative memory, and C and F are deduced in the following inference. (iv) The short-term memory is now updated to =K; E; F ; C=. In the next inference step, C, F and A are deduced. A is deduced, because A ) B _ C is an ‘or’ rule as indicated by the threshold value one of the corresponding unit (represented by a black dot).

module 2

module 3

1. 2. 3. 4. 4.

1. A ) B _ module 2½G

A)B^C^E E)D F)E_G G)I_H I ) module 4½C

module 4 1. C ) A _ B

ARTICLE IN PRESS A. Wichert / Neurocomputing 69 (2006) 810–824

816

4. Implications

Module 1 G

A

9

I

1

8

Module 3 2

A C

D

2

E 1

3

4

B

5

F

C

3

6

D

7

H

Module 2 6

F

A

9

7

G

I

8

5

H

1

E

B

4 D

2

Module 5

Module 4 3 1 A

1

C

4 D

2 B

5

E

2 5 K

T

A 3

P

4 L

Fig. 6. Representation of 16 rules arranged in five modules by two directed AND/OR graphs. A name and a number in the context of a module indicate each feature. Each module is represented by an associative memory. The number of a feature indicates the position in the corresponding vector. A disorder that is defined in a certain module can cause a manifestation in other modules. This relation corresponds to connections between modules. The ‘and’ rules are indicated by an arc between the links connecting the nodes, which indicate the manifestations.

2. D ) B _ E module 5 1. A ) T _ P 2. T ) K _ L The five modules are represented by five associative memories. From the five associative memories, we compose a global associative memory by the arrangement on the diagonal. An associative memory of the dimension 30 evolves in which local feature addresses are translated into global feature addresses (see Fig. 7). In this global context, connections between modules can be easily indicated by weights outside the modules, in the column of the first module and the row to the second module (connection from the first module to the second module). In Fig. 7 we can see three connections between the modules corresponding to our rule base.

A straightforward transformation from the symbolic rules into a representation by associative memory was presented with the aid of a toy example. Another way to look at the same transformation is the representation by directed AND/OR graphs of the search space generated by the deduction system. Unlike the approaches [28,17,31] the question and the answer vectors of the associative memory represent the same address space. This means that a particular feature is represented by the same position in both the question and the answer vectors. It should be noted that both vectors have the same dimension n. The disorders are represented by units (one-of n code). Each time a new feature is learned, the address space grows and a new unit is introduced. The number of represented disorders is correlated with the associative memory size; this number is only limited by the size of the growing associative memory. During the inference the short-term memory is initialized with the present manifestations. It is then presented to the associative memory, which determines the disorders by the retrieval rule. In the retrieval step the answer vector is calculated, however, no reconstruction of a fault question vector is made like in other models [21,30]. The newly obtained disorders are added to the short-term memory through the feedback connections. This procedure is repeated until the short-term memory does not change, i.e. the number of features does not grow. A breadth-first search in the directed AND/OR graph is executed. The features that were not manifestations in the performed inference process represent the answers. The number of inference cycles made by the associative memory with feedback connections corresponds to the maximum depth of the represented graphs. On a serial computer, a pointer representation of the associative memory can save memory space if the weight matrix is not dense [4]. This fact is important when the weight matrix becomes very large. In the pointer format, only the positions of the vector components unequal to zero are represented. This is done because most synaptic weights are zero. For example, the binary vector ½0 1 0 0 1 1 0 is represented as the pointer vector ð2 5 6Þ, which represents the positions of ‘ones’. In a matrix each row is represented by a vector. During the retrieval only the addressed rows indicated by the pointers of the question vector are added together and stored in the answer vector. The ratio between the number of stored weights and the size of the associative memory depends on the mean number b of stored weights unequal to zero in a unit, divided by n, the number of weights of a unit. In a directed AND/OR graph this ratio is represented by the mean of the branching factor of the nodes, b, divided by the number of the nodes, n: b ratio ¼ ; n

ratiop1.

ARTICLE IN PRESS A. Wichert / Neurocomputing 69 (2006) 810–824

817

Module 5 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1

5 4 3 2 1

Module 2

Module 1

9 8 7 6 5 4 3 2 1

Module 4 1 2 3 4 5 5 4 3 2 Modlue 3 1 1 2 3 4 5 2 1 1 2

1 2 3 4 5 6 7 8 9

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Fig. 7. The five modules are represented by five associative memories. By arranging them diagonally, we compose a global associative memory. An associative memory of dimension 30 evolves. In this global context, connections between modules can be easily indicated by weights depicted outside the modules. Connection from the first to the second module is indicated by the weights in the column of the first module and the row of the second module. In this figure we can see three connections between the modules.

A ratio of 1 indicates that all the weights of the associative matrix are equal to one. In the example in Fig. 1, b ¼ ð2 þ 2 þ 2 þ 3 þ 2 þ 1Þ=11 and ratio ¼ 0:099. A weight matrix is not dense if ratiooðsize of the pointer in bitsÞ1 .




4.1. Power analysis

The principles will be demonstrated by a real world application in Section 5.

The stability and scalability characteristics of the representation of directed AND/OR graphs by associative memory with feedback are summarized in the following points:






The inference in the AND/OR graphs corresponds to breadth-first search. The number of cycles performed by the associative memory with the feedback correspond to the maximum depth of the represented graphs. The directed AND/OR graph can contain cycles, because only new, not yet present disorders are added to the short-term memory, and the feedback is done until the number of features in the short-term memory does not grow.

The number of represented disorders is correlated with the size of the associative memory; this number is only limited by the size of the expandable associative memory.

5. Embedded diagnostic expert system for a bonding machine In modern industrial machines like for example, a bonding machine, more and more physical devices are microcontroller based. Bonding machines are a kind of industrial robot that glue chips on a platine or connect chips on a platine with wire. The microcontroller based devices of such machines communicate with each other over a so-called controller or communication area network (CAN). CAN is a serial bus system which was originally developed for automotive applications in the early 1980s. The CAN protocol was internationally standardized in 1993. If a device is faulty, it sends its error code over a high

ARTICLE IN PRESS 818

A. Wichert / Neurocomputing 69 (2006) 810–824

layer protocol to the user. The error code is interpreted and an error message appears on the user interface of a machine. With the growing complexity of modern machines hierarchical chains often occur in which a particular disorder causes other disorders. In this case, several error messages might be present. The user has to find the cause of the error messages. For this task he has to have a deep knowledge and understanding about the architecture of the machine. For example, if a certain fuse fails in the power supply of a bonding machine, many other disorders will occur. Because of the nature of the CAN protocol, they do not need to appear in succession of their occurrence. A human expert, who analyses them, has to deduce their cause. In our example the cause could be a certain wiring defect of the bonding machine, which causes the fuse to fail. The fuse fail will cause back-to-back failures. In order to save time and costs, an embedded diagnostic expert system on a bonding machine could indicate the possible cause of the occurring errors. Such embedded diagnostic expert system applications require limited computer resources. Examples are diagnostic systems in modern cars or on modern industrial machines. These systems run on small computers with real time operating systems. For a bonder machine application, additional difficulties are posed by the individual nature of such a device, as each machine may differ depending on the user. A characteristic of expert systems is the separation of the knowledge base and the inference engine. The latter applies the knowledge to the actual diagnostic problem. For each individual bonding machine, different knowledge bases in the form of a weight matrix can be delivered with the same inference engine.

5.1. Architecture The core of our diagnose system is the inference engine that is implemented by the associative memory with feedback connections. A prototypic knowledge base was created with the aim of estimating the ratio. The ratio was much smaller than the intended pointer size1 , which was a short unsigned integer (16 bit). Because of this, the associative memory could be implemented in pointer format to save memory space. To allow fast execution on computers with limited resources, the core was implemented in C þ þ instead of Java. This inference engine and a user interface form the diagnosis system of the bonder machine. They are integrated in the software framework of the bonder machine that runs on a personal computer with the real time operating system (lynx). The embedded diagnose system on the bonder machine is not a ‘hard’ real-time system, however. In this case it would have features that guarantee a response within a fixed amount of real-time. The inference engine is connected over a buffer to the high layer protocol that receives the incoming error codes from the CAN. Each time a new error occurs, the inference

engine diagnoses the cause with the aid of all the error codes in the buffer. The knowledge base is developed and tested with the aid of a knowledge-base editor on an external personal computer (MS Windows). In the knowledge-base editor every feature has an unambiguous name and an identification number in the context of a module. The user can compose new rules using already present or newly defined features, and can also edit the corresponding attributes and delete or change rules or modules, see Fig. 8.

5.2. Inference engine After the diagnose, the determined hypothesis that represents the causes of the errors are shown to the user in a fixed arrangement which reflects their plausibility, with the hypothesis that the disorders with less inference steps are the most probable. It is possible to perform an explanation, which justifies the determined hypothesis. For the explanation the performed inference chain is traced from the hypothesis to the manifestations that occurred. This explanation corresponds to the traditional ‘‘HOW’’ question of the expert system shells. The trace is performed by a backward projection. In a backward projection the original question vector and the answer vector are interchanged. The ‘‘answer’’ vector representing one chosen hypothesis is presented to the associative memory and the ‘‘question’’ vector is determined. This process can be repeated; in this case a depth-first search is performed. Additionally, the inference system can be invoked any time in the safety query modus. This procedure supposes that a global Error LED is on and that the CAN communication failed. The system asks the user if certain LEDs, which indicate a malfunction of parts, are on or not. The query behaviour of the system is controlled by two additional attributes that are advised for each feature. The following attributes can be present or absent:





observable; show information.

If the attribute observable is present, a query has to be performed. A query is performed if zi 40 and the corresponding rule contains at least one observable manifestation. If the user invokes the system, it is supposed that the observable Error LED is present. For all rules that contain this manifestation and at least one additional observable manifestation, a query to the user is posed. After the performance of the query, zi is recomputed. If the corresponding feature attribute show information is present during a query, information is presented which describes the position of a LED in the bonding machine.

ARTICLE IN PRESS A. Wichert / Neurocomputing 69 (2006) 810–824

819

Fig. 8. The development user interface is implemented in Java. The user can compose new rules using either already present features or newly defined ones. Every feature has an unambiguous name and an identification number in the context of a module. The user can verify in which rules certain features are present and can edit the corresponding attributes.

Power Supply

Electrical Interface

Peripheral Connector Unit

C-PCI CPU Board

PNE-Heater Controller

Indexer System

Lift System

Process Control

Motion Functional

Wire Spool Control

Pattern Recognition Unit

Fig. 9. Eleven modules model the microcontroller based units. They are arranged in a taxonomy that corresponds to the hierarchical chains which model the relationship between disorders and manifestations. This taxonomy is represented by two trees with root nodes Power Supply and C-PCI CPU Board.

5.3. Knowledge base The wire-bonding machine, whose error behaviour is modelled by the diagnostic expert system, is composed of the following microcontroller based units: 1. 2. 3. 4. 5. 6. 7.

Electrical Interface Process Control Motion Functional Pattern Recognition Unit C-PCI CPU Board PNE-Heater Controller Peripheral Connector Unit

8. 9. 10. 11.

Lift system Indexer system Wire Spool Control Power Supply

The knowledge base that models the error behaviour of the machine is composed of 350 rules stored in 11 modules. These modules model the microcontroller based units and are arranged in a taxonomy that corresponds to the hierarchical chains that model the relationship between disorders and manifestations, see Fig. 9. The safety query modus is described by 16 observable manifestations in the

ARTICLE IN PRESS A. Wichert / Neurocomputing 69 (2006) 810–824

820 Voltage selector defect

Primary voltage oscillate

LM-Amplifier defect or overloaded

Bonder temperature high

DC link overload

NOT ZK ok

NOTsame

Trafo temperature high

Temperature Error Warning

Error LED

Voltage selector fail LED ON

DC link LED off

Fig. 10. The relationship between the four root disorders of the module Power Supply, namely Voltage selector defect, Primary voltage oscillate, LMAmplifier defect or overloaded, Bonder temperature high, and the corresponding manifestations are represented by an AND/OR graph. The leaf manifestations NOT same, NOT ZK ok and Temperature Error Warning receive the information directly from CAN. Leaf manifestations which receive their information from CAN are indicated in the AND/OR graph by a thick border. The observable leaf manifestations of the safety query mode are indicated in the AND/OR graph by a dashed border, namely Error LED, Voltage selector fail LED on and DC link LED off . The ‘and’ rules are indicated by an arc between the links connecting the nodes indicating the manifestations.

Pneumatic terminal not connected

Ring fail

Fuse Terminal fail

Terminal fail

Output open short circuit to Vbat

Output open load or GND

pressure/vaccum error pressure/vaccum warning

Output 7 open load or GND

Output 7 open short circuit to Vbat

Output 6 open load or GND

Output 6 open short circuit to Vbat

Output 5 open load or GND Output 4 open load or GND

vaccum error

vaccum warning

Output 5 open short circuit to Vbat

forming gas warning

forming gas error

Output 4 open short circuit to Vbat

Output 3 open load or GND

Output 3 open short circuit to Vbat

Output 2 open load or GND Output 1 open load or GND

air secondary warning

Output 2 open short circuit to Vbat

air primary warning

air secondary error air primary error

Output 1 open short circuit to Vbat

Fig. 11. The relationship between the root disorder Pneumatic terminal not connected and the manifestations in the module PNE-Heater Controller is shown. In this representation an OR rule is also present: Ring fail ) Output open load or GND _ Output open short circuit to Vbat _ pressure/vaccum warning _ pressure/vacuum error. The module PNE-Heater Controller has no observable leaf manifestations. Leaf manifestations which receive their information from CAN are indicated in the AND/OR graph by a thick border. The ‘and’ rules are indicated by an arc between the links connecting the nodes indicating the manifestations.

knowledge base. They are present in 13 rules in the module number 11, ‘‘Power Supply’’. The abductive interpretation of the relationship between disorders and manifestations allows the transfer of the knowledge into logically sound AND/OR rules. In the following, we demonstrate the principle by examples from the knowledge base. Fig. 10 illustrates the relationship between the four root disorders of the module Power Supply, namely Voltage selector defect, Primary voltage oscillate, LM-Amplifier defect or overloaded, Bonder temperature high, and the corresponding manifestations on an AND/OR graph. These corresponding manifestations occur when the disorders are present. Corresponding error codes from the electronic devices or sensors, which are transmitted over the CAN, indicates the presence of the leaf manifestations. The leaf manifestations NOT same, NOT ZK ok and Temperature Error Warning receive the information directly from CAN. Leaf manifestations which

receive their information from CAN are indicated in the AND/OR graph by a thick border. In the presence of some leaf manifestations, the hypothesis about the disorders is deduced by the abductive inference. For example, if the leaf manifestations NOT same, NOT ZK ok are present, then the manifestation NOT same could be caused either by Voltage selector defect or Primary voltage oscillate or by both of them. The observable leaf manifestations of the safety query mode are indicated in the AND/OR graph by a dashed border, namely Error LED, Voltage selector fail LED on and DC link LED off . Fig. 11 depicts the relationship between the root disorder Pneumatic terminal not connected and the manifestations in the module PNE-Heater Controller. In this representation an OR rule is also present: Ring fail ) Output open load or GND _ Output open short circuit to Vbat _ pressure/vacuum warning _ pressure/vacuum error. The module PNE-Heater Controller has no observable leaf manifestations.

ARTICLE IN PRESS A. Wichert / Neurocomputing 69 (2006) 810–824

821

24V-NT defect

NOT +24 Global ok

Electrical Interface

Wiring to PNE-HeaterRack defect

Fuse Heater fail

Wiring to INX defect

Fuse INX fail

Wiring to Inlift defect

PNE-Heater Controller

Wiring to Outlift defect

Fuse Inlift fail

Wiring to Back defect

Wiring to Erack defect

Fuse Back fail

Fuse Erack fail

Fuse Outlift fail

Not Erack

Fan Rear not connected

Fan Rear damaged

NOT Fan Rear ok

NOT Heater ok

NOT 24 INX

NOT Inlift

Front Fans not connected

Fan Left damaged

NOT Fan Front Left ok

Not Outlift

Fan Middle damaged

NOT Fan Middle ok

Fan Rigth damaged

NOT Fan Rigth ok

Not Back

NOT Fan ok

NOT 24 PNE LED

NOT 24 INX LED

NOT Inlift LED

Not Outlift LED

Not Back LED

NOT Fan ok LED

Error LED

Fig. 12. The manifestation NOT +24 Global ok caused directly by the root disorder 24V-NT defect of the module Power Supply receives the information from CAN. This manifestation is a part of a hierarchical chain; it is also a disorder that causes many other manifestations, even in the modules Electrical Interface and PNE-Heater Controller. Corresponding error codes, which are transmitted over CAN, indicate the presence of the nine leaf manifestations. The observable leaf manifestations of the safety query mode are indicated in the AND/OR graph by a dashed border. The ‘and’ rules are indicated by an arc between the links connecting the nodes indicating the manifestations.

The manifestation NOT +24 Global ok receives its error code directly from CAN. It is caused directly by the root disorder 24V-NT defect of the module Power Supply, see Fig. 12. This manifestation is part of a hierarchical chain; it is also a disorder that causes many other manifestations, even in the modules Electrical Interface and PNE-Heater Controller. Corresponding error codes, which are transmitted over CAN, indicate the presence of the nine leaf manifestations. If 24V-NT defect is present, indicated by CAN error message NOT +24 Global ok, all nine leaf manifestations have to be present. If not, then the CAN communication is faulty. However, if all nine leaf manifestations are present as well as the corresponding manifestation in the modules Electrical Interface and PNE-

Heater Controller, then the NOT +24 Global ok disorder does not to need be present. This is because the abductive inference is not sound. The knowledge base made up of 350 rules generates a search space of 69 directed AND/OR graphs, see Table 1. It is represented by 11 associative memories from which a global associative memory is composed by the diagonal arrangement. A global associative memory of the dimension 571 evolves. The mean global branching factor is 1. This value results from the fact that some manifestations are not disorders, and that the branching factor of the represented rules is seldom bigger than two. The global associative memory is composed of only 571 weights, which are not equal to zero. The maximum number of

ARTICLE IN PRESS A. Wichert / Neurocomputing 69 (2006) 810–824

822

Table 1 The table represents the data of the 11 associative memories and the global associative memory Module

No of rules

No of graphs

Max depth

Mean branching

Dim.

Ratio

1 2 3 4 5 6 7 8 9 10 11 S

24 40 25 10 4 14 16 60 50 51 56 350

7 5 4 4 4 2 4 12 8 8 11 69

5 4 3 2 1 4 3 7 6 6 7 7

1.2 1.3 1.5 0.9 0.5 0.9 1 1.2 0.9 0.9 1.1 1

35 55 28 39 8 44 45 85 75 77 80 571

0.034 0.023 0.05 0.023 0.0625 0.02 0.022 0.014 0.012 0.02 0.0136 0.0018

The first column indicates the modules. The second column indicates the number of rules stored in the associative memory. The third column indicates the number of directed AND/OR graphs. Their maximum depth is also indicated, followed by the mean branching and the dimension of the associative memory. The last column gives the ratio. The ratio of the global associative memory is much smaller than the pointer size1 ¼ 0:0625, a short unsigned integer (16 bit). The mean branching factor of the global associative memory is 1.

cycles performed by the associative memory with feedback corresponds to the maximum depth of the represented graphs, which are seven without the depth of the observable manifestations. 5.4. Diagnosis The following error code is transmitted over CAN to the diagnose expert system of the wire bonder machine:




Error ‘NOT Heater ok’ in Module Power Supply

After the diagnose the determined hypothesis which represents the possible cause of the error is presented to the user: Module Power Supply: most probably Wiring to PNE-Heater-Rack defect. It is possible to perform an explanation that justifies the determined hypothesis. For this explanation the performed inference chain is traced from the hypothesis to the manifestations that occurred. This trace corresponds in the AND/OR graph to the path from the hypothesis described by the disorder to the manifestation, see also Fig. 12. The user interface of the bonder machine allows the answers to the questions to be yes, no or quit: 4 Power Supply: explain Wiring to PNEHeater-Rack defect (y/n/q) y Wiring to PNE-Heater-Rack defect is mostly probably present because it is supported by Fuse Heater fail. 4 Power Supply: explain Fuse Heater fail (y/n/q) y Fuse Heater fail is mostly probably present because it is supported by NOT Heater ok. In the next example several error codes are transmitted over CAN to the diagnose expert system of the wire bonder machine:






Error ‘NOT Fan Front Left ok’ in Power Supply Error ‘NOT Fan Front Middle ok’ in Power Supply Error ‘NOT Fan Front Right ok’ in Power Supply

After the diagnose the obtained hypothesis representing the possible cause of the errors is presented to the user, compare Fig. 12: Module Power Supply: most probably Front Fans not connected. An additional error code is transmitted over CAN to the diagnose expert system of the wire bonder machine. Together there are four present manifestations:







Error Error Error Error

‘NOT ‘NOT ‘NOT ‘NOT

Fan Fan Fan Fan

Front Left ok’ in Power Supply Front Middle ok’ in Power Supply Front Right ok’ in Power Supply Rear ok’ in Power Supply

The diagnose system indicates that the four manifestations could be caused by the following hypothesis: Module Power Supply: most probably Wiring to Erack defect. This hypothesis about the disorders is deduced from an abductive inference that is not sound. The manifestations could also be caused by the disorders Fan Rear not connected and Front Fan not connected as shown by the explanation that justifies the determined hypothesis, see also Fig. 12: 4 Power Supply: explain Wiring to Erack defect (y/n/q) y Wiring to Erack defect is mostly probably present because it is supported by Fuse Erack fail. 4 Power Supply : explain Fuse Erack fail (y/ n/q) y Fuse Erack fail is mostly probably present because it is supported by Not Erack. 4 Power Supply : explain Not Erack (y/n/q) y

ARTICLE IN PRESS A. Wichert / Neurocomputing 69 (2006) 810–824

Not Erack fail is mostly probably present because it is supported by Fan Rear not connected and Front Fans not connected. 4 Power Supply : explain Fan Rear not connected (y/n/q) q An additional error code is transmitted:








Error Error Error Error Error

‘NOT Fan Front Left ok’ in Power Supply ‘NOT Fan Front Middle ok’ in Power Supply ‘NOT Fan Front Right ok’ in Power Supply ‘NOT Fan Rear ok’ in Power Supply ‘Terminal fail’ in PNE-Heater Controller

In this case the determined hypotheses about the disorders are causally independent. They are presented to the user in a fixed arrangement, which reflects their plausibility. It is supposed that the disorder Fuse Terminal fail is likely to occur than Wiring to Erack defect. This is because less abductive inference steps were performed: one for Fuse Terminal fail and five for Wiring to Erack defect, see Figs. 11 and 12: 1 -4 Module PNE-Heater Controller: most probably Fuse Terminal fail. 2 -4 Module Power Supply: most probably Wiring to Erack defect. Additionally, the inference system can be invoked at any time in the safety query modus. This procedure supposes that a global Error LED is on and that the CAN communication failed. The system asks the user whether certain LEDs, which indicate a part malfunction, are on or not. 4 Fuse F1 is located at the second slot. 4 Fuse F1 Line ok LED OFF present (y/n/q) n 4 On the right side near the power supply. 4 VoltageSelectorFail LED ON present (y/n/ q) y 4 USG Supply LED OFF on the fourth slot. 4 USG Supply LED OFF present (y/n/q) q The hypothesis, which represents the possible cause of the error, is presented to the user as following: Module Power Supply: most probably VoltageSelector defect. 6. Conclusion It was demonstrated how cell assemblies could perform diagnostic problem solving. The diagnose process is described by diagnostic deduction systems which perform an abductive inference. The relationship between the disorders and manifestations is represented by logical rules; the abductive inference is characterized by the verbal category theory. The deduction system is modelled by an associative memory with feedback connections. The associative memory with feedback connections can be viewed as a data structure, which represents trees and directed AND/OR graphs. This view also indicates how trees and graphs may be represented and processed by the

823

human brain. The associative memory with feedback offers a self-contained architecture for the administration and representation of manifestations and disorders. The manifestations correspond to one layer and the disorders to another one; the hierarchical chains of disorders and manifestations are represented by cycles. This model representing a diagnose system can be implemented efficiently on a serial computer, requiring low memory space and low computational costs. Because of these advantages, this model was chosen for the implementation of a real embedded diagnostic system for a wire bonder machine. The knowledge base of this system is composed of 350 rules, stored in 11 modules. Acknowledgements The author would like to gratefully acknowledge two anonymous reviewers and the editor for their valuable suggestions. References [1] J.S. Aikins, Prototypical knowledge for expert systems, Artif. Intell. 20 (1986) 163–210. [2] J.A. Anderson, An Introduction to Neural Networks, MIT Press, Cambridge, MA, 1995. [3] D.H. Ballard, An Introduction to Natural Computation, MIT Press, Cambridge, MA, 1997. [4] H.J. Bentz, M. Hagstroem, G. Palm, Information storage and effective data retrieval in sparse matrices, Neural Networks 2 (4) (1989) 289–293. [5] V. Braitenberg, Gehirngespinste, Neuroanatomie fu¨r kybernetisch Interessierte, Springer, Berlin, 1973. [6] V. Braitenberg, Cell assemblies in the cerebral cortex, in: R. Heim, G. Palm (Eds.), Theoretical Approaches to Complex Systems, Springer, Berlin, 1978, pp. 171–188. [7] V. Braitenberg, Vehicles: Experiments in Synthetic Psychology, MIT Press, Cambridge, MA, 1984. [8] E. Franse´n, Biophysical simulation of cortical associative memory, Ph.D. Thesis, Stockholms Universitet, Department of Numerical Analysis and Computing Science, Royal Institute of Technology, S-100 44 Stockholm, Sweden, 1996. [9] D. Hebb, The Organization of Behavior, Wiley, New York, 1949. [10] D. Hebb, Textbook of Psychology, Sanders, Philadelphia, London, Toronto, 1958. [11] R. Hecht-Nielsen, Neurocomputing, Addison-Wesley, Reading, MA, 1989. [12] J. Hertz, A. Krogh, R.G. Palmer, Introduction to the Theory of Neural Computation, Addison-Wesley, Reading, MA, 1991. [13] J.R. Josephson, S.G. Josephson, Abductive Inference, Computation, Philosophy, Technology, Cambridge University Press, Cambridge, 1996. [14] G. Kahn, A. Kepner, J. Pepper, Test: a model-driven application shell, in: National Conference on Artificial Intelligence, 1987, pp. 814–818. [15] T. Kohonen, Self-Organization and Associative Memory, third ed., Springer, Berlin, 1989. [16] G.F. Luger, W.A. Stubblefield, Artifical Intelligence, Structures and Strategies for Complex Probelm Solving, third ed., Addison-Wesley, Reading, MA, 1989. [17] J.L. McClelland, D.E. Rumelhart, Distributed memory and the representation of general and specific memory, J. Exp. Psychol.: General 114 (1985) 159–188.

ARTICLE IN PRESS 824

A. Wichert / Neurocomputing 69 (2006) 810–824

[18] A. Newell, Unified Theories of Cognition, Harvard University Press, Cambridge, MA, 1990. [19] K. Opwis, R. Plo¨tzner, Kognitive Psychologie mit dem Computer, Spektrum Akademischer Verlag, Heidelberg, Berlin, Oxford, 1996. [20] D.N. Osherson, New axioms for the contrast model of similarity, J. Math. Psychol. 31 (1987) 93–103. [21] G. Palm, Neural Assemblies, an Alternative Approach to Artificial Intelligence, Springer, Berlin, 1982. [22] G. Palm, Assoziatives Geda¨chtnis und Gehirntheorie, in: Gehirn und Kognition, Spektrum der Wissenschaft, 1990, pp. 164–174. [23] G. Palm, Cell assemblies, coherence, and corticohippocampal interplay, Hippocampus 3 (1993) 219–226. [24] G. Palm, F. Schwenker, A. Bibbig, Theorie Neuronaler Netze 1. Skript zur Vorlesung, Department of Neural Information Processing, University of Ulm, 1992. [25] Y. Peng, J.A. Reggia, Abductive inference models for diagnostic problem-solving, Symbolic Computation, Springer, Berlin, 1990. [26] E.E. Smith, Concepts and categorization, in: E.E. Smith, D.N. Osherson (Eds.), Thinking, vol. 3, second ed., MIT Press, Cambridge, 1995, pp. 3–33 (Chapter 1). [27] K. Steinbuch, Die Lernmatrix, Kybernetik 1 (1961) 36–45. [28] K. Steinbuch, Automat und Mensch, fourth ed., Springer, Berlin, 1971.

[29] A. Tversky, Feature of similarity, Psychol. Rev. 84 (1977) 327–352. [30] T. Wennekers, Synchronisation und Assoziation in Neuronalen Netzen, Shaker Verlag, Aachen, 1999. [31] A. Wichert, A categorical expert system ‘‘jurassic’’, Expert Syst. Appl. 19 (3) (2000) 149–158. [32] P.H. Winston, Artificial Intelligence, third ed., Addison-Wesley, Reading, MA, 1992. Andreas Wichert studied computer science at the University of Saarland, from where he graduated in 1993. Afterwards, he became a PhD student at the Department of Neural Information Processing, University of Ulm. He received a PhD in computer science in 2000. He has since worked in the field of fMRI as a researcher with an interdisciplinary group, Department of Psychiatry III Ulm, changing to F&K Delvotec bonding machines where he led the development of a diagnostic expert system. From 2004 to 2005 he was the scientific director of MITI research group Klinikum rechts der Isar of the Technical University Munich. Recently, he joined the Faculdade de Cieˆncias da Universidade de Lisboa Departamento de Informa´tica.