- Email: [email protected]
Neural nets for fuzzy systems
JZZY
sets and systems Fuzzy Sets and Systems 71 (1995)265-276
ELSEVIER
Neural nets for fuzzy systems J a m e s J. B u c k l e y a'*, Y o i c h i H a y a s h i b aMathematics Department, University of Alabama at Birmingham, Birmingham, AL 35294, USA bDepartment of Computer Science, Meiji University, 1-1-1 Higashimita, Tama-ku, Kawasaki 214, Japan
Abstract We show how to represent fuzzy expert systems and fuzzy controllers as neural nets and as fuzzy neural nets. We show that fuzzy neural nets produce a more compact representation of fuzzy systems. Keywords: Expert systems; Control theory; Neural nets
1. Introduction In this paper we will construct hybrid fuzzy neural nets that will be computationally equivalent to fuzzy expert systems and certain fuzzy controllers 1 (including Sugeno, Mamdani, and expert system type). We will tell you the number of neurons in all hidden layers and we define all the weights. A motivation for this research is to build computationally fast parallel methods for fuzzy systems. In Section 2 we build hybrid neural nets for discrete fuzzy expert systems and hybrid fuzzy neural nets for (non-discrete) fuzzy expert systems. Section 3 is concerned with constructing hybrid fuzzy neural nets for certain fuzzy controllers. Section 4 contains a brief summary and our conclusions. In this section we will first describe what we mean by a neural net, hybrid neural net, fuzzy neural net, and hybrid fuzzy neural net.
Consider the three-layered, feedforward, neural net shown in Fig. 1. For simplicity we have assumed only two input neurons, one hidden layer, and one output neuron. We begin by having the signals and weights all real numbers. All neurons have a transfer function f which translates input to output. Usually the input neurons have y = f ( x ) = x (no change in input) and all the other neurons have the sigmoidal function y = f ( x ) = (1 - e-X) - 1. However, the transfer function, in general, can be any mapping f from the real numbers into the real numbers. Also, w e will usually not use a bias term'in the sigmoidal function. Assume for now that we will have f ( x ) = x in the two input neurons and the sigmoidal function in all other neurons. If the input signals are Xl and x2 (see Fig. 1), then the output from neuron # 1 ( # 2) in the input layer is xl (x2). The input to neuron # k in the hidden layer is Ik =
*Corresponding author. Tel: (205)934-2154. Fax: (205)9349025. E-mail: [email protected]. Without the method of defuzzifyinga fuzzyset neededin the Mamdani type controller.
X l W l k + .~2W2k,
1 <~ k ~< K.
(1)
The output from hidden neuron # k will be Zk=f(Ik),
0165-0114/95/$09.50 © 1995 - ElsevierScienceB.V. All rights reserved SSDI 0165-01 14(94)00282-7
l~
(2)
266
SJ. Buckley, E Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
Hidden Layer
Inputs XI
W
Output
~
/v.
Fig. 1. Neural network.
for sigmoidal f It follows that the input to the output neuron is Io = zlvl
+
...
+ ZKVK
(3)
and its output is Y =f(Io)
(4)
for sigmoidal f. We will have sigmoidal f in the output neuron when we require y to be in [0, 1], otherwise we omit f and have y = Io. What we have just described is what we call a regular neural net (NN). If we employ other operations like a t-norm, or a t-conorm, to combine the incoming data to a neuron (Eqs. (1) and (3)) we obtain what we call a hybrid neural net (HNN). HNNs also process real number signals and have real number weights. There are many applications of H N N s (see [13, 14, 34, 38, 40]). We will be concerned with fuzzifying both NNs and H N N s in this paper. There has been a lot of interest in the fact that regular neural nets are universal approximators. What this means is that given a continuous F : ~ " ~ ~ there is a regular neural net that can uniformly approximate F, to any degree of accuracy, on compact subsets of ~". See [4, 25 27, 31, 49-59,65-68,76] for a survey of this literature. Now consider a discrete fuzzy expert system or a fuzzy controller. In a discrete fuzzy expert system you input a discrete approximation to the fuzzy sets and obtain a discrete approximation to the output
fuzzy set. Usually, discrete fuzzy expert systems and fuzzy controllers are continuous mappings from some subset of ~" into 0~" (usually m = 1 for the fuzzy controller). We may use the result cited above to conclude that given a continuous (discrete) fuzzy expert system, or a continuous fuzzy controller, there is a regular neural net that can uniformly approximate it to any degree of accuracy on compact sets. The problem with this result is that it is non-constructive and only approximate. The main problem is that the theorems are existence types and do not tell you how to build the net. In this paper we will construct hybrid neural nets that will be computationally equivalent to discrete fuzzy expert systems/ Obviously, a problem with hybrid nets is that usually there is no learning algorithm. However, our hybrid nets do not have to learn anything. Our hybrid nets are for computation and not for learning. In [47] it was shown that given any continuous F: [0, 1]" ~ ~ there is a three-layered hybrid neural net computationally equivalent to F. However, the transfer functions inside the neurons were very complicated and not the type usually employed in neural nets. Therefore, the neural net researchers preferred the approximation results referenced above to getting a hybrid neural net identical to a continuous function. As we shall see our hybrid neural nets employ simple operations and will be easy to construct. There is a large, and growing, literature pertaining to the "joining" of neural nets and fuzzy systems. Many of these papers discuss: (1) using a neural net to "tune" the fuzzy system (e.g. [3, 33, 48, 64, 74, 84]); (2) combining neural nets and fuzzy systems, the so-called hybrid systems (e.g. [2, 30, 32, 46, 64, 70, 77, 78, 85]); or (3) using a neural net to approximate a fuzzy system (e.g. [7-11, 13,21,22,37,41,45,61-63,86]). The research most relevant to this paper is that of using a neural net to approximate a fuzzy system. The following items are common to the papers that use a neural net to approximate a fuzzy
2Our fuzzy expert systems are forward chaining with no uncertainties attached to the rules or the data.
267
J.J. Buekley, Y. Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
system: 3 (1) they prove a theorem that says there exists a neural net to approximate a fuzzy system but you do not know how many neurons to use it in the hidden layers; (2) the neural net only approximates the fuzzy system on the training data and its performance off the training data is never exactly known; (3) the values of the weights must be learned producing the possible difficulties of excessive learning times and convergence to local minima. This paper is different from the approximation papers referenced above in the following respect: (1) the hybrid neural net will be computationally identical to the (discrete) fuzzy expert system (no approximation); (2) we give the construction of the hybrid neural net so you will know the number of neurons in all hidden layers; and (3) we tell you all the weights so the net does not have to learn its weight values. A motivation for this research is to build computationally fast parallel methods (our hybrid neural nets) for fuzzy expert systems. Now let us discuss fuzzy neural nets. A regular fuzzy neural net (FNN) is an NN with fuzzy signals and/or fuzzy weights. We differentiate between different types of FNNs as follows: (1) FNN1 has real number input signals but fuzzy weights; (2) FNN2 has fuzzy set input signals and real number weights; and (3) F N N 3 has both fuzzy set input signals and fuzzy weights. There has been a growing interest in the theory and applications of FNNs [12, 13, 15-20, 35, 36, 39, 42-44, 51-57]. We will not discuss learning algorithms for FNNs because our fuzzy neural nets will be for fast parallel computation and they will not need to be trained to learn their weights. Learning algorithms for FNNs are discussed in [12, 17-20, 36, 39, 42-44, 46, 51-57, 66]. Let us now describe in more detail the internal computations of an FNN3. We place a bar over a symbol if it represents a fuzzy set. In an FNN3 the inputs X1, X2, the weights Wik, Vk, and the output will all be fuzzy. The architecture of FNN3 will be the same as in Fig. 1. The output from input neuron # 1 (#2) is X1 (X2). So, the input to hidden
neuron # k is Tk = X 1 W l k -Jv X 2 W 2 k ,
1 <<.k <~ K ,
(5)
where we use standard fuzzy arithmetic to compute [k. The output from the kth hidden neuron will be Zk=f([k),
l <~k <<.K,
(6)
for sigmoidal f, where the extension principle is used to obtain Zk. It follows that the input to the output neuron is To = Z1V1 + "" + Z~V~
(7)
and the final output will be 17 = f(/-o),
(8)
using regular fuzzy arithmetic in Eq. (7) and the extension principle in Eq. (8). When we do not need to be a fuzzy subset of [0, 1] we omit f i n Eq. (8) and set 17 =/-o. What we have just described is a regular FNN3 where standard fuzzy arithmetic (add, multiply) is used to compute the output. In a hybrid fuzzy neural net (I-IFNN) we may combine the fuzzy signals and weights using other operations besides addition and multiplication to obtain /-k and /-oHFNNs are of recent research interest (see [12, 13, 15-18]) and we shall discuss them again in the following sections. It is interesting to note that it has been shown that regular FNNs are not universal approximators [13, 15,18,35]. However, it is known that HFNNs can be universal approximators [13, 15, 16, 18]. Suppose we have a continuous (non-discrete) fuzzy expert system with fuzzy numbers as input and a fuzzy number as output. Then one can build a H F N N to uniformly approximate this fuzzy expert system on compact subsets of .~" (n-dimensional space of fuzzy numbers). Again, the problem with this result is that it is non-constructive and it does not tell you how to build the H F N N (the number of hidden neurons is unknown).
2. Fuzzy expert systems 3We do not imply that all of these items are contained in every approximationpaper. These items belong to the union of all approximationpapers.
Let us first introduce the basic notation to be employed in this paper. We place a bar over a
268
J.J.
Buckley, E H a y a s h i / F u z z y Sets a n d S y s t e m s 71 (1995) 2 6 5 - 2 7 6
symbol if it represents a fuzzy set. So A,B, C, .... X, Y, Z . . . . are fuzzy sets. All our fuzzy sets will be fuzzy subsets of the real numbers but fuzzy relations/~ will be fuzzy subsets of R" for n = 2 or 3. If.4 is a fuzzy set, then .4(x) denotes its membership function evaluated at x. Consider a fuzzy expert system with one block of rules ~i:
I f X =.4i, then Y = B~, 1 ~ i ~ n.
(9)
For simplicity we have only one clause in the antecedent but our results easily extend to many clauses in the antecedent. Given some data on X, say X =/1', the fuzzy expert system comes up with its final conclusion Y = B'. In computer applications we usually employ discrete versions of the continuous fuzzy sets A~,A',Bi and B'. So, let us first consider a discrete fuzzy expert system.
2.1. Discrete fuzzy expert system Let [ a l , b l ] contain the support of all the ,4~, 1 ~< i ~ n, plus the support of all the A' we might have as input to the system. Also, let [a2,b2] contain the support of all the B~, 1 ~< i ~< n, plus the support of all the/~' we can obtain as outputs from the system. Let Mo (No) be positive integers and set M = Mo+l, N=No+I. Choose x l = a ~ and xi= ax+(i-- 1)(bl-aa)/Mo, 2~i<<.M and pick yx = a2, y~ = a2 + (i - 1)(b2 - aE)/So, 2 <<.i <<.S. The discrete version of the system is to input a ' = (,4'(x0, ...,-4'(Xu)) and obtain output b ' =
(B'(yl) . . . . . B'tYN)). We now need to describe the internal workings of the fuzzy expert system. There are two cases [28,29]: (1) combine all the rules into one rule which is used to obtain b' from a'; or (2) fire the rules individually, given a', and combine their resuits into b'. We consider each case separately. 2.1.1. Combine the rules We first construct a fuzzy relation ll~k to model rule ~k, 1 ~< k <~ n. This is called modeling the implication in Eq. (9) and there are many ways to do this [69,72]. One takes the data Ak(Xl),Bk(Yj), 1 ~< i ~< M, 1 ~
rule k, 1 ~< k ~< n. One way to do this is
Rk(Xi, y j) = min(Ak(Xi), Bk(yj)).
(10)
In this paper it does not matter how/lk are defined. Then we combine all the/~k into one/~. Again, this operation may be performed in many different ways and one procedure would be to intersect the Rk to get/i. In any case let
(11)
rij = R(xi, yj),
the value o f / i at the pair (xi, yj). Now we are ready to input a' to get output b'. The method of computing b' from a' is called the compositional rule of inference (generalized modus ponens) and see [69, 72] for a survey of the procedures. Let 2ij :
(12)
a'i*rij, ! ~
-t
ar
t
,..
t
where ai A (xi), = (at, ,aM) and * is some method of combining the data into 2 u. Usually, • is a t-norm with min generally employed. Then set b' = (b'~. . . . . b~) and b) = ~b(21j. . . . . 2Mi),
1 ~ j ~ N,
(13)
for ~ a method of aggregating the information. Often a co-t-norm is used for ~ and usually it would be max. Therefore, we know how the fuzzy expert system computes b' given a'. A hybrid neural net computationally the same as this fuzzy expert system is shown in Fig. 2. The weights are the r u in/i. Let us look at node # 1 and argue that its output will be b'~ as in Eq. (13). We first combine the signals (al) and the weights (r,) as in Eq. (12) and then aggregate the data a'l*rll, ...,a'u*rMl using ~, sO the input to the neuron is b'l. Now the transfer function f is the identity function so that the output is b'~. Similarly for all neurons, which implies the net gives b' from a'. What is good about the H N N in Fig. 2 is that it gives fast parallel computation for a discrete fuzzy expert system. However, it can get too large to be useful. For example let [at, b1] = [ a 2 , b 2 ] = [ - 100, 100] with discrete increments of 0.1 so that M = N -- 2001. Then there will be: (1) 2001 input neurons; (2) (2001) 2 connections from the input
J.J. Buckley, Y. Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
a'
Input:
Input
r l ~
~
~
r l ~
269
1
Output
Output
r 2 2 ~
~ . Y b22 Fig. 2. Hybrid neural net as a discrete fuzzy expert system (combine the rules).
nodes to the output nodes; and (3) 2001 output nodes. Next consider having ,41i. . . . ,ALi in each rule's antecedent (1 ~< i ~< n) and you see that the H N N gets too big to be useful at this time.
2.1.2. Fire each rule In this case we compose a' with each/~k producing intermediate result bk = (bkl . . . . . bkN) and then combine all the b;, into b'. Let Rk(Xi,yj) ----rkij. Then we have, as in Eqs. (12) and (13), 2klj = a~*rkij and b'kj = ~(2klj, ..., 2kUj). The method of combining the b;, would be done componentwise so let !
b) = O(b'lj . . . . . b',j),
!
1 <<.j<<.N ,
¢
(14)
for some other aggregating operator 0. A t-norm, or co-t-norm, might be used for 0 and in particular 0 could be max, or min. A hybrid neural net computationally equal to this type of fuzzy expert system is in Fig. 3. For simplicity we have drawn the figure for M = N = 2. In the hidden layer: (1) the top two nodes operate as the first rule ~1 (as in Section 2.1.1); and (2) the bottom two-nodes model the second rule ~2 ( as previously described). In the two output nodes: (1) the top node, all weights are one, aggregates b'll and b~l using 0 (Eq. (14)) to produce b'l, and (2) the bottom node, weights are one, computes b~ as in Eq. (14). Therefore, the hybrid neural net computes the same output b' given a' as the fuzzy expert system. As in the previous case this H N N quickly gets too big to be practical at this time. Suppose
Fig. 3. Hybrid neural net as a discrete fuzzy expert systems (fire rules first).
M = N = 2001 (as in Section 2.1.1) and there are 25 rules. Then the H N N in Fig. 3 has: (1) 2001 input neurons; (2) (25)(2001) 2 connections from the input layer to the hidden layer; (3) (25)(2001) neurons in the hidden layer; (4) (25)(2001) connections from the hidden layer to the output neurons; and (5) 2001 output neurons. And this H N N has only one clause in each rule's antecedent. Having to discretize all the fuzzy sets in a fuzzy expert system can lead to an enormous H N N as we have seen above. It is the use of an H N N that dictates the discretization because it processes real numbers. We will obtain much smaller networks if we use fuzzy neural nets. So, let us now represent fuzzy expert systems as HFNNs.
2.2. Non-discrete fuzzy expert systems The fuzzy expert system will have one block of rules as shown in Eq. (9). The input to the system will be X = / 1 ' with final conclusion Y = B'. As in Section 2.1 we will consider two cases: (1) combine all rules into one rule; and (2) fire the rules individually and then combine their results.
2.2.1. Combine the rules We first obtain a fuzzy relation /~k to model the implication in each rule, 1 ~< k ~< n. Let Rk(X, y) = some function of Ak(X) and Bk(y) for x in [al, bl], y in [a2,b2]. For example this function could be MIN. Then we combine all the /~k,
270
J.J. Buckley, Y. Hayashi / Fuzzy Sets and Systems 71 (1995) 265 276
Input
1 ~
for y in [a2,b2],
Output
B,
(15)
where • represents the compositional rule of inference (generalized modus ponens) for the system. For example, one could have B'(y)=
sup
a I <~x<~bl
(16)
(min(A'(x),R(x,y))),
for each y in [a2, b2]. A hybrid fuzzy neural net (type FNN2), the same as this fuzzy expert system, is shown in Fig. 4. There is only one neuron with input weight equal to one. The transfer functions (which maps fuzzy sets into fuzzy sets) inside the neuron is the fuzzy relation/~. So, we have input ,4' to the neuron with its output B' = A ' . R .
We obtain the simplest possible H F N N for the fuzzy expert system. The major drawback is that at this time there is no hardware available to implement fuzzy neural nets.
Fig. 5. Hybrid fuzzy neural net as a fuzzy expert system (fire rules first).
transfer functions for the neurons in the hidden layer. The input signals to the output neuron are the B~, which are aggregated by 0 (Eq. (18)) for input to the output neuron. The transfer function in the output neuron is f ( x ) = x (no change) so the final output is B'. Again, H F N N is much smaller than the HNN in Fig. 3. However, there is no hardware available at this time to build the H F N N in Fig. 5.
2.2.2. Fire each rule
We first compose ,4' with each/~k to get B~,, the conclusion of the kth rule, and then combine all the B~, into one final conclusion B'. Let B~, be defined as (17)
B'k(y) = A ' ( x ) * R k ( X , y )
for y in [az, b2] and * the compositional rule of inference. Then /~'(y) = 0(/~'1(y). . . . . /~',(y))
(18)
for some aggregating function 0. An H F N N (type FNN2) the same as this fuzzy expert system is displayed in Fig. 5. All the weights are equal to one and the fuzzy relations/~k are the
3. Fuzzy controller In [5, 6] the author identifies three basic types of elementary fuzzy controllers: (1) Sugeno; (2) expert system; and (3) Mamdani. For another survey of elementary fuzzy controllers see [69]. We will now show how to build a hybrid fuzzy neural net to be computationally identical to each of these three types of fuzzy controller (see footnote 1). 3.1. Sugeno
We will first discuss how the Sugeno controller operates [69, 71, 79-83]. The fuzzy control rules are of the type ~:
A'
1@
~B'
Fig. 4. Hybrid fuzzy neural net as a fuzzy expert system (combine the rules).
If error =/i~ and change in error = B~, thenz=aie+fli(Ae)+7i,
1 <~ i <~ n,
(19)
where e = error and Ae = change in error. The Ai, Bi, cq, fl~, and 7~ are all given. The input to the controller is values for e and Ae and one first
J.J. Buckley, E Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
3.2. Expert system controller
evaluates each rule's antecedent as follows
tr~ = T (Ai(e), Bi(Ae)),
1 <~i <. n,
(20)
where T is some t-norm. Next, we evaluate each conclusion given e and Ae as
This fuzzy controller was based on the fuzzy expert system shell called F L O P S [23,24]. The fuzzy control rules are ~:
zi=~ie+fli(Ae)+Ti,
1 ~i~n.
aizi i=1
~ a,.
(22)
i=1
A hybrid fuzzy neural net (type FNN1) computationally equivalent to the Sugeno controller is displayed in Fig. 6. For simplicity, we have assumed only 2 rules in Fig. 6. First consider the first hidden layer. The inputs to the top two neurons are cqe + fll(Ae) and ~2e + fl2(Ae), respectively. The transfer function in these two neurons is f ( x ) = x + 7~, i = 1, 2, so the outputs are zl. A1 other neurons in this net have f ( x ) = x. The inputs to the bottom two nodes is T(fiq(e),/7i(Ae)) where: (1) the interaction between e (Ae) and ,4~ (Bi) is s/i(e) (B~(Ae)); and (2) T is used to aggregate these values for input to the neuron. In the rest of the net all weights are equal to one. What happens at the next four neurons is self-explanatory because we either multiply, or add, signals. The output produces 3 because we aggregate the two input signals using division (el + e2)/(O'i + 0"2).
Input / / &
If error = ,4~ and change in error = Bi,
(21)
The output is
6=
271
then control action = C~,
1 ~< i ~< n.
(23)
We assume that the C'~ are triangular shaped fuzzy numbers so let C~(x)= 1 if and only if x = cg, 1 ~< i ~< n. Given input values e = error and Ae = change in error, each rule is evaluated producing the tri given in Eq. (20). Then ai is assigned to the rule's consequence C~ and the controller takes all the data (O'i, C i ) , 1 ~< i ~< n, and defuzzifies to output 6. Let
6= ~
trici / ~,
i=1
/ i=1
ai.
(24)
A hybrid fuzzy neural net (type FNN~) computationally identical to this controller is shown in Fig. 7. Again, for simplicity we assume only two control rules in Fig. 7. The operations in this hybrid net are similar to those in Fig. 6.
3.3. Mamdani This was the first fuzzy controller I-1,73]. The fuzzy control rules are like those in Eq. (23) and they are evaluated like Eq. (20) given input values
~~'
Fig. 6. Hybrid fuzzy neural net as a Sugeno controller.
272
J.J. Buckley, Y. Hayashi / Fuzzy Sets and Systems 71 (1995) 265 276
Input e
c
Fig. 7. Hybrid fuzzy neural net as a fuzzy expert system controller.
e and Ae. However, we will adopt a more general approach to this type of controller that has been presented in [75]. This controller is based on some type of approximate reasoning (like in a fuzzy expert system) to evaluate the rules (Eq. (23)) given input values of e and Ae. Let /~(Z) = @1(,4i(e), Bi(Ae), Ci(z)),
-~'-~
-J--
- '-----~Pl(z])
zae Z1
(25)
where @1 is some general AND operator, for 1 <~ i ~< n. We are assuming that there is an interval [a3, b3] that contains the support of all the Ci and (7 (to be the fuzzy set output from the controller). Then Eq. (25) specifies /~/(z) for z in [a3,b3], 1 ~< i <~ n. Next define (7(z) = @2(/](z) l 1 ~< i ~< n),
Input e
Zp
--
"
(Ze) 8
(26)
where @2 is some general OR operator. (7 is the fuzzy set output from the controller. One must now defuzzify (7 into real numbers ~ for input to the process. We will discuss defuzzification at the end of this section. We now wish to construct a H F N N to represent this controller which maps the real number pair (e, Ae) into fuzzy set (7. See also [86]. Let us first consider a net to obtain a discrete approximation to (7 which will be very useful for the defuzzification process.
3.3.1. Discrete (7 Let Po be a positive integer and set P = Po + 1. Choose zl = a3 and zi = a3 + (i - 1)(b3 - a 3 ) / P o , 2 ~< i ~< P. We will compute (7(zl), ..., C(zp). A H F N N (type F N N 0 for this controller is shown in Fig. 8. The input neurons make no cha-
PI (Zi):
~
1
~ •
~
1 C(zi)
e.¢zi ~ Fig. 8. Hybrid fuzzy neural net as a discrete Mamdani type fuzzy controller: (a) hidden layer in the hybrid fuzzy neural net; (b) output neurons in the hybrid fuzzy neural net (1 <~ i ~< P).
nges in the input signals. In Fig. 8(a) the hidden layer is to compute/~(zi), 1 ~< i ~< P, for each rule (1 ~< k ~< n). Consider the top neuron in the hidden layer (labeled k = 1 for the first rule and i = 1 for z0. The interaction between the signals e, Ae, zl and the weights A 1, B1, C1 is ,~1 (e), B1 (Ae), C1 (zl). This data is aggregated using @1 (Eq. (25)) for input to
273
JJ. Buckley, E Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
the neuron. So, the input is Pt(zl). All the neurons in the hidden layer have f ( x ) = x (no change) for their transfer functions so the outputs from the neurons in the hidden layer are as shown in Fig. 8(a). These outputs are fed into the output neurons in Fig. 8(b). There will be P output neurons. The weights are all one and the aggregation function will be ~k2 (Eq. (26)) so the input to an output neuron is (7(zi), 1 ~< i ~< P. The transfer functions are f ( x ) = x in all the output neurons producing the final output as shown in Fig. 8(b). As in Sections 2.1.1 and 2.1.2 assume [a3, b3] = [ - 100, 100] producing P = 2001 and there are 25 (n = 25) rules. Then the H F N N in Fig. 8 has: (1) 2003 input neurons; (2) (3)(25)(2001) connections from input neurons to hidden layer neurons; (3) (25)(2001) hidden layer neurons; (4) (25)(2001) connections from the hidden layer to the output layer; and (5) 2001 output neurons. This is a very large H F N N . Alternatively, we could use an H N N as in Fig. 3 to produce a discrete approximation to (7. However, this would make an even larger net than the one in Fig. 8. The H F N N in Fig. 8 affords fast parallel computation for a Mamdani type controller and its output (discrete C) is ideal for defuzzification (discussed later in Section 3.3). However, as pointed out above it may be too large to be useful at this time. Now let us look at an H F N N that computes (7 (not discrete). 3.3.2. Non-discrete (7
The discussion here will be based on the results from Section 2.2.2. First construct a fuzzy relation /~k for each rule as RR(X,y,z) = dA(Ak(X),Bk(y),Ck(Z)),
Now the inputs are the fuzzy vector (~, Ae). Now compose the input with each/~k producing (~(x), Ae(y))*/~k (X, y, Z) = /~ (Z),
(30)
from Eq. (25). We then aggregate the Pk to get (7 as
(31)
(7(z) = O~(P,(z), ..., P.(z)),
from Eq. (26). Fig. 9 presents an H F N N (type FNN2) for this controller. All the weights are one. The input and output neurons have f ( x ) = x as their transfer function. The transfer functions in the neurons in the hidden layer are the fuzzy relations /~k which operate as in Eq. (30). The method of aggregating the incoming signals to the output neuron is ~02(Eq. (31)). So, the H F N N in Fig. 9 is this type of fuzzy controller. Obviously, the H F N N in Fig. 9 is much simpler than the one in Fig. 8 but, as mentioned before, at this time there is no hardware available to build the net in Fig. 9. 3.3.3. Defuzzify
When the controller produces a fuzzy set C it must then be defuzzified for input to the process. To use a neural net to defuzzify it will be most helpful to have a discrete approximation to (7. Assume we have (7(zi), 1 ~< i ~< P, as from the net in Fig. 8. A very common defuzzifier, the center of gravity, would be 6 = ~ zi(7(zi) i
(32)
C(zi). i=1
It would be easy to build an H N N to produce 3 given inputs (7(z~) and zi, 1 ~< i ~< P, so we will
1 <<.k <<.n.
(27) Input to the system will be real numbers e and Ae. The fuzzy sets ~, ~e are defined as follows ~(x)
= ~ )1, 0,
x = e, otherwise
Input
Output
(28)
and 1, y = Ae, A-e(Y)= 0, otherwise.
(29) Fig. 9. Hybridfuzzyneural net as a Mamdani fuzzycontroller.
274
J.J. Buckley, ~ Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
omit the details. Then this H N N would be attached to the H F N N in Fig. 8 or 9, to give the complete Mamdani type fuzzy controller.
4. Summary and conclusions The motivation for this research is to build computationally fast parallel methods for fuzzy systems. We first showed how to make a hybrid neural net equal to a discrete fuzzy expert system and then how to construct a hybrid fuzzy neural net equal to a (non-discrete) fuzzy expert system. We saw remarkable simplification in the nets when we went from real signals and weights to fuzzy signals and/or fuzzy weights. We then showed how to obtain hybrid fuzzy neural nets equal to a Sugeno, or an expert system, type of controller. We also discussed two different kinds of hybrid fuzzy neural nets for the Mamdani type of fuzzy controller. In the Mamdani case we obtained a major simplification in the net when we modeled the controller as a (non-discrete) fuzzy expert system and built a hybrid fuzzy neural net to represent it. The major problem at this time in representing fuzzy systems as fuzzy neural nets is that there is no hardware available to build these nets. Such hardware is needed for the future applications of fuzzy neural nets.
References [1] S. Assilian and E. Mamdani, An experiment in linguistic synthesis with a fuzzy logic controller, lnternat. J. Man Machine Studies 7 (1974) 1-13. [2] H.R. Berenji, Neural networks for fuzzy logic inference, Proc. 2nd IEEE Internat. Conf. on Fuzzy Systems, San Francisco (1993) Vol. II, 1395. [3] H.R. Berenji and P. Khedkar, Learning and tuning fuzzy logic controllers through reinforcements, IEEE Trans. Neural Networks 3 (1992) 724-740. [4] E.K. Blum and L.K. Li, Approximation theory and feedforward networks, Neural Networks 4 (1991) 511-515. [5] J.J Buckley, Theory of the fuzzy controller, in: C.V. Negoita, Ed., Cybernetics and Applied Systems (Marcel Dekker, New York, 1992) 293-307. [6] J.J. Buckley, Theory of the fuzzy controller: an introduction, Fuzzy Sets and Systems 51 (1992) 249-258.
[7] J.J. Buckley, Approximation paper: part I, Proc. 3rd Internat. Workshop on Neural Networks and Fuzzy Logic, NASA, Houston, TX (1992) 170-173. [8] J.J. Buckley, Approximations between nets, controllers, expert systems and processes, Proc. 2nd Internat. Conf. on Fuzzy Logic and Neural Networks, Iizuka, Japan (1992) Vol. 1, 89-90. [9] J.J. Buckley, Equivalences between nets, controllers, expert systems and processes, Proc. 1st lnternat. Conf. on Fuzzy Theory and Technology, Durham, NC (1992). [10] J.J. Buckley and E. Czogala, Fuzzy models, fuzzy controllers and neural nets, Arch. Theoret. Appl. Comput. Sci. 5 (1993) 149-165. [11] J.J. Buckley and E. Czogala, Modelling fuzzy processes using neural nets and fuzzy controllers, in: L. McAllister, Ed., Frontiers of Applied Mathematics (SIAM, Philadelphia, PA., to appear). [12] J.J. Buckley and Y. Hayashi, Fuzzy neural nets and applications, Fuzzy Systems and AI. 1(3) (1992) 11-41. [13] J.J. Buckley and Y. Hayashi, Numerical relationships between neural networks, continuous functions, and fuzzy systems, Fuzzy Sets and Systems 60 (1993) 1-8. [14] J.J. Buckley and Y. Hayashi, Hybrid neural nets can be fuzzy controllers and fuzzy expert systems, Fuzzy Sets and Systems 60 (1993) 135-142. [15] J.J. Buckley and Y. Hayashi, Can fuzzy neural nets approximate continuous fuzzy functions? Fuzzy Sets and Systems 61 (1993) 43-51. [16] J.J. Buckley and Y. Hayashi, Hybrid fuzzy neural nets are universal approximators, Proc. IEEE Internat. Conf. on Fuzzy Systems, Orlando (1994) Vol. 1,238-243. [17] J.J. Buckley and Y. Hayashi, Fuzzy neural networks, in: L.A. Zadeh and R.R. Yager, Eds., Fuzzy Sets, Neural Networks and Soft Computing (Van Nostrand Reinhold, New York, 1994) 233-249. [18] J.J. Buckley and Y. Hayashi, Fuzzy neural networks: a survey, Fuzzy Sets and Systems 66 (1994) 1-13. [19] J.J. Buckley and Y. Hayashi, Fuzzy backpropagation for fuzzy neural nets, unpublished manuscript. [20] J.J. Buckley and Y. Hayashi, Fuzzy genetic algorithm for fuzzy neural nets, unpublished manuscript. [21] J.J. Buckley, Y. Hayashi and E. Czogala, On the equivalence of neural nets and fuzzy expert system, Proc. Internat. Joint Conf. on Neural Networks, Baltimore (1992) Vol. II 691-695. [22] J.J. Buckley, Y. Hayashi and E. Czogala, On the equivalence of neural nets and fuzzy expert systems, Fuzzy Sets and Systems 53 (1993) 129-134. [23] J.J. Buckley, W. Siler and D. Tucker, Fuzzy expert system, Fuzzy Sets and Systems 20 (1986) 1 16. [24] J.J. Buckley and H. Ying, Expert fuzzy controller, Fuzzy Sets and Systems 44 (1991) 373-390. 1-25] P. Cardaliaguest and G. Euvrard, Approximation of a function and its derivative with a neural network, Neural Networks 5 (1992) 207 220. [26] N.E. Cotter and J.J. Gaillern, The CMAC and a theorem of Kolmogorov, Neural Networks 5 (1992) 221-228.
J.J. Buckley, Y. Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
[27] G. Cybenko, Approximation by Superpositions of a Sigmoidal Function, Math. Control Signals Systems 2 (1989) 303-314. [28] D. Dubois, R. Martin-Clouaire and H. Prade, Practical computing in fuzzy logic, in: M.M. Gupta and T. Yamakawa, Eds., Fuzzy Computing (North-Holland, Amsterdam, 1988) 11-34. [29] D. Dubois and H. Prade, The generalized modus ponens under sup-min composition, in: M.M. Gupta, A. Kandel, W. Bandler and J.B. Kisha, Eds., Approximate Reasoning in Expert Systems (Elsevier, Amsterdam, 1985) 217-232. [30] B. Freisleben and T. Kunkelmann, Combining fuzzy logic and neural networks to control an autonomous vehicle, Proc. 2nd lEEE lnternat. Conf. on Fuzzy Systems, San Francisco (1993) Vol. I, 321-326. [31] S. Geva and J. Sitte, A constructive method for multivariate function approximation by multilayer perceptrons, IEEE Trans. Neural Networks 3 (1992) 621-623. [32] M.M. Gupta and M.B. Gorzalczany, Fuzzy neuro-computational technique and its application to modelling and control, Proc. IEEE Internat. Joint Conf. on Neural Networks, Singapore (1991) Vol. 2, 1454-1457. [33] Y. Hayashi, A neural expert system using fuzzy teaching input, Proc. IEEE Internat. Conf. on Fuzzy Systems, San Diego (1992) 485-491. [34] Y. Hayashi and J.J. Buckley, Fuzzy controllers and fuzzy expert systems as hybrid neural nets, Proc. 5th IFSA World Congr., Seoul, Korea (1993) Vol. I, 70 72. [35] Y. Hayashi and J.J. Buckley, Are regular fuzzy neural nets universal approximators? Proc. Internat. Joint Conf. on Neural Networks, Nagoya, Japan (1993) Vol. I, 721 724. [36] Y. Hayashi and J.J. Buckley, Direct fuzzification of neural networks, Proc. 1st Asian Fuzzy Systems Symposium, Singapore (1993) 560-567. [37] Y. Hayashi and J.J. Buckley, Approximations between fuzzy expert systems and neural networks, lnternat. J. Approx. Reasoning 10 (1994) 63-73. [38] Y. Hayashi and J.J. Buckley, Fuzzy max-min neural controller, unpublished manuscript. [39] Y. Hayashi, J.J. Buckley and E. Czogala, Systems engineering applications of fuzzy neural networks, J. Systems Engrg. 2 (1992) 232-236. [40] Y. Hayashi, J.J. Buckley and E. Czogala, Fuzzy neural controller, Proc. IEEE lnternat. Conf. on Fuzzy Systems, San Diego (1992) 197-202. [41] Y. Hayashi, J.J. Buckley and E. Czogala, Fuzzy expert systems versus neural networks, Proc. Internat. Joint Conf. on Neural Networks, Baltimore (1992) Vol. II 720-726. [42] Y. Hayashi, J.J. Buckley and E. Czogala, Fuzzy neural network with fuzzy signals and weights, Proc. Internat. Joint Conf. on Neural Networks, Baltimore (1992) Vol. II, 696-701. [43] Y. Hayashi, J.J. Buckley and E. Czogala, Systems engineering applications of fuzzy neural networks, Proc. Internat. Joint Conf. on Neural Networks, Baltimore (1992) Vol. II, 413 418.
275
[44] Y. Hayashi, J.J. Buckley and E. Czogala, Direct fuzzification of neural networks and fuzzified delta rule, Proc. 2nd lnternat. Conf. on Fuzzy Logic and Neural Networks, Iizuka (1992) Vol. I, 73-76. [45] Y. Hayashi, J.J. Buckley and E. Czogala, Approximation between fuzzy expert systems and neural networks, Proc. 2nd Internat. Conf. on Fuzzy Logic and Neural Networks, lizuka, Japan (1992) Vol. I, 135-139. [46] Y. Hayashi, J.J. Buckley and E. Czogala, Fuzzy neural network with fuzzy signals and weights, Internat. J. Intell. Systems $ (1993) 527-537. [47] R. Hecht-Nielsen, Kolmogorov's mapping neural network existence theorem, Proc. IEEE 1st Internat. Conf. on Neural Networks, San Diego (1987)Vol. III, 11-14. [48] S. Horikawa, T. Furuhashi and Y. Uchikawa, Composition methods of fuzzy neural networks (Ili), Proc. 7th Fuzzy System Symp, Nagoya, Japan (1991) 493-496 (in Japanese). [49] K. Hornik, Approximation capabilities of multilayer feedforward networks, Neural Networks 4 (1991) 251-257. [50] K. Hornik, M. Stinchcombe and H. White, Multilayer feedforward networks are universal approximators, Neural Networks 2 (1989) 359-366. [51] H. Ishibuchi, R. Fujioka and H. Tanaka, An architecture of neural networks for input vectors of fuzzy numbers, Proc. IEEE Internat. Conf. on Fuzzy Systems, San Diego (1992) 1293-1300. [52] H. Ishibuchi, R. Fujioka and H. Tanaka, Neural networks that learn from fuzzy if-then rules, IEEE Trans. Fuzzy Systems 1 (1993) 85-97. [53] H. Ishibuchi, K. Kwon and H. Tanaka, Learning of fuzzy neural networks from fuzzy inputs and fuzzy targets, Proc. 5th IFSA World Congr., Seoul, Korea (1993) Vol. I, 147 150. [54] H. Ishibuchi, K. Kwon and H. Tanaka, Implementation of fuzzy if-then rules by fuzzy neural networks with fuzzy weights, Proc. 1st European Congr. on Fuzzy and Intelligent Technologies, Aachen, Germany (1993) Vol. 1,209-215. [55] H. Ishibuchi, H. Okada and H. Tanaka, Interpolation of fuzzy i~then rules by neural networks, Proc. 2nd lnternat. Conf. on Fuzzy Logic and Neural Networks, Iizuka, Japan (1992) 337-340. [56] H. Ishibuchi, H. Okada and H. Tanaka, Learning of neural networks from fuzzy inputs and fuzzy targets, Proc. Internat. Joint Conf. on Neural Networks, Beijing (1992) Vol. II1, 447-452. [57] H. Ishibuchi, H. Okada and H. Tanaka, Fuzzy neural networks with fuzzy weights and fuzzy biases, Proc. IEEE lnternat. Conf. on Neural Networks, San Francisco (1993) Vol. III, 1650-1655. [58] Y. Ito, Approximation of functions on compact sets by finite sums of a sigmoid function without scaling, Neural Networks 4 (1991) 817-826. [59] Y. Ito, Approximation of continuous functions on R ~ by linear combinations of shifted rotations of a sigmoid function with and without scaling, Neural Networks 5 (1992) 105 115.
276
J.J. Buckley, Y. Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
[60] J.-S.R. Jang and C.T. Sun, Functional equivalence between radial basis function networks and fuzzy inference systems, IEEE Trans. Neural Networks 4 (1993) 156 159. [61] J.M. Keller and H. Tahani, Implementation of conjunctive and disjunctive fuzzy logic rules with neural networks, Internat. J. Approx. Reasoning 6 (1992) 221 240. [62] J.M. Keller and H. Tahani, Backpropagation neural networks for fuzzy logic, Inform. Sci. 62 (1992) 205-221. [63] J.M. Keller, R.R. Yager and H. Tahani, Neural network implementation of fuzzy logic, Fuzzy Sets and Systems 45 (1992) 1-12. [64] B. Kosko, Neural Networks and Fuzzy Systems (PrenticeHall, Englewood Cliffs, N J, 1992). [65] V.Y. Kreinovich, Arbitrary nonlinearity is sufficient to represent all functions by neural networks: a theorem, Neural Networks 4 (1991) 381-383. [66] P.V. Krishnamraju, J.J. Buckley, K.D. Reilly and Y. Hayashi, Genetic learning algorithms for fuzzy neural nets, Proc. IEEE Internat. Conf. on Fuzzy Systems, Orlando (1994) Vol. 3, 1969 1974. [67] V. Kfirkovh, Kolmogorov's theorem is relevant, Neural Comput. 3 (1991) 617-622. [68] V. KSrkovh, Kolmogorov's theorem and multilayer neural networks, Neural Networks 5 (1992) 501-506. [69] C.C. Lee, Fuzzy logic in control systems: fuzzy logic controller, Part II, IEEE Trans. Systems Man Cybernet. 20 (1990) 419-435. [70] C.-J. Lin and C.-C. Jou, A fuzzy-neural hybrid system for identifying nonlinear dynamic systems, Proc. Internat. Joint Conf. on Neural Networks, Beijing (1992) Vol. Ill, 625-630. [71] M. Maeda and S. Murakami, A design for a fuzzy logic controller, Inform. Sci. 45 (1988) 315 330. [72] P. Magrez and P. Smets, Fuzzy modus pones: a model suitable for applications in knowledge-based systems, Internat. J. Intelligent Systems 4 (1989) 181-200. [73] E.H. Mamdani, J.J. Ostergaard and E. Lembessis, Use of fuzzy logic for implementing rule-based controllers for industrial processes, in: H.-J. Zimmermann, L.A. Zadeh and B.R. Gaines, Eds., Fuzzy Sets and Decision Analysis (North-Holland, Amsterdam, 1984) 307-323.
[74] D. Nauck and R. Kruse, A fuzzy neural network learning fuzzy control rules and membership functions by fuzzy error backpropagation, Proc. IEEE lnternat. Conf. on Neural Networks, San Francisco (1993) Vol. I|, 1022-1027. [75] H.T. Nguyen and V. Kreinovich, On approximation of controls by fuzzy systems, Proc. 5th 1FSA World Congr., Seoul, Korea (1993) Vol. II, 1414-1417. [76] J. Park and I.W. Sandberg, Universal approximation using radial-basis-function networks, Neural Comput. 3 (1991) 246-257. [77] C. Posey, A. Kandel and G. Langholz, Fuzzy hybrid systems, in: A. Kandei and G. Langholz, Eds., Hybrid Architectures for Intelligent Systems (CRC Press, Boca Raton, Florida, 1992) 174 196. [78] P.K. Simpson, Fuzzy min-max Neural networks for function approximation, Proc. IEEE Internat. Conf. on Neural Networks, San Francisco (1993) Vol. III, 1967-1972. [79] M. Sugeno, An introductory survey of fuzzy control, Inform. Sci. 36 (1985) 59-83. [80] M. Sugeno and G.T Kang, Fuzzy modeling and control of multilayer incinerator, Fuzzy Sets and Systems 18 (1986) 329-346. [81J M. Sugeno and G.T. Kang, Structure identification fuzzy model, Fuzzy Sets and Systems 28 (1988) 15-33. [82] M. Sugeno and K. Murakami, An experimental study on fuzzy parking control using a model car, in: M. Sugeno, Ed., Industrial Applications of Fuzzy Control (NorthHolland, Amsterdam, 1985) 125-138. [83] M. Sugeno and M. Nishida, Fuzzy control of model car, Fuzzy Sets and Systems 16 (1985) 103-113. [84] H. Takagi and I. Hayashi, NN-driven fuzzy reasoning, Internat. d. Approx. Reasoning 5 (1991) 191-212. [85] P.J. Werbos, Neurocontrol and fuzzy logic: connections and designs, Internat. J. Approx. Reasoning 6 (1992) 185-219. [86] R.R. Yager, Implementing fuzzy logic controller using a neural network framework, Fuzzy Sets and Systems 48 (1992) 53-64.
sets and systems Fuzzy Sets and Systems 71 (1995)265-276
ELSEVIER
Neural nets for fuzzy systems J a m e s J. B u c k l e y a'*, Y o i c h i H a y a s h i b aMathematics Department, University of Alabama at Birmingham, Birmingham, AL 35294, USA bDepartment of Computer Science, Meiji University, 1-1-1 Higashimita, Tama-ku, Kawasaki 214, Japan
Abstract We show how to represent fuzzy expert systems and fuzzy controllers as neural nets and as fuzzy neural nets. We show that fuzzy neural nets produce a more compact representation of fuzzy systems. Keywords: Expert systems; Control theory; Neural nets
1. Introduction In this paper we will construct hybrid fuzzy neural nets that will be computationally equivalent to fuzzy expert systems and certain fuzzy controllers 1 (including Sugeno, Mamdani, and expert system type). We will tell you the number of neurons in all hidden layers and we define all the weights. A motivation for this research is to build computationally fast parallel methods for fuzzy systems. In Section 2 we build hybrid neural nets for discrete fuzzy expert systems and hybrid fuzzy neural nets for (non-discrete) fuzzy expert systems. Section 3 is concerned with constructing hybrid fuzzy neural nets for certain fuzzy controllers. Section 4 contains a brief summary and our conclusions. In this section we will first describe what we mean by a neural net, hybrid neural net, fuzzy neural net, and hybrid fuzzy neural net.
Consider the three-layered, feedforward, neural net shown in Fig. 1. For simplicity we have assumed only two input neurons, one hidden layer, and one output neuron. We begin by having the signals and weights all real numbers. All neurons have a transfer function f which translates input to output. Usually the input neurons have y = f ( x ) = x (no change in input) and all the other neurons have the sigmoidal function y = f ( x ) = (1 - e-X) - 1. However, the transfer function, in general, can be any mapping f from the real numbers into the real numbers. Also, w e will usually not use a bias term'in the sigmoidal function. Assume for now that we will have f ( x ) = x in the two input neurons and the sigmoidal function in all other neurons. If the input signals are Xl and x2 (see Fig. 1), then the output from neuron # 1 ( # 2) in the input layer is xl (x2). The input to neuron # k in the hidden layer is Ik =
*Corresponding author. Tel: (205)934-2154. Fax: (205)9349025. E-mail: [email protected]. Without the method of defuzzifyinga fuzzyset neededin the Mamdani type controller.
X l W l k + .~2W2k,
1 <~ k ~< K.
(1)
The output from hidden neuron # k will be Zk=f(Ik),
0165-0114/95/$09.50 © 1995 - ElsevierScienceB.V. All rights reserved SSDI 0165-01 14(94)00282-7
l~
(2)
266
SJ. Buckley, E Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
Hidden Layer
Inputs XI
W
Output
~
/v.
Fig. 1. Neural network.
for sigmoidal f It follows that the input to the output neuron is Io = zlvl
+
...
+ ZKVK
(3)
and its output is Y =f(Io)
(4)
for sigmoidal f. We will have sigmoidal f in the output neuron when we require y to be in [0, 1], otherwise we omit f and have y = Io. What we have just described is what we call a regular neural net (NN). If we employ other operations like a t-norm, or a t-conorm, to combine the incoming data to a neuron (Eqs. (1) and (3)) we obtain what we call a hybrid neural net (HNN). HNNs also process real number signals and have real number weights. There are many applications of H N N s (see [13, 14, 34, 38, 40]). We will be concerned with fuzzifying both NNs and H N N s in this paper. There has been a lot of interest in the fact that regular neural nets are universal approximators. What this means is that given a continuous F : ~ " ~ ~ there is a regular neural net that can uniformly approximate F, to any degree of accuracy, on compact subsets of ~". See [4, 25 27, 31, 49-59,65-68,76] for a survey of this literature. Now consider a discrete fuzzy expert system or a fuzzy controller. In a discrete fuzzy expert system you input a discrete approximation to the fuzzy sets and obtain a discrete approximation to the output
fuzzy set. Usually, discrete fuzzy expert systems and fuzzy controllers are continuous mappings from some subset of ~" into 0~" (usually m = 1 for the fuzzy controller). We may use the result cited above to conclude that given a continuous (discrete) fuzzy expert system, or a continuous fuzzy controller, there is a regular neural net that can uniformly approximate it to any degree of accuracy on compact sets. The problem with this result is that it is non-constructive and only approximate. The main problem is that the theorems are existence types and do not tell you how to build the net. In this paper we will construct hybrid neural nets that will be computationally equivalent to discrete fuzzy expert systems/ Obviously, a problem with hybrid nets is that usually there is no learning algorithm. However, our hybrid nets do not have to learn anything. Our hybrid nets are for computation and not for learning. In [47] it was shown that given any continuous F: [0, 1]" ~ ~ there is a three-layered hybrid neural net computationally equivalent to F. However, the transfer functions inside the neurons were very complicated and not the type usually employed in neural nets. Therefore, the neural net researchers preferred the approximation results referenced above to getting a hybrid neural net identical to a continuous function. As we shall see our hybrid neural nets employ simple operations and will be easy to construct. There is a large, and growing, literature pertaining to the "joining" of neural nets and fuzzy systems. Many of these papers discuss: (1) using a neural net to "tune" the fuzzy system (e.g. [3, 33, 48, 64, 74, 84]); (2) combining neural nets and fuzzy systems, the so-called hybrid systems (e.g. [2, 30, 32, 46, 64, 70, 77, 78, 85]); or (3) using a neural net to approximate a fuzzy system (e.g. [7-11, 13,21,22,37,41,45,61-63,86]). The research most relevant to this paper is that of using a neural net to approximate a fuzzy system. The following items are common to the papers that use a neural net to approximate a fuzzy
2Our fuzzy expert systems are forward chaining with no uncertainties attached to the rules or the data.
267
J.J. Buekley, Y. Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
system: 3 (1) they prove a theorem that says there exists a neural net to approximate a fuzzy system but you do not know how many neurons to use it in the hidden layers; (2) the neural net only approximates the fuzzy system on the training data and its performance off the training data is never exactly known; (3) the values of the weights must be learned producing the possible difficulties of excessive learning times and convergence to local minima. This paper is different from the approximation papers referenced above in the following respect: (1) the hybrid neural net will be computationally identical to the (discrete) fuzzy expert system (no approximation); (2) we give the construction of the hybrid neural net so you will know the number of neurons in all hidden layers; and (3) we tell you all the weights so the net does not have to learn its weight values. A motivation for this research is to build computationally fast parallel methods (our hybrid neural nets) for fuzzy expert systems. Now let us discuss fuzzy neural nets. A regular fuzzy neural net (FNN) is an NN with fuzzy signals and/or fuzzy weights. We differentiate between different types of FNNs as follows: (1) FNN1 has real number input signals but fuzzy weights; (2) FNN2 has fuzzy set input signals and real number weights; and (3) F N N 3 has both fuzzy set input signals and fuzzy weights. There has been a growing interest in the theory and applications of FNNs [12, 13, 15-20, 35, 36, 39, 42-44, 51-57]. We will not discuss learning algorithms for FNNs because our fuzzy neural nets will be for fast parallel computation and they will not need to be trained to learn their weights. Learning algorithms for FNNs are discussed in [12, 17-20, 36, 39, 42-44, 46, 51-57, 66]. Let us now describe in more detail the internal computations of an FNN3. We place a bar over a symbol if it represents a fuzzy set. In an FNN3 the inputs X1, X2, the weights Wik, Vk, and the output will all be fuzzy. The architecture of FNN3 will be the same as in Fig. 1. The output from input neuron # 1 (#2) is X1 (X2). So, the input to hidden
neuron # k is Tk = X 1 W l k -Jv X 2 W 2 k ,
1 <<.k <~ K ,
(5)
where we use standard fuzzy arithmetic to compute [k. The output from the kth hidden neuron will be Zk=f([k),
l <~k <<.K,
(6)
for sigmoidal f, where the extension principle is used to obtain Zk. It follows that the input to the output neuron is To = Z1V1 + "" + Z~V~
(7)
and the final output will be 17 = f(/-o),
(8)
using regular fuzzy arithmetic in Eq. (7) and the extension principle in Eq. (8). When we do not need to be a fuzzy subset of [0, 1] we omit f i n Eq. (8) and set 17 =/-o. What we have just described is a regular FNN3 where standard fuzzy arithmetic (add, multiply) is used to compute the output. In a hybrid fuzzy neural net (I-IFNN) we may combine the fuzzy signals and weights using other operations besides addition and multiplication to obtain /-k and /-oHFNNs are of recent research interest (see [12, 13, 15-18]) and we shall discuss them again in the following sections. It is interesting to note that it has been shown that regular FNNs are not universal approximators [13, 15,18,35]. However, it is known that HFNNs can be universal approximators [13, 15, 16, 18]. Suppose we have a continuous (non-discrete) fuzzy expert system with fuzzy numbers as input and a fuzzy number as output. Then one can build a H F N N to uniformly approximate this fuzzy expert system on compact subsets of .~" (n-dimensional space of fuzzy numbers). Again, the problem with this result is that it is non-constructive and it does not tell you how to build the H F N N (the number of hidden neurons is unknown).
2. Fuzzy expert systems 3We do not imply that all of these items are contained in every approximationpaper. These items belong to the union of all approximationpapers.
Let us first introduce the basic notation to be employed in this paper. We place a bar over a
268
J.J.
Buckley, E H a y a s h i / F u z z y Sets a n d S y s t e m s 71 (1995) 2 6 5 - 2 7 6
symbol if it represents a fuzzy set. So A,B, C, .... X, Y, Z . . . . are fuzzy sets. All our fuzzy sets will be fuzzy subsets of the real numbers but fuzzy relations/~ will be fuzzy subsets of R" for n = 2 or 3. If.4 is a fuzzy set, then .4(x) denotes its membership function evaluated at x. Consider a fuzzy expert system with one block of rules ~i:
I f X =.4i, then Y = B~, 1 ~ i ~ n.
(9)
For simplicity we have only one clause in the antecedent but our results easily extend to many clauses in the antecedent. Given some data on X, say X =/1', the fuzzy expert system comes up with its final conclusion Y = B'. In computer applications we usually employ discrete versions of the continuous fuzzy sets A~,A',Bi and B'. So, let us first consider a discrete fuzzy expert system.
2.1. Discrete fuzzy expert system Let [ a l , b l ] contain the support of all the ,4~, 1 ~< i ~ n, plus the support of all the A' we might have as input to the system. Also, let [a2,b2] contain the support of all the B~, 1 ~< i ~< n, plus the support of all the/~' we can obtain as outputs from the system. Let Mo (No) be positive integers and set M = Mo+l, N=No+I. Choose x l = a ~ and xi= ax+(i-- 1)(bl-aa)/Mo, 2~i<<.M and pick yx = a2, y~ = a2 + (i - 1)(b2 - aE)/So, 2 <<.i <<.S. The discrete version of the system is to input a ' = (,4'(x0, ...,-4'(Xu)) and obtain output b ' =
(B'(yl) . . . . . B'tYN)). We now need to describe the internal workings of the fuzzy expert system. There are two cases [28,29]: (1) combine all the rules into one rule which is used to obtain b' from a'; or (2) fire the rules individually, given a', and combine their resuits into b'. We consider each case separately. 2.1.1. Combine the rules We first construct a fuzzy relation ll~k to model rule ~k, 1 ~< k <~ n. This is called modeling the implication in Eq. (9) and there are many ways to do this [69,72]. One takes the data Ak(Xl),Bk(Yj), 1 ~< i ~< M, 1 ~
rule k, 1 ~< k ~< n. One way to do this is
Rk(Xi, y j) = min(Ak(Xi), Bk(yj)).
(10)
In this paper it does not matter how/lk are defined. Then we combine all the/~k into one/~. Again, this operation may be performed in many different ways and one procedure would be to intersect the Rk to get/i. In any case let
(11)
rij = R(xi, yj),
the value o f / i at the pair (xi, yj). Now we are ready to input a' to get output b'. The method of computing b' from a' is called the compositional rule of inference (generalized modus ponens) and see [69, 72] for a survey of the procedures. Let 2ij :
(12)
a'i*rij, ! ~
-t
ar
t
,..
t
where ai A (xi), = (at, ,aM) and * is some method of combining the data into 2 u. Usually, • is a t-norm with min generally employed. Then set b' = (b'~. . . . . b~) and b) = ~b(21j. . . . . 2Mi),
1 ~ j ~ N,
(13)
for ~ a method of aggregating the information. Often a co-t-norm is used for ~ and usually it would be max. Therefore, we know how the fuzzy expert system computes b' given a'. A hybrid neural net computationally the same as this fuzzy expert system is shown in Fig. 2. The weights are the r u in/i. Let us look at node # 1 and argue that its output will be b'~ as in Eq. (13). We first combine the signals (al) and the weights (r,) as in Eq. (12) and then aggregate the data a'l*rll, ...,a'u*rMl using ~, sO the input to the neuron is b'l. Now the transfer function f is the identity function so that the output is b'~. Similarly for all neurons, which implies the net gives b' from a'. What is good about the H N N in Fig. 2 is that it gives fast parallel computation for a discrete fuzzy expert system. However, it can get too large to be useful. For example let [at, b1] = [ a 2 , b 2 ] = [ - 100, 100] with discrete increments of 0.1 so that M = N -- 2001. Then there will be: (1) 2001 input neurons; (2) (2001) 2 connections from the input
J.J. Buckley, Y. Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
a'
Input:
Input
r l ~
~
~
r l ~
269
1
Output
Output
r 2 2 ~
~ . Y b22 Fig. 2. Hybrid neural net as a discrete fuzzy expert system (combine the rules).
nodes to the output nodes; and (3) 2001 output nodes. Next consider having ,41i. . . . ,ALi in each rule's antecedent (1 ~< i ~< n) and you see that the H N N gets too big to be useful at this time.
2.1.2. Fire each rule In this case we compose a' with each/~k producing intermediate result bk = (bkl . . . . . bkN) and then combine all the b;, into b'. Let Rk(Xi,yj) ----rkij. Then we have, as in Eqs. (12) and (13), 2klj = a~*rkij and b'kj = ~(2klj, ..., 2kUj). The method of combining the b;, would be done componentwise so let !
b) = O(b'lj . . . . . b',j),
!
1 <<.j<<.N ,
¢
(14)
for some other aggregating operator 0. A t-norm, or co-t-norm, might be used for 0 and in particular 0 could be max, or min. A hybrid neural net computationally equal to this type of fuzzy expert system is in Fig. 3. For simplicity we have drawn the figure for M = N = 2. In the hidden layer: (1) the top two nodes operate as the first rule ~1 (as in Section 2.1.1); and (2) the bottom two-nodes model the second rule ~2 ( as previously described). In the two output nodes: (1) the top node, all weights are one, aggregates b'll and b~l using 0 (Eq. (14)) to produce b'l, and (2) the bottom node, weights are one, computes b~ as in Eq. (14). Therefore, the hybrid neural net computes the same output b' given a' as the fuzzy expert system. As in the previous case this H N N quickly gets too big to be practical at this time. Suppose
Fig. 3. Hybrid neural net as a discrete fuzzy expert systems (fire rules first).
M = N = 2001 (as in Section 2.1.1) and there are 25 rules. Then the H N N in Fig. 3 has: (1) 2001 input neurons; (2) (25)(2001) 2 connections from the input layer to the hidden layer; (3) (25)(2001) neurons in the hidden layer; (4) (25)(2001) connections from the hidden layer to the output neurons; and (5) 2001 output neurons. And this H N N has only one clause in each rule's antecedent. Having to discretize all the fuzzy sets in a fuzzy expert system can lead to an enormous H N N as we have seen above. It is the use of an H N N that dictates the discretization because it processes real numbers. We will obtain much smaller networks if we use fuzzy neural nets. So, let us now represent fuzzy expert systems as HFNNs.
2.2. Non-discrete fuzzy expert systems The fuzzy expert system will have one block of rules as shown in Eq. (9). The input to the system will be X = / 1 ' with final conclusion Y = B'. As in Section 2.1 we will consider two cases: (1) combine all rules into one rule; and (2) fire the rules individually and then combine their results.
2.2.1. Combine the rules We first obtain a fuzzy relation /~k to model the implication in each rule, 1 ~< k ~< n. Let Rk(X, y) = some function of Ak(X) and Bk(y) for x in [al, bl], y in [a2,b2]. For example this function could be MIN. Then we combine all the /~k,
270
J.J. Buckley, Y. Hayashi / Fuzzy Sets and Systems 71 (1995) 265 276
Input
1 ~
for y in [a2,b2],
Output
B,
(15)
where • represents the compositional rule of inference (generalized modus ponens) for the system. For example, one could have B'(y)=
sup
a I <~x<~bl
(16)
(min(A'(x),R(x,y))),
for each y in [a2, b2]. A hybrid fuzzy neural net (type FNN2), the same as this fuzzy expert system, is shown in Fig. 4. There is only one neuron with input weight equal to one. The transfer functions (which maps fuzzy sets into fuzzy sets) inside the neuron is the fuzzy relation/~. So, we have input ,4' to the neuron with its output B' = A ' . R .
We obtain the simplest possible H F N N for the fuzzy expert system. The major drawback is that at this time there is no hardware available to implement fuzzy neural nets.
Fig. 5. Hybrid fuzzy neural net as a fuzzy expert system (fire rules first).
transfer functions for the neurons in the hidden layer. The input signals to the output neuron are the B~, which are aggregated by 0 (Eq. (18)) for input to the output neuron. The transfer function in the output neuron is f ( x ) = x (no change) so the final output is B'. Again, H F N N is much smaller than the HNN in Fig. 3. However, there is no hardware available at this time to build the H F N N in Fig. 5.
2.2.2. Fire each rule
We first compose ,4' with each/~k to get B~,, the conclusion of the kth rule, and then combine all the B~, into one final conclusion B'. Let B~, be defined as (17)
B'k(y) = A ' ( x ) * R k ( X , y )
for y in [az, b2] and * the compositional rule of inference. Then /~'(y) = 0(/~'1(y). . . . . /~',(y))
(18)
for some aggregating function 0. An H F N N (type FNN2) the same as this fuzzy expert system is displayed in Fig. 5. All the weights are equal to one and the fuzzy relations/~k are the
3. Fuzzy controller In [5, 6] the author identifies three basic types of elementary fuzzy controllers: (1) Sugeno; (2) expert system; and (3) Mamdani. For another survey of elementary fuzzy controllers see [69]. We will now show how to build a hybrid fuzzy neural net to be computationally identical to each of these three types of fuzzy controller (see footnote 1). 3.1. Sugeno
We will first discuss how the Sugeno controller operates [69, 71, 79-83]. The fuzzy control rules are of the type ~:
A'
1@
~B'
Fig. 4. Hybrid fuzzy neural net as a fuzzy expert system (combine the rules).
If error =/i~ and change in error = B~, thenz=aie+fli(Ae)+7i,
1 <~ i <~ n,
(19)
where e = error and Ae = change in error. The Ai, Bi, cq, fl~, and 7~ are all given. The input to the controller is values for e and Ae and one first
J.J. Buckley, E Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
3.2. Expert system controller
evaluates each rule's antecedent as follows
tr~ = T (Ai(e), Bi(Ae)),
1 <~i <. n,
(20)
where T is some t-norm. Next, we evaluate each conclusion given e and Ae as
This fuzzy controller was based on the fuzzy expert system shell called F L O P S [23,24]. The fuzzy control rules are ~:
zi=~ie+fli(Ae)+Ti,
1 ~i~n.
aizi i=1
~ a,.
(22)
i=1
A hybrid fuzzy neural net (type FNN1) computationally equivalent to the Sugeno controller is displayed in Fig. 6. For simplicity, we have assumed only 2 rules in Fig. 6. First consider the first hidden layer. The inputs to the top two neurons are cqe + fll(Ae) and ~2e + fl2(Ae), respectively. The transfer function in these two neurons is f ( x ) = x + 7~, i = 1, 2, so the outputs are zl. A1 other neurons in this net have f ( x ) = x. The inputs to the bottom two nodes is T(fiq(e),/7i(Ae)) where: (1) the interaction between e (Ae) and ,4~ (Bi) is s/i(e) (B~(Ae)); and (2) T is used to aggregate these values for input to the neuron. In the rest of the net all weights are equal to one. What happens at the next four neurons is self-explanatory because we either multiply, or add, signals. The output produces 3 because we aggregate the two input signals using division (el + e2)/(O'i + 0"2).
Input / / &
If error = ,4~ and change in error = Bi,
(21)
The output is
6=
271
then control action = C~,
1 ~< i ~< n.
(23)
We assume that the C'~ are triangular shaped fuzzy numbers so let C~(x)= 1 if and only if x = cg, 1 ~< i ~< n. Given input values e = error and Ae = change in error, each rule is evaluated producing the tri given in Eq. (20). Then ai is assigned to the rule's consequence C~ and the controller takes all the data (O'i, C i ) , 1 ~< i ~< n, and defuzzifies to output 6. Let
6= ~
trici / ~,
i=1
/ i=1
ai.
(24)
A hybrid fuzzy neural net (type FNN~) computationally identical to this controller is shown in Fig. 7. Again, for simplicity we assume only two control rules in Fig. 7. The operations in this hybrid net are similar to those in Fig. 6.
3.3. Mamdani This was the first fuzzy controller I-1,73]. The fuzzy control rules are like those in Eq. (23) and they are evaluated like Eq. (20) given input values
~~'
Fig. 6. Hybrid fuzzy neural net as a Sugeno controller.
272
J.J. Buckley, Y. Hayashi / Fuzzy Sets and Systems 71 (1995) 265 276
Input e
c
Fig. 7. Hybrid fuzzy neural net as a fuzzy expert system controller.
e and Ae. However, we will adopt a more general approach to this type of controller that has been presented in [75]. This controller is based on some type of approximate reasoning (like in a fuzzy expert system) to evaluate the rules (Eq. (23)) given input values of e and Ae. Let /~(Z) = @1(,4i(e), Bi(Ae), Ci(z)),
-~'-~
-J--
- '-----~Pl(z])
zae Z1
(25)
where @1 is some general AND operator, for 1 <~ i ~< n. We are assuming that there is an interval [a3, b3] that contains the support of all the Ci and (7 (to be the fuzzy set output from the controller). Then Eq. (25) specifies /~/(z) for z in [a3,b3], 1 ~< i <~ n. Next define (7(z) = @2(/](z) l 1 ~< i ~< n),
Input e
Zp
--
"
(Ze) 8
(26)
where @2 is some general OR operator. (7 is the fuzzy set output from the controller. One must now defuzzify (7 into real numbers ~ for input to the process. We will discuss defuzzification at the end of this section. We now wish to construct a H F N N to represent this controller which maps the real number pair (e, Ae) into fuzzy set (7. See also [86]. Let us first consider a net to obtain a discrete approximation to (7 which will be very useful for the defuzzification process.
3.3.1. Discrete (7 Let Po be a positive integer and set P = Po + 1. Choose zl = a3 and zi = a3 + (i - 1)(b3 - a 3 ) / P o , 2 ~< i ~< P. We will compute (7(zl), ..., C(zp). A H F N N (type F N N 0 for this controller is shown in Fig. 8. The input neurons make no cha-
PI (Zi):
~
1
~ •
~
1 C(zi)
e.¢zi ~ Fig. 8. Hybrid fuzzy neural net as a discrete Mamdani type fuzzy controller: (a) hidden layer in the hybrid fuzzy neural net; (b) output neurons in the hybrid fuzzy neural net (1 <~ i ~< P).
nges in the input signals. In Fig. 8(a) the hidden layer is to compute/~(zi), 1 ~< i ~< P, for each rule (1 ~< k ~< n). Consider the top neuron in the hidden layer (labeled k = 1 for the first rule and i = 1 for z0. The interaction between the signals e, Ae, zl and the weights A 1, B1, C1 is ,~1 (e), B1 (Ae), C1 (zl). This data is aggregated using @1 (Eq. (25)) for input to
273
JJ. Buckley, E Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
the neuron. So, the input is Pt(zl). All the neurons in the hidden layer have f ( x ) = x (no change) for their transfer functions so the outputs from the neurons in the hidden layer are as shown in Fig. 8(a). These outputs are fed into the output neurons in Fig. 8(b). There will be P output neurons. The weights are all one and the aggregation function will be ~k2 (Eq. (26)) so the input to an output neuron is (7(zi), 1 ~< i ~< P. The transfer functions are f ( x ) = x in all the output neurons producing the final output as shown in Fig. 8(b). As in Sections 2.1.1 and 2.1.2 assume [a3, b3] = [ - 100, 100] producing P = 2001 and there are 25 (n = 25) rules. Then the H F N N in Fig. 8 has: (1) 2003 input neurons; (2) (3)(25)(2001) connections from input neurons to hidden layer neurons; (3) (25)(2001) hidden layer neurons; (4) (25)(2001) connections from the hidden layer to the output layer; and (5) 2001 output neurons. This is a very large H F N N . Alternatively, we could use an H N N as in Fig. 3 to produce a discrete approximation to (7. However, this would make an even larger net than the one in Fig. 8. The H F N N in Fig. 8 affords fast parallel computation for a Mamdani type controller and its output (discrete C) is ideal for defuzzification (discussed later in Section 3.3). However, as pointed out above it may be too large to be useful at this time. Now let us look at an H F N N that computes (7 (not discrete). 3.3.2. Non-discrete (7
The discussion here will be based on the results from Section 2.2.2. First construct a fuzzy relation /~k for each rule as RR(X,y,z) = dA(Ak(X),Bk(y),Ck(Z)),
Now the inputs are the fuzzy vector (~, Ae). Now compose the input with each/~k producing (~(x), Ae(y))*/~k (X, y, Z) = /~ (Z),
(30)
from Eq. (25). We then aggregate the Pk to get (7 as
(31)
(7(z) = O~(P,(z), ..., P.(z)),
from Eq. (26). Fig. 9 presents an H F N N (type FNN2) for this controller. All the weights are one. The input and output neurons have f ( x ) = x as their transfer function. The transfer functions in the neurons in the hidden layer are the fuzzy relations /~k which operate as in Eq. (30). The method of aggregating the incoming signals to the output neuron is ~02(Eq. (31)). So, the H F N N in Fig. 9 is this type of fuzzy controller. Obviously, the H F N N in Fig. 9 is much simpler than the one in Fig. 8 but, as mentioned before, at this time there is no hardware available to build the net in Fig. 9. 3.3.3. Defuzzify
When the controller produces a fuzzy set C it must then be defuzzified for input to the process. To use a neural net to defuzzify it will be most helpful to have a discrete approximation to (7. Assume we have (7(zi), 1 ~< i ~< P, as from the net in Fig. 8. A very common defuzzifier, the center of gravity, would be 6 = ~ zi(7(zi) i
(32)
C(zi). i=1
It would be easy to build an H N N to produce 3 given inputs (7(z~) and zi, 1 ~< i ~< P, so we will
1 <<.k <<.n.
(27) Input to the system will be real numbers e and Ae. The fuzzy sets ~, ~e are defined as follows ~(x)
= ~ )1, 0,
x = e, otherwise
Input
Output
(28)
and 1, y = Ae, A-e(Y)= 0, otherwise.
(29) Fig. 9. Hybridfuzzyneural net as a Mamdani fuzzycontroller.
274
J.J. Buckley, ~ Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
omit the details. Then this H N N would be attached to the H F N N in Fig. 8 or 9, to give the complete Mamdani type fuzzy controller.
4. Summary and conclusions The motivation for this research is to build computationally fast parallel methods for fuzzy systems. We first showed how to make a hybrid neural net equal to a discrete fuzzy expert system and then how to construct a hybrid fuzzy neural net equal to a (non-discrete) fuzzy expert system. We saw remarkable simplification in the nets when we went from real signals and weights to fuzzy signals and/or fuzzy weights. We then showed how to obtain hybrid fuzzy neural nets equal to a Sugeno, or an expert system, type of controller. We also discussed two different kinds of hybrid fuzzy neural nets for the Mamdani type of fuzzy controller. In the Mamdani case we obtained a major simplification in the net when we modeled the controller as a (non-discrete) fuzzy expert system and built a hybrid fuzzy neural net to represent it. The major problem at this time in representing fuzzy systems as fuzzy neural nets is that there is no hardware available to build these nets. Such hardware is needed for the future applications of fuzzy neural nets.
References [1] S. Assilian and E. Mamdani, An experiment in linguistic synthesis with a fuzzy logic controller, lnternat. J. Man Machine Studies 7 (1974) 1-13. [2] H.R. Berenji, Neural networks for fuzzy logic inference, Proc. 2nd IEEE Internat. Conf. on Fuzzy Systems, San Francisco (1993) Vol. II, 1395. [3] H.R. Berenji and P. Khedkar, Learning and tuning fuzzy logic controllers through reinforcements, IEEE Trans. Neural Networks 3 (1992) 724-740. [4] E.K. Blum and L.K. Li, Approximation theory and feedforward networks, Neural Networks 4 (1991) 511-515. [5] J.J Buckley, Theory of the fuzzy controller, in: C.V. Negoita, Ed., Cybernetics and Applied Systems (Marcel Dekker, New York, 1992) 293-307. [6] J.J. Buckley, Theory of the fuzzy controller: an introduction, Fuzzy Sets and Systems 51 (1992) 249-258.
[7] J.J. Buckley, Approximation paper: part I, Proc. 3rd Internat. Workshop on Neural Networks and Fuzzy Logic, NASA, Houston, TX (1992) 170-173. [8] J.J. Buckley, Approximations between nets, controllers, expert systems and processes, Proc. 2nd Internat. Conf. on Fuzzy Logic and Neural Networks, Iizuka, Japan (1992) Vol. 1, 89-90. [9] J.J. Buckley, Equivalences between nets, controllers, expert systems and processes, Proc. 1st lnternat. Conf. on Fuzzy Theory and Technology, Durham, NC (1992). [10] J.J. Buckley and E. Czogala, Fuzzy models, fuzzy controllers and neural nets, Arch. Theoret. Appl. Comput. Sci. 5 (1993) 149-165. [11] J.J. Buckley and E. Czogala, Modelling fuzzy processes using neural nets and fuzzy controllers, in: L. McAllister, Ed., Frontiers of Applied Mathematics (SIAM, Philadelphia, PA., to appear). [12] J.J. Buckley and Y. Hayashi, Fuzzy neural nets and applications, Fuzzy Systems and AI. 1(3) (1992) 11-41. [13] J.J. Buckley and Y. Hayashi, Numerical relationships between neural networks, continuous functions, and fuzzy systems, Fuzzy Sets and Systems 60 (1993) 1-8. [14] J.J. Buckley and Y. Hayashi, Hybrid neural nets can be fuzzy controllers and fuzzy expert systems, Fuzzy Sets and Systems 60 (1993) 135-142. [15] J.J. Buckley and Y. Hayashi, Can fuzzy neural nets approximate continuous fuzzy functions? Fuzzy Sets and Systems 61 (1993) 43-51. [16] J.J. Buckley and Y. Hayashi, Hybrid fuzzy neural nets are universal approximators, Proc. IEEE Internat. Conf. on Fuzzy Systems, Orlando (1994) Vol. 1,238-243. [17] J.J. Buckley and Y. Hayashi, Fuzzy neural networks, in: L.A. Zadeh and R.R. Yager, Eds., Fuzzy Sets, Neural Networks and Soft Computing (Van Nostrand Reinhold, New York, 1994) 233-249. [18] J.J. Buckley and Y. Hayashi, Fuzzy neural networks: a survey, Fuzzy Sets and Systems 66 (1994) 1-13. [19] J.J. Buckley and Y. Hayashi, Fuzzy backpropagation for fuzzy neural nets, unpublished manuscript. [20] J.J. Buckley and Y. Hayashi, Fuzzy genetic algorithm for fuzzy neural nets, unpublished manuscript. [21] J.J. Buckley, Y. Hayashi and E. Czogala, On the equivalence of neural nets and fuzzy expert system, Proc. Internat. Joint Conf. on Neural Networks, Baltimore (1992) Vol. II 691-695. [22] J.J. Buckley, Y. Hayashi and E. Czogala, On the equivalence of neural nets and fuzzy expert systems, Fuzzy Sets and Systems 53 (1993) 129-134. [23] J.J. Buckley, W. Siler and D. Tucker, Fuzzy expert system, Fuzzy Sets and Systems 20 (1986) 1 16. [24] J.J. Buckley and H. Ying, Expert fuzzy controller, Fuzzy Sets and Systems 44 (1991) 373-390. 1-25] P. Cardaliaguest and G. Euvrard, Approximation of a function and its derivative with a neural network, Neural Networks 5 (1992) 207 220. [26] N.E. Cotter and J.J. Gaillern, The CMAC and a theorem of Kolmogorov, Neural Networks 5 (1992) 221-228.
J.J. Buckley, Y. Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
[27] G. Cybenko, Approximation by Superpositions of a Sigmoidal Function, Math. Control Signals Systems 2 (1989) 303-314. [28] D. Dubois, R. Martin-Clouaire and H. Prade, Practical computing in fuzzy logic, in: M.M. Gupta and T. Yamakawa, Eds., Fuzzy Computing (North-Holland, Amsterdam, 1988) 11-34. [29] D. Dubois and H. Prade, The generalized modus ponens under sup-min composition, in: M.M. Gupta, A. Kandel, W. Bandler and J.B. Kisha, Eds., Approximate Reasoning in Expert Systems (Elsevier, Amsterdam, 1985) 217-232. [30] B. Freisleben and T. Kunkelmann, Combining fuzzy logic and neural networks to control an autonomous vehicle, Proc. 2nd lEEE lnternat. Conf. on Fuzzy Systems, San Francisco (1993) Vol. I, 321-326. [31] S. Geva and J. Sitte, A constructive method for multivariate function approximation by multilayer perceptrons, IEEE Trans. Neural Networks 3 (1992) 621-623. [32] M.M. Gupta and M.B. Gorzalczany, Fuzzy neuro-computational technique and its application to modelling and control, Proc. IEEE Internat. Joint Conf. on Neural Networks, Singapore (1991) Vol. 2, 1454-1457. [33] Y. Hayashi, A neural expert system using fuzzy teaching input, Proc. IEEE Internat. Conf. on Fuzzy Systems, San Diego (1992) 485-491. [34] Y. Hayashi and J.J. Buckley, Fuzzy controllers and fuzzy expert systems as hybrid neural nets, Proc. 5th IFSA World Congr., Seoul, Korea (1993) Vol. I, 70 72. [35] Y. Hayashi and J.J. Buckley, Are regular fuzzy neural nets universal approximators? Proc. Internat. Joint Conf. on Neural Networks, Nagoya, Japan (1993) Vol. I, 721 724. [36] Y. Hayashi and J.J. Buckley, Direct fuzzification of neural networks, Proc. 1st Asian Fuzzy Systems Symposium, Singapore (1993) 560-567. [37] Y. Hayashi and J.J. Buckley, Approximations between fuzzy expert systems and neural networks, lnternat. J. Approx. Reasoning 10 (1994) 63-73. [38] Y. Hayashi and J.J. Buckley, Fuzzy max-min neural controller, unpublished manuscript. [39] Y. Hayashi, J.J. Buckley and E. Czogala, Systems engineering applications of fuzzy neural networks, J. Systems Engrg. 2 (1992) 232-236. [40] Y. Hayashi, J.J. Buckley and E. Czogala, Fuzzy neural controller, Proc. IEEE lnternat. Conf. on Fuzzy Systems, San Diego (1992) 197-202. [41] Y. Hayashi, J.J. Buckley and E. Czogala, Fuzzy expert systems versus neural networks, Proc. Internat. Joint Conf. on Neural Networks, Baltimore (1992) Vol. II 720-726. [42] Y. Hayashi, J.J. Buckley and E. Czogala, Fuzzy neural network with fuzzy signals and weights, Proc. Internat. Joint Conf. on Neural Networks, Baltimore (1992) Vol. II, 696-701. [43] Y. Hayashi, J.J. Buckley and E. Czogala, Systems engineering applications of fuzzy neural networks, Proc. Internat. Joint Conf. on Neural Networks, Baltimore (1992) Vol. II, 413 418.
275
[44] Y. Hayashi, J.J. Buckley and E. Czogala, Direct fuzzification of neural networks and fuzzified delta rule, Proc. 2nd lnternat. Conf. on Fuzzy Logic and Neural Networks, Iizuka (1992) Vol. I, 73-76. [45] Y. Hayashi, J.J. Buckley and E. Czogala, Approximation between fuzzy expert systems and neural networks, Proc. 2nd Internat. Conf. on Fuzzy Logic and Neural Networks, lizuka, Japan (1992) Vol. I, 135-139. [46] Y. Hayashi, J.J. Buckley and E. Czogala, Fuzzy neural network with fuzzy signals and weights, Internat. J. Intell. Systems $ (1993) 527-537. [47] R. Hecht-Nielsen, Kolmogorov's mapping neural network existence theorem, Proc. IEEE 1st Internat. Conf. on Neural Networks, San Diego (1987)Vol. III, 11-14. [48] S. Horikawa, T. Furuhashi and Y. Uchikawa, Composition methods of fuzzy neural networks (Ili), Proc. 7th Fuzzy System Symp, Nagoya, Japan (1991) 493-496 (in Japanese). [49] K. Hornik, Approximation capabilities of multilayer feedforward networks, Neural Networks 4 (1991) 251-257. [50] K. Hornik, M. Stinchcombe and H. White, Multilayer feedforward networks are universal approximators, Neural Networks 2 (1989) 359-366. [51] H. Ishibuchi, R. Fujioka and H. Tanaka, An architecture of neural networks for input vectors of fuzzy numbers, Proc. IEEE Internat. Conf. on Fuzzy Systems, San Diego (1992) 1293-1300. [52] H. Ishibuchi, R. Fujioka and H. Tanaka, Neural networks that learn from fuzzy if-then rules, IEEE Trans. Fuzzy Systems 1 (1993) 85-97. [53] H. Ishibuchi, K. Kwon and H. Tanaka, Learning of fuzzy neural networks from fuzzy inputs and fuzzy targets, Proc. 5th IFSA World Congr., Seoul, Korea (1993) Vol. I, 147 150. [54] H. Ishibuchi, K. Kwon and H. Tanaka, Implementation of fuzzy if-then rules by fuzzy neural networks with fuzzy weights, Proc. 1st European Congr. on Fuzzy and Intelligent Technologies, Aachen, Germany (1993) Vol. 1,209-215. [55] H. Ishibuchi, H. Okada and H. Tanaka, Interpolation of fuzzy i~then rules by neural networks, Proc. 2nd lnternat. Conf. on Fuzzy Logic and Neural Networks, Iizuka, Japan (1992) 337-340. [56] H. Ishibuchi, H. Okada and H. Tanaka, Learning of neural networks from fuzzy inputs and fuzzy targets, Proc. Internat. Joint Conf. on Neural Networks, Beijing (1992) Vol. II1, 447-452. [57] H. Ishibuchi, H. Okada and H. Tanaka, Fuzzy neural networks with fuzzy weights and fuzzy biases, Proc. IEEE lnternat. Conf. on Neural Networks, San Francisco (1993) Vol. III, 1650-1655. [58] Y. Ito, Approximation of functions on compact sets by finite sums of a sigmoid function without scaling, Neural Networks 4 (1991) 817-826. [59] Y. Ito, Approximation of continuous functions on R ~ by linear combinations of shifted rotations of a sigmoid function with and without scaling, Neural Networks 5 (1992) 105 115.
276
J.J. Buckley, Y. Hayashi / Fuzzy Sets and Systems 71 (1995) 265-276
[60] J.-S.R. Jang and C.T. Sun, Functional equivalence between radial basis function networks and fuzzy inference systems, IEEE Trans. Neural Networks 4 (1993) 156 159. [61] J.M. Keller and H. Tahani, Implementation of conjunctive and disjunctive fuzzy logic rules with neural networks, Internat. J. Approx. Reasoning 6 (1992) 221 240. [62] J.M. Keller and H. Tahani, Backpropagation neural networks for fuzzy logic, Inform. Sci. 62 (1992) 205-221. [63] J.M. Keller, R.R. Yager and H. Tahani, Neural network implementation of fuzzy logic, Fuzzy Sets and Systems 45 (1992) 1-12. [64] B. Kosko, Neural Networks and Fuzzy Systems (PrenticeHall, Englewood Cliffs, N J, 1992). [65] V.Y. Kreinovich, Arbitrary nonlinearity is sufficient to represent all functions by neural networks: a theorem, Neural Networks 4 (1991) 381-383. [66] P.V. Krishnamraju, J.J. Buckley, K.D. Reilly and Y. Hayashi, Genetic learning algorithms for fuzzy neural nets, Proc. IEEE Internat. Conf. on Fuzzy Systems, Orlando (1994) Vol. 3, 1969 1974. [67] V. Kfirkovh, Kolmogorov's theorem is relevant, Neural Comput. 3 (1991) 617-622. [68] V. KSrkovh, Kolmogorov's theorem and multilayer neural networks, Neural Networks 5 (1992) 501-506. [69] C.C. Lee, Fuzzy logic in control systems: fuzzy logic controller, Part II, IEEE Trans. Systems Man Cybernet. 20 (1990) 419-435. [70] C.-J. Lin and C.-C. Jou, A fuzzy-neural hybrid system for identifying nonlinear dynamic systems, Proc. Internat. Joint Conf. on Neural Networks, Beijing (1992) Vol. Ill, 625-630. [71] M. Maeda and S. Murakami, A design for a fuzzy logic controller, Inform. Sci. 45 (1988) 315 330. [72] P. Magrez and P. Smets, Fuzzy modus pones: a model suitable for applications in knowledge-based systems, Internat. J. Intelligent Systems 4 (1989) 181-200. [73] E.H. Mamdani, J.J. Ostergaard and E. Lembessis, Use of fuzzy logic for implementing rule-based controllers for industrial processes, in: H.-J. Zimmermann, L.A. Zadeh and B.R. Gaines, Eds., Fuzzy Sets and Decision Analysis (North-Holland, Amsterdam, 1984) 307-323.
[74] D. Nauck and R. Kruse, A fuzzy neural network learning fuzzy control rules and membership functions by fuzzy error backpropagation, Proc. IEEE lnternat. Conf. on Neural Networks, San Francisco (1993) Vol. I|, 1022-1027. [75] H.T. Nguyen and V. Kreinovich, On approximation of controls by fuzzy systems, Proc. 5th 1FSA World Congr., Seoul, Korea (1993) Vol. II, 1414-1417. [76] J. Park and I.W. Sandberg, Universal approximation using radial-basis-function networks, Neural Comput. 3 (1991) 246-257. [77] C. Posey, A. Kandel and G. Langholz, Fuzzy hybrid systems, in: A. Kandei and G. Langholz, Eds., Hybrid Architectures for Intelligent Systems (CRC Press, Boca Raton, Florida, 1992) 174 196. [78] P.K. Simpson, Fuzzy min-max Neural networks for function approximation, Proc. IEEE Internat. Conf. on Neural Networks, San Francisco (1993) Vol. III, 1967-1972. [79] M. Sugeno, An introductory survey of fuzzy control, Inform. Sci. 36 (1985) 59-83. [80] M. Sugeno and G.T Kang, Fuzzy modeling and control of multilayer incinerator, Fuzzy Sets and Systems 18 (1986) 329-346. [81J M. Sugeno and G.T. Kang, Structure identification fuzzy model, Fuzzy Sets and Systems 28 (1988) 15-33. [82] M. Sugeno and K. Murakami, An experimental study on fuzzy parking control using a model car, in: M. Sugeno, Ed., Industrial Applications of Fuzzy Control (NorthHolland, Amsterdam, 1985) 125-138. [83] M. Sugeno and M. Nishida, Fuzzy control of model car, Fuzzy Sets and Systems 16 (1985) 103-113. [84] H. Takagi and I. Hayashi, NN-driven fuzzy reasoning, Internat. d. Approx. Reasoning 5 (1991) 191-212. [85] P.J. Werbos, Neurocontrol and fuzzy logic: connections and designs, Internat. J. Approx. Reasoning 6 (1992) 185-219. [86] R.R. Yager, Implementing fuzzy logic controller using a neural network framework, Fuzzy Sets and Systems 48 (1992) 53-64.