- Email: [email protected]
Solving fuzzy equations using neural nets
FUZZY
sets and systems ELSEVIER
Fuzzy Sets and Systems 86 (1997) 271-278
Solving fuzzy equations using neural nets J a m e s J. B u c k l e y a'*, E s f a n d i a r E s l a m i b, Y o i c h i H a y a s h i c a Mathematics Department, University of Alabama at Birmingham, Birmingham, AL 35294, USA bFaculo, o/' Mathematics and Computer Science, Kerman University, Kerman, Iran CDepartment of Computer Science, Meiji Universio', 1-1-1 Hi~lashi-Mita, Tama-ku, Kawasaki 214-71, Japan Received June 1995; revised December 1995
Abstract In this paper we show how a neural net can be used to solve A3~ C', for X, even though for some values of A and there is no fuzzy arithmetic solution for X. The neural net solution is identified with our new solution [6] to fuzzy equations. @ 1997 Elsevier Science B.V.
Keywords: Algebra; Fuzzy numbers; Neural nets
1. Introduction
[a, b] (on [b, c]). An a-cut of fuzzy set A is defined as
Let us first introduce the notation we will be using in the paper and then we will discuss what we mean by neural net solutions to fuzzy equations. We will place a bar over a capital letter to denote a fuzzy subset of the real numbers. So ~i,)(, I7, C, etc. all represent fuzzy subsets of the real numbers. If ii is a fuzzy set, then we write /~(x ]~i) to be its membership function. Therefore,/~(x I ~i) belongs to [0, 1] for all x. A fuzzy set N is a triangular fuzzy number if the graph o f y =/~(x ] N ) is a triangle with base on some interval [a, c] and vertex at x - b for a < b < c. We denote triangular fuzzy numbers N as (a/b/c). is a triangular shaped fuzzy number if the graph of y - l~(xlN ) is: (1) zero outside some interval [a, c] a n d y = 1 if and only i f x = b f o r a < b < c ; and (2) continuous and monotonically increasing (decreasing) from zero to one (from one to zero) on
3(~) = {x : #(x 13)>~},
* Corresponding author. 0165-0114/97/$17.00 (~) 1997 Elsevier Science B.V. All rights reserved Pll S 0 1 6 5 - 0 1 1 4 ( 9 6 ) 0 0 0 0 8 - 5
(1)
for 0 < ~ < 1. A(0) (the support of,4) is the closure of the union of A(~), 0 < ~ < 1. If N is a fuzzy number (triangular, triangular shaped) then N(0) = [a,c] a closed, bounded, interval. ,4 ~ 0 if a > 0; (2) N>~0 if a~>0; (3)/V < 0 when c < 0; and (4) N~<0 when c~<0. In this paper we wish to solve the fuzzy linear equation A)( = C,
(2)
for )(, given triangular fuzzy numbers/] -- ( a l / a 2 / a 3 ) and C = (cl/ce/c3). We know [5], for certain values of ,4 and C, Eq. (2) has no solution for )7. That is, for some triangular fuzzy numbers A and (~ there is no fuzzy set )( so that, using regular fuzzy arithmetic, A X is exactly equal to C. The objective of this paper
J..L Buckle)' et al./ Fuzzy Sets and Systems 86 (1997) 271 278
272
is to train a neural net to solve Eq. (2) for )?, for all :i and C, when zero does not belong to the support of/i. In the next section we will discuss two neural net "solutions" which we call 17 and ):*. I7 will be a neural net output when we put no restrictions on the weights in the net. We will show, using computer experiments, in the third section that there does not appear to be any relationship between 17 and k (solutions to Eq. (2) when they exist). )?* is the neural net solution when we place certain sign restrictions on the weights. We argue in the next section that )?* is an approximation to what we have called the new solution to fuzzy equations [1-4, 6]. We show that (approximately)): ~< )~*, whenever ): exists. The third section contains examples of our neural net "solutions" 17 and )?*, compared to )( when it exists, for selected values of/i, C in [0, 1],/i > 0 and ~>0. The last section contains a brief summary and suggestions for future research.
2. Neural net solutions
In this section we will always assume that /i = (al/az/a3), /i > O, C = (Cl/C2/C3) and C~>0. The
other cases (/i < 0 and C/> 0, etc.) are in Appendix B. We know from [5] that Eq. (2) has a solution for X, a triangular shaped fuzzy number, if and only if alc2 > cla2 and a3c2 < c3a2. Let /i(e) = [al (e), a2(cQ], C ( e ) = [cl (e), c2(e)] and (if it exists) )?(c~) = [xl(e),x2(e)], 0~
y = ~
vif(wia + uic q- Oi),
(3)
i=1
for bias terms Oi and sigmoidal f ( x ) = (1 + e-X) - t . The wi, ui, vi are called the weights of the network and can be any real number. The input neurons just distribute the input signals (a, c) to the neurons in the second layer. The input to node number 1 in the second layer is wl a + ul c + 01 with output f(Wl a + UlC + 01 ). This is the same for nodes number 1 through number m in the second layer. Therefore, the input to the output node, which is the same as its output y, is given by Eq. (3).
wl
a
Y Vm C
Fig. 1. N e u r a l net a r c h i t e c t u r e to s o l v e Eq. (2).
For a > 0, c >~ 0, a and c in some interval [0,M], we train this net so that y ~ c/a. First, assume that the training is done with no restrictions on the weights. That is, the wi, ui, vi can be positive, zero, or negative. After training we input ~i > 0, C ~>0,/i and C" in [0, M] and obtain output I7. Actually, we do not input a whole triangular fuzzy number but instead we input e-cuts of zi and C (intervals), do interval arithmetic within the net [7], to get a-cuts of I7 as output. So, we input [aa(c0,a2(e)] an e-cut of~i, [cl(e),c2(e)] an a-cut of C, and obtain [yl(e),y2(e)] an a-cut of I7, a triangular shaped fuzzy number. Computing y i ( e ) is some what complicated and is presented in Appendix A. We are unable to argue that i? represents any solution to Eq. (2). Comparisons between 17 and k (when it exists) are presented in the next section. We now argue that in order to obtain some type of solution to Eq. (2) using a neural net we need to put sign constraints on the weights. The type of sign constraints depends on the signs of ,4 and C. The full details are in Appendix B. Here we only consider /i > 0, C"~>0, both in [0,M]. Train the net in Fig. 1, with inputs a > 0, c/> 0, both in [0, M], so that y ~ c/a but with weight constraints (Table B. 1, Appendix B) wi < 0 ,
ui>>-O,
vi>-O,
(4)
ui < O,
vi < 0,
(5)
or wi >~0,
J.J. Buckley et al./Fuzzy Sets and Systems 86 (1997) 271 278 l ~< i ~ m. We would only use Eq. (4) because then the net output y>~0 since it is to approximate (c/a)>~O. Assuming this can be done, now input :~-cuts of A and giving (see also Appendix A) yl(3~) = YI4 = ~ vif(wia2(o{) Jr- UiCI(0{) -- Oi), (6) I--I t#l
Y2(~) = Y24 = Z
vif(wial(ot) + uic2(~) + Oi), (7)
i--1
273
Theorem 1. (1) 2 " * ( ~ ) = ~2(~)for all ~. (2) 2"* = 2. (3) 2~< 2"*.
Proof. ( 1 ) Clearly, Q(~) = [Cl (3{)/a2(g), c2(g)/al (0~)], 0~<~< 1. Hence )(**(cQ = f2(~) for all cc (2) The proof of 2 ( : 0 = ~ ( e ) , for all ~, can be found in [1] (or [2 4]) and is omitted. Since 2 ( ~ ) = f)(:~) = k * * ( ~ ) for all ~, we get )(** = 2. (3) This is obvious since xi(~) = cl(~)/al(c~)>~ el (~)/a2(~), x2(0~) = C2( 00/a2( 0~) <~c2 ( ~()/al ( 9~).
0 ~ : ~ < 1. We obtain Eqs. (6) and (7) as follows: (1) the sign constraints of Eqs. (4) and (5) place us in F4 which is Eq. (A.4) of Appendix A; (2) this means that y l ( ~ ) is given by y14, Eq. (A.8) of Appendix A, and yz(cQ is equal to Y24, Eq. (A.12) of Appendix A; and (3) we then substitute a2 (ct) for a2, c 1(c~) for c l in Eq. (A.8) and al(cQ for al, c2(~) for cl in Eq. (A.12) to obtain Eqs. (6) and (7), respectively. But since the net was trained to approximate c/a, and the net output y is given by Eq. (3), we obtain
However, our neural net solution )(* is only an approximation to )(**. But, we will also call )(* a solution to ~i)( = C obtained from a neural net. )(* will always exist as long as zero does not belong to the support of A. We have approximately )( ~< X-*, )?*(~) = f2(~) and )(* = 2". Now, let us actually compute )(* and I7 for various values ofzi > 0, C'~>0 both in [0, 1].
yl(0Q ~ Cl(~)/a2(~),
3. Computer experiments
(8)
and y2(~) ~ c2(o~)/al(oO,
(9)
for 0~
fa(~) = {x:
a x = c, a ~ 7~(c~), c ~
C'(:0},
(10)
0 ~<:~ ~ 1. Define a fuzzy set 2. by its membership function p ( x ] Z ) = sup{c< x < (2(:Q}.
(11)
Let 3~** be the fuzzy set defined by the :~-cuts [cl(c~)/a2(~),c2(~)/m(~)], 0~
We did six computer experiments, to be discussed below, for si, C in [0, 1], ii > 0 and C" ~>0. We first trained the neural net in Fig. 1 with five hidden neurons, using the software package NNDT [8], without any sign restrictions on the weights, to approximate c/a for inputs a,c. We will briefly explain the learning algorithm in this software package at the end of this section. The training set had 50 patterns with a chosen from the set {0.1,0.2,0.25,0.3,0.4,0.5,0.6,0.75,0.8,1.0} and c from the set {0, 0.1,0.25, 0.3, 0.4, 0.5, 0.75, 0.8, 0.9, 1.0}. Not all combinations of a and c were used. The number of iterations until convergence was 40000 with root mean square error 1.425 x (10) -2 and max/[neti targetil = 0.032. From this trained net we are able to calculate l7. We cannot find 1p directly from the trained net because it is not constructed to handle interval inputs and interval arithmetic. What we did was to take the values of the weights, and bias terms, from the trained net, and separately calculate ,~-cuts of 17 from the expressions given in Appendix A. That is, we computed Yl and Ye from Eqs. ( A . 5 ) - ( A . 12) in Appendix A, for selected :~-cuts of ti and C, given the w~, ui, Oi, vi from the trained net.
274
J.J. Buckley et al./Fuzz), Sets and Systems 86 (1997) 271 278
Next we trained the net in Fig. 1 using ten hidden neurons, using [8], with the sign restrictions given in Eq. (4). Actually, we could not do these sign restrictions directly with this training program. The training program allowed us to restrict all the weights to be in [ 0 , M ] for some M > 0. To get wi < 0 in Eq. (4) we had the input n e u r o n n u m b e r 1 in Fig. 1 change the sign on a. So, the signal from input n e u r o n n u m b e r 1 to all hidden n e u r o n s was - a . The input to a hidden n e u r o n then becomes ( - a )wi + cui + Oi = a ( - wi ) + cui + Oi a(w[)+cui+Oi with weight w/* < 0, 14~'*i ~ --14'i~ 14'i 0. That is, we transferred the negative sign from the signal to the weight m a k i n g a > 0 and the weight ~i,, < 0. In this w a y we were able to impose the sign restrictions on the weights given in Eq. (4). The net was trained so that it approximates c/a for inputs a,c. The training set had 82 patterns with a = 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0 and c was usually 0 . 1 , 0 . 2 , . . . , 0.9, 1.0. In four cases the total values for c, always selected from the set {0.1,0.2 . . . . . 1.0}, were 8, 7, 9, 8 g i v i n g the total o f 82 in the training set. The learning algorithm converged at 10, 164 iterations with root m e a n square error 8.291 × (10) -3 and max~lnet~ - target~l = 0.04. Actually, ]netz - target~l was usually zero to two decimal places. W e were able to use the trained net directly, using Eqs. (6) and (7), to calculate ~-cuts o f J(*. Case 1: A = (0.2/0.4/0.6), C (0.1/0.3/0.5). Table 1 presents some z-cuts o f ) ( , . g * , and 17. In this case i i 2 = C has a solution for X. Alfa-cuts o f >i are i i ( c 0 = [ a l ( z ) , a 2 ( z ) ] for r e ( z ) = 0.2 + 0.2z, a2(z) = 0.6 - 0.2z and alfacuts o f C" are C'(z) = [cl(c~),c2(z)] with c i ( ~ ) = 0.1 + 0.2z, c2(~) = 0.5 - 0.2z. We k n o w that 2 ( z ) = [Cl(~)/al(z),c2(z)/a2(z)] , 2 " ~ 2 * * with 2 * * ( z ) = [cl (z)/a2(z), c2(~)/aj (z)]. W e see that from Table 1 that )? ~< )(**, as predicted b y T h e o r e m 1. Table 1 Alfa-cuts of AT,)(*, l~ in Case 1
0 0.25 0.50 0.75 1.00
[0.50,0.83] [0.60, 0.82] [0.67, 0.80] [0.71,0.78] 0.75
[0.182,2.50] [0.274, 1.80] [0.392, 1.34] [0.550, 1.00] 0.752
[ 71.27, 71.871 [ - 33.05, 34.03] -0.7723
Table 2 Alfa-cuts of 3~*, 17 in Case 2
0 0.25 0.50 0.75 I
[I.00,4.82] [1.18, 3.77] [I.40, 3.01] [I .67, 2.43] 1.99
[--60.56,65.37] [ 27.39,31.11] 1.9881
Table 3 Alfa-cuts of J(,ff* in Case 3
0 0.25 0.50 0.75 1.00
[0.500, [0.588, [0.667, [0.737, 0.800
0.875] 0.862] 0.846] 0.826]
[0.250, [0.341, [0.453, [0.607, 0.802
1.77] 1.47] 1.22] 1.00]
Also, I7 is far away from k and X* and are unable to d e d u c e any reasons for considering I7 a solution to A X = ~'. If we set 2 * * ( z ) [ssl(:O, ss2(z)], X * ( z ) = [Sl(Z),Sz(Z)] all z, then define E* to be the m a x i m u m o f Issi(z) - si(z) I for i = l, 2 and z = 0, 0.25, 0.50, 0.75, 1.00. W e see that )(* is a reasonable approximation to 2 * * since we calculated E* = 0.0l. Case 2: ,~i = (0.2/0.4/0.6), d = (0.6/0.8/'1.0). N o w ~i)( = C has no solution f o r ) ( . Table 2 presents z-cuts of)~* and l7. Alfa-cuts of~i are the same as in Case I but cl(:~) = 0.6 + 0.2z, c2(~) 1.0 - 0.2z. X* is a reasonable approximation to )(** except for the right end point o f the z 0 cut. We see that Iss:(0) - s 2 ( 0 ) l = 0.18 and Issz(0.25) - s 2 ( 0 . 2 5 ) 1 = 0.03. W i t h o u t these two terms E* = 0.0 l. The reason for the relatively large error o f 0.18 is that we had no values o f a = a l ( 0 ) = 0.2 and c c : ( 0 ) = 1.0 in the training set for the neural net. I f we would retrain the net with a = 0.2, c = 1.0 we feel this error would b e c o m e much smaller. Again i~ is nowhere near )(* and cannot be considered any solution t o / i J( = C. We shall not report I~ in any further computer experiments. Case 3: A = (0.4/0.5/0.8), C = (0.2/'0.4/0.7). Table 3 has z-cuts o f J( a n d X * . In this case i i ) ( = has a solution for X.
275
J.J. Buckley et al./Fuzz)' Sets and Systems 86 (1997) 271 278
Table 4 Alfa-cuts of X* in Case 4
Table 6 Alfa-cuts of)Z,)~* in Case 6
2~ 0 0.25 0.50 0.75 1.00
[0.628, 1.99] [0.723, 1.76] [0.848, 1.56] [1.00, 1.37] 1.20
Table 5 AIfa-cuts of)(* in Case 5 A~* 0 0.25 0.50 0.75 1.00
[0.31 I, 1.20] [0.372, 1.00] [0.440, 0.848] [0.530, 0.723] 0.628
Alfa-cuts o f / ] are given by a l ( e ) = 0.4 + 0.1a, a2(:x) = 0.8 - 0.3a and o f C by c l ( ~ ) = 0.2 + 0.2a, cz(a) = 0.7 - 0.3e. We see from Table 3 t h a t X ~< 2 * and 2~* is a reasonable approximation to 27"* because E* = 0.02. C a s e 4: ~i : (0.4/0.5/0.8), C = (0.5/0.6/0.8). In this case i i ) ? = C has no solution for )(. Table 4 presents a-cuts of.g* from the trained neural net. Alfacuts o f A are the same as in Case 3 but a-cuts of~" are computed from c l ( a ) = 0.5 +0.1 a, cz(a) = 0.8 0.2e. Again, 2~* is a reasonable approximation to our new solution 2, : 2** to ~ i 2 = ~" since E* = 0.01. C a s e 5: /i = 10.5/0.8/1.0), C = (0.3/0.5/0.6). For these values o f ~i and C the equation ~i21 C has no solution for )(. Table 5 has a-cuts o f )(*. Alfacuts o f / i are given by a l ( a ) = 0.5 + 0.3~, a z ( e ) = 1.0 0.2a and for C we have c l ( a ) = 0.3 + 0.2a, cz(a) = 0.6 0.1a. These are used to obtain a-cuts o f X**. X* is again a reasonable approximation to 2** = 2 because E* = 0.01. C a s e 6 : / i = 10.5/0.8/1.0), d = (0.4/0.7/0.9). For t h i s / i and d the fuzzy linear e q u a t i o n / i X = E" has a solution for )?. Alfa-cuts o f both 2 and X* are in Table 6. Alfa-cuts of~i are the same as in Case 5 but a-cuts o f C" are c l ( a ) = 0.4 + 0.3e, cz(e) = 0.9 - 0.2a. These a-cuts are used to find 2 ( a ) and )(**(a). We
0 0.25 0.50 0.75 1.00
[0.800, 0.900] [0.826, 0.894] [0.846, 0.889] [0.862, 0.882] 0.875
[0.401, 1.79] [0.498, 1.48] [0.617, 1.23] [0.731, 1.03] 0.877
see from Table 6 that 21 ~ 2~* and )?* is a reasonable approximation to )(** having E** = 0.01. We close this section with a brief discussion o f the learning algorithm in the N N D T package [8]. Let W be a vector containing all the unknown weights and bias terms in the neural net. Superscript (t) denotes values at the tth iteration. Then W ~t+l) = W (° + 610,
(12)
for t = 0, 1 , 2 , . . . , where 61t) is the correction at the tth iteration. The value o f 6 (t) is obtained from the expression ( j ( t ) j ( t ) y 4- 21°I)6(° = - J ( t ) r ( ° ,
(13)
where j / o is the matrix o f first derivatives o f the residuals (errors) with respect to the unknowns, I is the identity matrix, r (° is the residual (error) vector, and 2 IO is the Marquardt parameter (see [8]). The Marquardt parameter is automatically adjusted during training with: (1) the method approaching the Gauss Newton procedure as 2 ~ 0; and (2) the method approaching the steepest descent method with small step length as ). ~ vc. Analytical expressions are used for the calculation o f J (see [8]).
4. Summary and conclusions In this paper we were concerned with solving ~i 21 = for 21, given triangular fuzzy numbers for .d and C. By a solution we mean find a fuzzy set )( so that i i ) ( , computed from the extension principle, is identical to C. It is known [5] that too often i i ) ( = C has no solution for )(. Our idea was to train a neural net to solve ax = c for x given real numbers a ~ 0 and c. That is, train a 2 m 1 net to approximate c / a given inputs a and c. Then input ii and C" and obtain as output an
276
J.J. Buekley et al./ Fuzz), Sets and Systems 86 (1997) 271~78
approximation to )(, a "solution" to ~i?( = C. The neural net does not know about fuzzy arithmetic so it will produce a "solution" for all A and ~'. This paper describes how we implemented this idea. Future research will be concerned with doing this for other fuzzy equations (fuzzy quadratic equation) and certain fuzzy problems. That is, we first train a neural net to solve the crisp problem and then fuzzify it to solve the fuzzy problem.
Appendix A In this appendix we compute the interval output from the neural net in Fig. 1 when we input intervals for a and c. The interval inputs will be :~-cuts o f triangular fuzzy numbers ~i and C. Let the a-cuts o f / i and C be [al, a2] and [ct, c2], respectively. For simplicity we write [11,a2] in place of[al(cQ, a2(~)] for an a-cut o f A. Also, we use [cl,c2] in place o f [cl(cQ, c2(o:)] for an ~-cut of C. Within the net we use interval arithmetic [7] to compute the output interval [y~, Y2]. We do not assume any sign restrictions on ,~i, C or on the weights in the network. Therefore, we can have ~i > 0 or ,q < 0 and C~>0 or C~<0. Let
12if(wial + uic2 + Oi),
(A.7)
):14 = Z uif(wia2 @ bliCl -]- Oi)" 1"4
(A.8)
Y13 = ~
And Y2 = Y2] + Y22 + Y23 2r- Y24, where
Y2J = Z vif(wia2 + uic2 + Oi), Fi
(A.9)
Y22 = Z vif(wial + UiCl -- Oi), Fz
(A.10)
vif(wia2 + uicl + Oi),
(A.I 1 )
vif(wial + uic2 + Oi).
(A.12)
Y23 =
Z
F:, Y24 =
Z
Y4 Let us go through in detail one of the four cases. Assume we are on paths through the network where the signs on the weights put us into F 3. Then these paths make the following contribution to the output neuron
[Y13,Y23] = ~ l)if(wi[al,a2] q- Hi[CI,C2] q- Oi). F3 (A.13)
Ft = {i: wi>~O, ui>~O, vi>-O orwi<0,
ui < O, vi < O},
(A.1)
[Y13, Y23] = Z vif([ wia2 + UiCl + 0i, F~
F2 = {i" wi >~0, ui >~0, vi < 0 orwi<0,
F3
u i < O , vi>~O},
(A.2)
u~>0, v i < 0 } .
wial + uic2 + Oi]),
(A.14)
which equals
{i'wi>~O, u i < O , vi>~O orwi<0,
First assume that we are on a path where wi < 0, ui/> 0, vi < 0. Then Eq. (A.13) becomes
(A.3)
vi[f(wia2 + bliCl @ Oi), f ( w i a l -- bliC2 -]- Oi)], 1-3 (1.15)
F4 = {i: wi>~O, ui~O}.
(A.4)
since f is monotonically increasing, and we finally obtain
Then Yl = Y]l + 3'12 + Y13 + YI4, where
Yll = Z vif(wiaj + uicl + Oi), Fi
(A.5)
vif(wia2 + uic2 + Oi),
(A.6)
Y12 = Z
F2
[~F3 cif(wial q-uic2+Oi),
Z l)if(wia2 if- uicl if- Oi)] F~
(A. 16)
J.J, B u c k l e ) ' et a l . / F u z z ) '
for [YI3, Y23] because ui < 0 and f is positive. It is now easy to see that for the other case wi >~O, ui < O, vi>~O in F3 we get the same result shown in Eq. (A. 16). This justifies Eqs. (A.7) and (A. l l ).
Appendix B In this appendix we present theoretical justifications for imposing sign restrictions on the weights, in the neural network in Fig. 1, to obtain a solution to/] )( = C. We also discuss the capabilities, and applications, o f networks with sign restrictions on their weights. Finally, we show how we implement sign restrictions, on our software, for training neural nets. We will consider the four cases: ( l ) 2 > O, C ~>0; ( 2 ) 2 < 0 , C~<0; ( 3 ) 2 > 0 , d~<0; ( 4 ) 2 < 0 , C'>~0. C a s e l : A > 0 , C~>0. If the inputs are a > 0, c>/0 in the 2 - m - 1 net of Fig. 1 let y be the output. From Appendix A we have the four /'i, 1 ~< i ~< 4, types of sign restrictions on the weights. Table B.I gives the modeling possibilities for each /"i. In Table B.1, the + ( - ) means a positive (negative) derivative. For example consider F3. The row marked F3 in Table B. 1 means that the weights (wi, u~, vi) have sign restrictions as indicated in Appendix A. The output y is given by Eq. (3) in the paper with sum on i in F3. Let Ai = wia 4- uic AF 0 i. Then we see that ~y/[)a = Z v i w i f ( A i ) ( l
f(Ai)),
(B.1)
viuif(Ai)( 1 - f ( A i ) ) .
(B.2)
F~
?:y/?~c - Z F~
It follows, due to the sign restrictions in F3, and the fact that f ( A i ) E (0, 1 ), that Oy/~a >~0 and @/Oc <~O. The Model column in Table B.1 gives the elementary functions o f a and c that the net can possibly Table B.1 Modeling capabilities when a > 0, c ~>0 Signs
~y/'~a
?y,,'&"
Model
F~
+
+
z
F3
+
-
Z
H/C, a
--
c
F4
-
+
Z =
c/a~ C --
a
=
ac, a
-- c
z -- 1~(at),
F2
a -- c
277
S e t s a n d S y s t e m s 86 ( 1 9 9 7 ) 271 2 7 8
model (approximate), given the sign restrictions in the Fi, 1 ~< i~< 4. Continuing in the F3 row we see that c~z/~a > 0 and ~z/Oc < 0 if z - a/c or z = a - c, and a > 0, c ~>0. The signs on the partials of z match the signs on the partials of y, so the net may be able to approximate these elementary functions of a and c. As an application of Table B.1 let us assume we wish to model 1/(A C) for triangular fuzzy numbers ~,i and C (zero is not in the support of C). Train the net so that the weights have the sign restrictions in F2 and y ~ l/ca for inputs a > 0, c > 0 in some interval [0, M]. We would use only wi < O, ui < O, vi >~0 making net output and 1/(ac) both positive. Next input ~-cuts [al(c0, a2(a)], [cl(~),c2(a)] for A and C also in [0,M], respectively. We have assumed A > 0, (~ > 0. From Appendix A, we get Yl = YI2 ~ ( a 2 ( a ) e 2 ( a ) ) - 1 ,
(B.3)
Y2 - Y22 ~ ( a l ( a ) c l ( a ) )
(B.4)
1,
which are the a-cuts o f ( A ~ ) - 1 . In our software package [8] for training a neural net we had the option of putting an interval restriction on all the weights simultaneously. That is, we could restrict all the weights to be in [0, 106], or all the weights in [ - 106, 0], or all the weights in [ - 106, 106]. To obtain the restrictions wi < 0, ui >~O, vi >>.0 in F4 discussed in the paper we had neuron number 1 in the input layer change the sign of a and restrict all weights to be in [0, 106]. So if Ai = wia + uic + Oi in Eq. (3) for the nets output, we first get Ai = w i ( - a ) + uic + Oi and then Ai = ( - w i ) a z_ uic 4- Oi and - w i < 0. That is, we transfer the negative sign from ( - a ) to wi so that now wi is negative. Case 2: A < 0, C~<0. Inputs to the net (Fig. 1) are a < 0, c~<0 with output y. The explanation o f the signs of the derivatives is the same as in the previous case. The Model column changes because a < 0, c ~<0 (Table B.2). If we wished to approximate C/A (to solve 2 ) ( = C) we would impose the sign restrictions in /"3 on the weights and train the net for y ~ c/a. Use only wi ~>0, ui < 0, vi ~>0 so that both net output and c/a are positive. Theninput a-cuts of.d and C" to approximate a-cuts o f C/A which are [c2(a)/at(a),ct(a)/a2(a)]. We see that this approximates our new solution to A 2 = C for~i < 0 , C'~<0.
278
J.J. Buckles' et a l . / F u z z ) ' S e t s a n d S:vstems 8 6 ( 1 9 9 7 ) 271 2 7 8
Table B.2 Modeling capabilities when a < 0, c~<0
Table B.4 Modeling capabilities for a < 0, c>~0
Signs
d3'/~a
~y/&'
Model
Signs
?y,,'~a
?y/'&'
Model
l'l
+
+
z = 1/'ac, a + c
l'l
+
+
z
1"2
--
z
-+
z z--
['2 F3
+
-
Z = ac, - - a -- C z -- c / a , a c
F4
-
--
2
--
a/c, c
a
Table B.3 Modeling capabilities for a > 0, c~<0 Signs
?y,,'i~a
i?y,/&.
Model
FI
+
+
z = c/a,a + c
I"2
1"3 ['4
z - a/c,-a
+
z+
z
- c
I/ac, a - c :
ac,
c -
a
To use our neural net software [8] we would constrain all the weights to be in [0, 106] but have neuron number 2 in the input layer change the sign on input c. The effect would be to have c ~ 0 but u~ < 0 and we obtain the sign constraints of Fs. Case 3: . 3 > 0 , C~<0. Input a > 0 , c~<0 (Table B.3) with output y. To approximate our new solution to ~ i ) ( = C we would use FI for the sign restrictions on the weights. Use w; < 0, u~ < 0, v; < 0 so that net output and c/a are both negative. Interval output from the trained net will approximate [cl(~)/al(:Q, c2(~)/a2(~)] which are :~-cuts of our new solution to A ) ( = C for ii > 0, C"~< O. N o w we can use our neural net training software directly because restricting all the weights to be in [ - 1 0 6 , 0 ] gives the correct signs in Fi.
F3 F4
+ -
a/c,a + c --
c/a,
c
a
ac, a c I/ac, c - a
Case 4: ~i < 0, C~>0. Input a < 0, c>~0 "Table B.4) to obtain output y. To approximate our new solution to ~i)( = (~ use F > Use wi>~O, u,>~O, vi < 0 making both c/a and net output negative. To use the software package (weights in [ - - 1 0 6 , 0 ] ) have both neurons change the sign on a and c. That is, the inputs to the hidden layer will be - a and - c . The effect will be to get wi>~O, ui>~O but v, < 0 which is what we need for F2.
References [1] J.J. Buckley and Y. Qu, Solving systems of fuzzy linear equations, F u z z ) , S e t s a n d S y s t e m s 43 (1991) 33-43. [2] J.J. Buckley, Solving fuzzy equations in economics and finance, F u z z y S e t s a n d S ) ' s t e m s 48 (I 992) 289-296. [3] J.J. Buckley, Solving fuzzy equations, F u z z y S e t s a n d S ) , s t e m s 50 (1992) 1-14. [4] J.J. Buckley, Joint solution to fuzzy linear programming, F u z z y S e t s a n d S ) , s t e m s 72 (1995) 215 220. [5] J.J. Buckley and Y. Qu, Solving linear and quadratic fuzzy equations, F u z z y S e t s a n d S y s t e m s 38 (1990) 43-59. [6] J.J. Buckley and Y. Qu, Solving fuzzy equations: a new solution concept, F u z z y S e t s a n d S y s t e m s 39 ( 1991 ) 291 301. [7] R.E. Moore, M e t h o d s a n d A p p l i c a t i o n s o f #lter~'al ana(vsis (SIAM, Philadelphia, 1979). [8] B. Saxen and H. Saxen, NNDT - a neural network development tool, Dept. Chemical Engineering, Abo Akademi Univ., Abo., Finland.
sets and systems ELSEVIER
Fuzzy Sets and Systems 86 (1997) 271-278
Solving fuzzy equations using neural nets J a m e s J. B u c k l e y a'*, E s f a n d i a r E s l a m i b, Y o i c h i H a y a s h i c a Mathematics Department, University of Alabama at Birmingham, Birmingham, AL 35294, USA bFaculo, o/' Mathematics and Computer Science, Kerman University, Kerman, Iran CDepartment of Computer Science, Meiji Universio', 1-1-1 Hi~lashi-Mita, Tama-ku, Kawasaki 214-71, Japan Received June 1995; revised December 1995
Abstract In this paper we show how a neural net can be used to solve A3~ C', for X, even though for some values of A and there is no fuzzy arithmetic solution for X. The neural net solution is identified with our new solution [6] to fuzzy equations. @ 1997 Elsevier Science B.V.
Keywords: Algebra; Fuzzy numbers; Neural nets
1. Introduction
[a, b] (on [b, c]). An a-cut of fuzzy set A is defined as
Let us first introduce the notation we will be using in the paper and then we will discuss what we mean by neural net solutions to fuzzy equations. We will place a bar over a capital letter to denote a fuzzy subset of the real numbers. So ~i,)(, I7, C, etc. all represent fuzzy subsets of the real numbers. If ii is a fuzzy set, then we write /~(x ]~i) to be its membership function. Therefore,/~(x I ~i) belongs to [0, 1] for all x. A fuzzy set N is a triangular fuzzy number if the graph o f y =/~(x ] N ) is a triangle with base on some interval [a, c] and vertex at x - b for a < b < c. We denote triangular fuzzy numbers N as (a/b/c). is a triangular shaped fuzzy number if the graph of y - l~(xlN ) is: (1) zero outside some interval [a, c] a n d y = 1 if and only i f x = b f o r a < b < c ; and (2) continuous and monotonically increasing (decreasing) from zero to one (from one to zero) on
3(~) = {x : #(x 13)>~},
* Corresponding author. 0165-0114/97/$17.00 (~) 1997 Elsevier Science B.V. All rights reserved Pll S 0 1 6 5 - 0 1 1 4 ( 9 6 ) 0 0 0 0 8 - 5
(1)
for 0 < ~ < 1. A(0) (the support of,4) is the closure of the union of A(~), 0 < ~ < 1. If N is a fuzzy number (triangular, triangular shaped) then N(0) = [a,c] a closed, bounded, interval. ,4 ~ 0 if a > 0; (2) N>~0 if a~>0; (3)/V < 0 when c < 0; and (4) N~<0 when c~<0. In this paper we wish to solve the fuzzy linear equation A)( = C,
(2)
for )(, given triangular fuzzy numbers/] -- ( a l / a 2 / a 3 ) and C = (cl/ce/c3). We know [5], for certain values of ,4 and C, Eq. (2) has no solution for )7. That is, for some triangular fuzzy numbers A and (~ there is no fuzzy set )( so that, using regular fuzzy arithmetic, A X is exactly equal to C. The objective of this paper
J..L Buckle)' et al./ Fuzzy Sets and Systems 86 (1997) 271 278
272
is to train a neural net to solve Eq. (2) for )?, for all :i and C, when zero does not belong to the support of/i. In the next section we will discuss two neural net "solutions" which we call 17 and ):*. I7 will be a neural net output when we put no restrictions on the weights in the net. We will show, using computer experiments, in the third section that there does not appear to be any relationship between 17 and k (solutions to Eq. (2) when they exist). )?* is the neural net solution when we place certain sign restrictions on the weights. We argue in the next section that )?* is an approximation to what we have called the new solution to fuzzy equations [1-4, 6]. We show that (approximately)): ~< )~*, whenever ): exists. The third section contains examples of our neural net "solutions" 17 and )?*, compared to )( when it exists, for selected values of/i, C in [0, 1],/i > 0 and ~>0. The last section contains a brief summary and suggestions for future research.
2. Neural net solutions
In this section we will always assume that /i = (al/az/a3), /i > O, C = (Cl/C2/C3) and C~>0. The
other cases (/i < 0 and C/> 0, etc.) are in Appendix B. We know from [5] that Eq. (2) has a solution for X, a triangular shaped fuzzy number, if and only if alc2 > cla2 and a3c2 < c3a2. Let /i(e) = [al (e), a2(cQ], C ( e ) = [cl (e), c2(e)] and (if it exists) )?(c~) = [xl(e),x2(e)], 0~
y = ~
vif(wia + uic q- Oi),
(3)
i=1
for bias terms Oi and sigmoidal f ( x ) = (1 + e-X) - t . The wi, ui, vi are called the weights of the network and can be any real number. The input neurons just distribute the input signals (a, c) to the neurons in the second layer. The input to node number 1 in the second layer is wl a + ul c + 01 with output f(Wl a + UlC + 01 ). This is the same for nodes number 1 through number m in the second layer. Therefore, the input to the output node, which is the same as its output y, is given by Eq. (3).
wl
a
Y Vm C
Fig. 1. N e u r a l net a r c h i t e c t u r e to s o l v e Eq. (2).
For a > 0, c >~ 0, a and c in some interval [0,M], we train this net so that y ~ c/a. First, assume that the training is done with no restrictions on the weights. That is, the wi, ui, vi can be positive, zero, or negative. After training we input ~i > 0, C ~>0,/i and C" in [0, M] and obtain output I7. Actually, we do not input a whole triangular fuzzy number but instead we input e-cuts of zi and C (intervals), do interval arithmetic within the net [7], to get a-cuts of I7 as output. So, we input [aa(c0,a2(e)] an e-cut of~i, [cl(e),c2(e)] an a-cut of C, and obtain [yl(e),y2(e)] an a-cut of I7, a triangular shaped fuzzy number. Computing y i ( e ) is some what complicated and is presented in Appendix A. We are unable to argue that i? represents any solution to Eq. (2). Comparisons between 17 and k (when it exists) are presented in the next section. We now argue that in order to obtain some type of solution to Eq. (2) using a neural net we need to put sign constraints on the weights. The type of sign constraints depends on the signs of ,4 and C. The full details are in Appendix B. Here we only consider /i > 0, C"~>0, both in [0,M]. Train the net in Fig. 1, with inputs a > 0, c/> 0, both in [0, M], so that y ~ c/a but with weight constraints (Table B. 1, Appendix B) wi < 0 ,
ui>>-O,
vi>-O,
(4)
ui < O,
vi < 0,
(5)
or wi >~0,
J.J. Buckley et al./Fuzzy Sets and Systems 86 (1997) 271 278 l ~< i ~ m. We would only use Eq. (4) because then the net output y>~0 since it is to approximate (c/a)>~O. Assuming this can be done, now input :~-cuts of A and giving (see also Appendix A) yl(3~) = YI4 = ~ vif(wia2(o{) Jr- UiCI(0{) -- Oi), (6) I--I t#l
Y2(~) = Y24 = Z
vif(wial(ot) + uic2(~) + Oi), (7)
i--1
273
Theorem 1. (1) 2 " * ( ~ ) = ~2(~)for all ~. (2) 2"* = 2. (3) 2~< 2"*.
Proof. ( 1 ) Clearly, Q(~) = [Cl (3{)/a2(g), c2(g)/al (0~)], 0~<~< 1. Hence )(**(cQ = f2(~) for all cc (2) The proof of 2 ( : 0 = ~ ( e ) , for all ~, can be found in [1] (or [2 4]) and is omitted. Since 2 ( ~ ) = f)(:~) = k * * ( ~ ) for all ~, we get )(** = 2. (3) This is obvious since xi(~) = cl(~)/al(c~)>~ el (~)/a2(~), x2(0~) = C2( 00/a2( 0~) <~c2 ( ~()/al ( 9~).
0 ~ : ~ < 1. We obtain Eqs. (6) and (7) as follows: (1) the sign constraints of Eqs. (4) and (5) place us in F4 which is Eq. (A.4) of Appendix A; (2) this means that y l ( ~ ) is given by y14, Eq. (A.8) of Appendix A, and yz(cQ is equal to Y24, Eq. (A.12) of Appendix A; and (3) we then substitute a2 (ct) for a2, c 1(c~) for c l in Eq. (A.8) and al(cQ for al, c2(~) for cl in Eq. (A.12) to obtain Eqs. (6) and (7), respectively. But since the net was trained to approximate c/a, and the net output y is given by Eq. (3), we obtain
However, our neural net solution )(* is only an approximation to )(**. But, we will also call )(* a solution to ~i)( = C obtained from a neural net. )(* will always exist as long as zero does not belong to the support of A. We have approximately )( ~< X-*, )?*(~) = f2(~) and )(* = 2". Now, let us actually compute )(* and I7 for various values ofzi > 0, C'~>0 both in [0, 1].
yl(0Q ~ Cl(~)/a2(~),
3. Computer experiments
(8)
and y2(~) ~ c2(o~)/al(oO,
(9)
for 0~
fa(~) = {x:
a x = c, a ~ 7~(c~), c ~
C'(:0},
(10)
0 ~<:~ ~ 1. Define a fuzzy set 2. by its membership function p ( x ] Z ) = sup{c< x < (2(:Q}.
(11)
Let 3~** be the fuzzy set defined by the :~-cuts [cl(c~)/a2(~),c2(~)/m(~)], 0~
We did six computer experiments, to be discussed below, for si, C in [0, 1], ii > 0 and C" ~>0. We first trained the neural net in Fig. 1 with five hidden neurons, using the software package NNDT [8], without any sign restrictions on the weights, to approximate c/a for inputs a,c. We will briefly explain the learning algorithm in this software package at the end of this section. The training set had 50 patterns with a chosen from the set {0.1,0.2,0.25,0.3,0.4,0.5,0.6,0.75,0.8,1.0} and c from the set {0, 0.1,0.25, 0.3, 0.4, 0.5, 0.75, 0.8, 0.9, 1.0}. Not all combinations of a and c were used. The number of iterations until convergence was 40000 with root mean square error 1.425 x (10) -2 and max/[neti targetil = 0.032. From this trained net we are able to calculate l7. We cannot find 1p directly from the trained net because it is not constructed to handle interval inputs and interval arithmetic. What we did was to take the values of the weights, and bias terms, from the trained net, and separately calculate ,~-cuts of 17 from the expressions given in Appendix A. That is, we computed Yl and Ye from Eqs. ( A . 5 ) - ( A . 12) in Appendix A, for selected :~-cuts of ti and C, given the w~, ui, Oi, vi from the trained net.
274
J.J. Buckley et al./Fuzz), Sets and Systems 86 (1997) 271 278
Next we trained the net in Fig. 1 using ten hidden neurons, using [8], with the sign restrictions given in Eq. (4). Actually, we could not do these sign restrictions directly with this training program. The training program allowed us to restrict all the weights to be in [ 0 , M ] for some M > 0. To get wi < 0 in Eq. (4) we had the input n e u r o n n u m b e r 1 in Fig. 1 change the sign on a. So, the signal from input n e u r o n n u m b e r 1 to all hidden n e u r o n s was - a . The input to a hidden n e u r o n then becomes ( - a )wi + cui + Oi = a ( - wi ) + cui + Oi a(w[)+cui+Oi with weight w/* < 0, 14~'*i ~ --14'i~ 14'i 0. That is, we transferred the negative sign from the signal to the weight m a k i n g a > 0 and the weight ~i,, < 0. In this w a y we were able to impose the sign restrictions on the weights given in Eq. (4). The net was trained so that it approximates c/a for inputs a,c. The training set had 82 patterns with a = 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0 and c was usually 0 . 1 , 0 . 2 , . . . , 0.9, 1.0. In four cases the total values for c, always selected from the set {0.1,0.2 . . . . . 1.0}, were 8, 7, 9, 8 g i v i n g the total o f 82 in the training set. The learning algorithm converged at 10, 164 iterations with root m e a n square error 8.291 × (10) -3 and max~lnet~ - target~l = 0.04. Actually, ]netz - target~l was usually zero to two decimal places. W e were able to use the trained net directly, using Eqs. (6) and (7), to calculate ~-cuts o f J(*. Case 1: A = (0.2/0.4/0.6), C (0.1/0.3/0.5). Table 1 presents some z-cuts o f ) ( , . g * , and 17. In this case i i 2 = C has a solution for X. Alfa-cuts o f >i are i i ( c 0 = [ a l ( z ) , a 2 ( z ) ] for r e ( z ) = 0.2 + 0.2z, a2(z) = 0.6 - 0.2z and alfacuts o f C" are C'(z) = [cl(c~),c2(z)] with c i ( ~ ) = 0.1 + 0.2z, c2(~) = 0.5 - 0.2z. We k n o w that 2 ( z ) = [Cl(~)/al(z),c2(z)/a2(z)] , 2 " ~ 2 * * with 2 * * ( z ) = [cl (z)/a2(z), c2(~)/aj (z)]. W e see that from Table 1 that )? ~< )(**, as predicted b y T h e o r e m 1. Table 1 Alfa-cuts of AT,)(*, l~ in Case 1
0 0.25 0.50 0.75 1.00
[0.50,0.83] [0.60, 0.82] [0.67, 0.80] [0.71,0.78] 0.75
[0.182,2.50] [0.274, 1.80] [0.392, 1.34] [0.550, 1.00] 0.752
[ 71.27, 71.871 [ - 33.05, 34.03] -0.7723
Table 2 Alfa-cuts of 3~*, 17 in Case 2
0 0.25 0.50 0.75 I
[I.00,4.82] [1.18, 3.77] [I.40, 3.01] [I .67, 2.43] 1.99
[--60.56,65.37] [ 27.39,31.11] 1.9881
Table 3 Alfa-cuts of J(,ff* in Case 3
0 0.25 0.50 0.75 1.00
[0.500, [0.588, [0.667, [0.737, 0.800
0.875] 0.862] 0.846] 0.826]
[0.250, [0.341, [0.453, [0.607, 0.802
1.77] 1.47] 1.22] 1.00]
Also, I7 is far away from k and X* and are unable to d e d u c e any reasons for considering I7 a solution to A X = ~'. If we set 2 * * ( z ) [ssl(:O, ss2(z)], X * ( z ) = [Sl(Z),Sz(Z)] all z, then define E* to be the m a x i m u m o f Issi(z) - si(z) I for i = l, 2 and z = 0, 0.25, 0.50, 0.75, 1.00. W e see that )(* is a reasonable approximation to 2 * * since we calculated E* = 0.0l. Case 2: ,~i = (0.2/0.4/0.6), d = (0.6/0.8/'1.0). N o w ~i)( = C has no solution f o r ) ( . Table 2 presents z-cuts of)~* and l7. Alfa-cuts of~i are the same as in Case I but cl(:~) = 0.6 + 0.2z, c2(~) 1.0 - 0.2z. X* is a reasonable approximation to )(** except for the right end point o f the z 0 cut. We see that Iss:(0) - s 2 ( 0 ) l = 0.18 and Issz(0.25) - s 2 ( 0 . 2 5 ) 1 = 0.03. W i t h o u t these two terms E* = 0.0 l. The reason for the relatively large error o f 0.18 is that we had no values o f a = a l ( 0 ) = 0.2 and c c : ( 0 ) = 1.0 in the training set for the neural net. I f we would retrain the net with a = 0.2, c = 1.0 we feel this error would b e c o m e much smaller. Again i~ is nowhere near )(* and cannot be considered any solution t o / i J( = C. We shall not report I~ in any further computer experiments. Case 3: A = (0.4/0.5/0.8), C = (0.2/'0.4/0.7). Table 3 has z-cuts o f J( a n d X * . In this case i i ) ( = has a solution for X.
275
J.J. Buckley et al./Fuzz)' Sets and Systems 86 (1997) 271 278
Table 4 Alfa-cuts of X* in Case 4
Table 6 Alfa-cuts of)Z,)~* in Case 6
2~ 0 0.25 0.50 0.75 1.00
[0.628, 1.99] [0.723, 1.76] [0.848, 1.56] [1.00, 1.37] 1.20
Table 5 AIfa-cuts of)(* in Case 5 A~* 0 0.25 0.50 0.75 1.00
[0.31 I, 1.20] [0.372, 1.00] [0.440, 0.848] [0.530, 0.723] 0.628
Alfa-cuts o f / ] are given by a l ( e ) = 0.4 + 0.1a, a2(:x) = 0.8 - 0.3a and o f C by c l ( ~ ) = 0.2 + 0.2a, cz(a) = 0.7 - 0.3e. We see from Table 3 t h a t X ~< 2 * and 2~* is a reasonable approximation to 27"* because E* = 0.02. C a s e 4: ~i : (0.4/0.5/0.8), C = (0.5/0.6/0.8). In this case i i ) ? = C has no solution for )(. Table 4 presents a-cuts of.g* from the trained neural net. Alfacuts o f A are the same as in Case 3 but a-cuts of~" are computed from c l ( a ) = 0.5 +0.1 a, cz(a) = 0.8 0.2e. Again, 2~* is a reasonable approximation to our new solution 2, : 2** to ~ i 2 = ~" since E* = 0.01. C a s e 5: /i = 10.5/0.8/1.0), C = (0.3/0.5/0.6). For these values o f ~i and C the equation ~i21 C has no solution for )(. Table 5 has a-cuts o f )(*. Alfacuts o f / i are given by a l ( a ) = 0.5 + 0.3~, a z ( e ) = 1.0 0.2a and for C we have c l ( a ) = 0.3 + 0.2a, cz(a) = 0.6 0.1a. These are used to obtain a-cuts o f X**. X* is again a reasonable approximation to 2** = 2 because E* = 0.01. C a s e 6 : / i = 10.5/0.8/1.0), d = (0.4/0.7/0.9). For t h i s / i and d the fuzzy linear e q u a t i o n / i X = E" has a solution for )?. Alfa-cuts o f both 2 and X* are in Table 6. Alfa-cuts of~i are the same as in Case 5 but a-cuts o f C" are c l ( a ) = 0.4 + 0.3e, cz(e) = 0.9 - 0.2a. These a-cuts are used to find 2 ( a ) and )(**(a). We
0 0.25 0.50 0.75 1.00
[0.800, 0.900] [0.826, 0.894] [0.846, 0.889] [0.862, 0.882] 0.875
[0.401, 1.79] [0.498, 1.48] [0.617, 1.23] [0.731, 1.03] 0.877
see from Table 6 that 21 ~ 2~* and )?* is a reasonable approximation to )(** having E** = 0.01. We close this section with a brief discussion o f the learning algorithm in the N N D T package [8]. Let W be a vector containing all the unknown weights and bias terms in the neural net. Superscript (t) denotes values at the tth iteration. Then W ~t+l) = W (° + 610,
(12)
for t = 0, 1 , 2 , . . . , where 61t) is the correction at the tth iteration. The value o f 6 (t) is obtained from the expression ( j ( t ) j ( t ) y 4- 21°I)6(° = - J ( t ) r ( ° ,
(13)
where j / o is the matrix o f first derivatives o f the residuals (errors) with respect to the unknowns, I is the identity matrix, r (° is the residual (error) vector, and 2 IO is the Marquardt parameter (see [8]). The Marquardt parameter is automatically adjusted during training with: (1) the method approaching the Gauss Newton procedure as 2 ~ 0; and (2) the method approaching the steepest descent method with small step length as ). ~ vc. Analytical expressions are used for the calculation o f J (see [8]).
4. Summary and conclusions In this paper we were concerned with solving ~i 21 = for 21, given triangular fuzzy numbers for .d and C. By a solution we mean find a fuzzy set )( so that i i ) ( , computed from the extension principle, is identical to C. It is known [5] that too often i i ) ( = C has no solution for )(. Our idea was to train a neural net to solve ax = c for x given real numbers a ~ 0 and c. That is, train a 2 m 1 net to approximate c / a given inputs a and c. Then input ii and C" and obtain as output an
276
J.J. Buekley et al./ Fuzz), Sets and Systems 86 (1997) 271~78
approximation to )(, a "solution" to ~i?( = C. The neural net does not know about fuzzy arithmetic so it will produce a "solution" for all A and ~'. This paper describes how we implemented this idea. Future research will be concerned with doing this for other fuzzy equations (fuzzy quadratic equation) and certain fuzzy problems. That is, we first train a neural net to solve the crisp problem and then fuzzify it to solve the fuzzy problem.
Appendix A In this appendix we compute the interval output from the neural net in Fig. 1 when we input intervals for a and c. The interval inputs will be :~-cuts o f triangular fuzzy numbers ~i and C. Let the a-cuts o f / i and C be [al, a2] and [ct, c2], respectively. For simplicity we write [11,a2] in place of[al(cQ, a2(~)] for an a-cut o f A. Also, we use [cl,c2] in place o f [cl(cQ, c2(o:)] for an ~-cut of C. Within the net we use interval arithmetic [7] to compute the output interval [y~, Y2]. We do not assume any sign restrictions on ,~i, C or on the weights in the network. Therefore, we can have ~i > 0 or ,q < 0 and C~>0 or C~<0. Let
12if(wial + uic2 + Oi),
(A.7)
):14 = Z uif(wia2 @ bliCl -]- Oi)" 1"4
(A.8)
Y13 = ~
And Y2 = Y2] + Y22 + Y23 2r- Y24, where
Y2J = Z vif(wia2 + uic2 + Oi), Fi
(A.9)
Y22 = Z vif(wial + UiCl -- Oi), Fz
(A.10)
vif(wia2 + uicl + Oi),
(A.I 1 )
vif(wial + uic2 + Oi).
(A.12)
Y23 =
Z
F:, Y24 =
Z
Y4 Let us go through in detail one of the four cases. Assume we are on paths through the network where the signs on the weights put us into F 3. Then these paths make the following contribution to the output neuron
[Y13,Y23] = ~ l)if(wi[al,a2] q- Hi[CI,C2] q- Oi). F3 (A.13)
Ft = {i: wi>~O, ui>~O, vi>-O orwi<0,
ui < O, vi < O},
(A.1)
[Y13, Y23] = Z vif([ wia2 + UiCl + 0i, F~
F2 = {i" wi >~0, ui >~0, vi < 0 orwi<0,
F3
u i < O , vi>~O},
(A.2)
u~>0, v i < 0 } .
wial + uic2 + Oi]),
(A.14)
which equals
{i'wi>~O, u i < O , vi>~O orwi<0,
First assume that we are on a path where wi < 0, ui/> 0, vi < 0. Then Eq. (A.13) becomes
(A.3)
vi[f(wia2 + bliCl @ Oi), f ( w i a l -- bliC2 -]- Oi)], 1-3 (1.15)
F4 = {i: wi>~O, ui
(A.4)
since f is monotonically increasing, and we finally obtain
Then Yl = Y]l + 3'12 + Y13 + YI4, where
Yll = Z vif(wiaj + uicl + Oi), Fi
(A.5)
vif(wia2 + uic2 + Oi),
(A.6)
Y12 = Z
F2
[~F3 cif(wial q-uic2+Oi),
Z l)if(wia2 if- uicl if- Oi)] F~
(A. 16)
J.J, B u c k l e ) ' et a l . / F u z z ) '
for [YI3, Y23] because ui < 0 and f is positive. It is now easy to see that for the other case wi >~O, ui < O, vi>~O in F3 we get the same result shown in Eq. (A. 16). This justifies Eqs. (A.7) and (A. l l ).
Appendix B In this appendix we present theoretical justifications for imposing sign restrictions on the weights, in the neural network in Fig. 1, to obtain a solution to/] )( = C. We also discuss the capabilities, and applications, o f networks with sign restrictions on their weights. Finally, we show how we implement sign restrictions, on our software, for training neural nets. We will consider the four cases: ( l ) 2 > O, C ~>0; ( 2 ) 2 < 0 , C~<0; ( 3 ) 2 > 0 , d~<0; ( 4 ) 2 < 0 , C'>~0. C a s e l : A > 0 , C~>0. If the inputs are a > 0, c>/0 in the 2 - m - 1 net of Fig. 1 let y be the output. From Appendix A we have the four /'i, 1 ~< i ~< 4, types of sign restrictions on the weights. Table B.I gives the modeling possibilities for each /"i. In Table B.1, the + ( - ) means a positive (negative) derivative. For example consider F3. The row marked F3 in Table B. 1 means that the weights (wi, u~, vi) have sign restrictions as indicated in Appendix A. The output y is given by Eq. (3) in the paper with sum on i in F3. Let Ai = wia 4- uic AF 0 i. Then we see that ~y/[)a = Z v i w i f ( A i ) ( l
f(Ai)),
(B.1)
viuif(Ai)( 1 - f ( A i ) ) .
(B.2)
F~
?:y/?~c - Z F~
It follows, due to the sign restrictions in F3, and the fact that f ( A i ) E (0, 1 ), that Oy/~a >~0 and @/Oc <~O. The Model column in Table B.1 gives the elementary functions o f a and c that the net can possibly Table B.1 Modeling capabilities when a > 0, c ~>0 Signs
~y/'~a
?y,,'&"
Model
F~
+
+
z
F3
+
-
Z
H/C, a
--
c
F4
-
+
Z =
c/a~ C --
a
=
ac, a
-- c
z -- 1~(at),
F2
a -- c
277
S e t s a n d S y s t e m s 86 ( 1 9 9 7 ) 271 2 7 8
model (approximate), given the sign restrictions in the Fi, 1 ~< i~< 4. Continuing in the F3 row we see that c~z/~a > 0 and ~z/Oc < 0 if z - a/c or z = a - c, and a > 0, c ~>0. The signs on the partials of z match the signs on the partials of y, so the net may be able to approximate these elementary functions of a and c. As an application of Table B.1 let us assume we wish to model 1/(A C) for triangular fuzzy numbers ~,i and C (zero is not in the support of C). Train the net so that the weights have the sign restrictions in F2 and y ~ l/ca for inputs a > 0, c > 0 in some interval [0, M]. We would use only wi < O, ui < O, vi >~0 making net output and 1/(ac) both positive. Next input ~-cuts [al(c0, a2(a)], [cl(~),c2(a)] for A and C also in [0,M], respectively. We have assumed A > 0, (~ > 0. From Appendix A, we get Yl = YI2 ~ ( a 2 ( a ) e 2 ( a ) ) - 1 ,
(B.3)
Y2 - Y22 ~ ( a l ( a ) c l ( a ) )
(B.4)
1,
which are the a-cuts o f ( A ~ ) - 1 . In our software package [8] for training a neural net we had the option of putting an interval restriction on all the weights simultaneously. That is, we could restrict all the weights to be in [0, 106], or all the weights in [ - 106, 0], or all the weights in [ - 106, 106]. To obtain the restrictions wi < 0, ui >~O, vi >>.0 in F4 discussed in the paper we had neuron number 1 in the input layer change the sign of a and restrict all weights to be in [0, 106]. So if Ai = wia + uic + Oi in Eq. (3) for the nets output, we first get Ai = w i ( - a ) + uic + Oi and then Ai = ( - w i ) a z_ uic 4- Oi and - w i < 0. That is, we transfer the negative sign from ( - a ) to wi so that now wi is negative. Case 2: A < 0, C~<0. Inputs to the net (Fig. 1) are a < 0, c~<0 with output y. The explanation o f the signs of the derivatives is the same as in the previous case. The Model column changes because a < 0, c ~<0 (Table B.2). If we wished to approximate C/A (to solve 2 ) ( = C) we would impose the sign restrictions in /"3 on the weights and train the net for y ~ c/a. Use only wi ~>0, ui < 0, vi ~>0 so that both net output and c/a are positive. Theninput a-cuts of.d and C" to approximate a-cuts o f C/A which are [c2(a)/at(a),ct(a)/a2(a)]. We see that this approximates our new solution to A 2 = C for~i < 0 , C'~<0.
278
J.J. Buckles' et a l . / F u z z ) ' S e t s a n d S:vstems 8 6 ( 1 9 9 7 ) 271 2 7 8
Table B.2 Modeling capabilities when a < 0, c~<0
Table B.4 Modeling capabilities for a < 0, c>~0
Signs
d3'/~a
~y/&'
Model
Signs
?y,,'~a
?y/'&'
Model
l'l
+
+
z = 1/'ac, a + c
l'l
+
+
z
1"2
--
z
-+
z z--
['2 F3
+
-
Z = ac, - - a -- C z -- c / a , a c
F4
-
--
2
--
a/c, c
a
Table B.3 Modeling capabilities for a > 0, c~<0 Signs
?y,,'i~a
i?y,/&.
Model
FI
+
+
z = c/a,a + c
I"2
1"3 ['4
z - a/c,-a
+
z+
z
- c
I/ac, a - c :
ac,
c -
a
To use our neural net software [8] we would constrain all the weights to be in [0, 106] but have neuron number 2 in the input layer change the sign on input c. The effect would be to have c ~ 0 but u~ < 0 and we obtain the sign constraints of Fs. Case 3: . 3 > 0 , C~<0. Input a > 0 , c~<0 (Table B.3) with output y. To approximate our new solution to ~ i ) ( = C we would use FI for the sign restrictions on the weights. Use w; < 0, u~ < 0, v; < 0 so that net output and c/a are both negative. Interval output from the trained net will approximate [cl(~)/al(:Q, c2(~)/a2(~)] which are :~-cuts of our new solution to A ) ( = C for ii > 0, C"~< O. N o w we can use our neural net training software directly because restricting all the weights to be in [ - 1 0 6 , 0 ] gives the correct signs in Fi.
F3 F4
+ -
a/c,a + c --
c/a,
c
a
ac, a c I/ac, c - a
Case 4: ~i < 0, C~>0. Input a < 0, c>~0 "Table B.4) to obtain output y. To approximate our new solution to ~i)( = (~ use F > Use wi>~O, u,>~O, vi < 0 making both c/a and net output negative. To use the software package (weights in [ - - 1 0 6 , 0 ] ) have both neurons change the sign on a and c. That is, the inputs to the hidden layer will be - a and - c . The effect will be to get wi>~O, ui>~O but v, < 0 which is what we need for F2.
References [1] J.J. Buckley and Y. Qu, Solving systems of fuzzy linear equations, F u z z ) , S e t s a n d S y s t e m s 43 (1991) 33-43. [2] J.J. Buckley, Solving fuzzy equations in economics and finance, F u z z y S e t s a n d S ) ' s t e m s 48 (I 992) 289-296. [3] J.J. Buckley, Solving fuzzy equations, F u z z y S e t s a n d S ) , s t e m s 50 (1992) 1-14. [4] J.J. Buckley, Joint solution to fuzzy linear programming, F u z z y S e t s a n d S ) , s t e m s 72 (1995) 215 220. [5] J.J. Buckley and Y. Qu, Solving linear and quadratic fuzzy equations, F u z z y S e t s a n d S y s t e m s 38 (1990) 43-59. [6] J.J. Buckley and Y. Qu, Solving fuzzy equations: a new solution concept, F u z z y S e t s a n d S y s t e m s 39 ( 1991 ) 291 301. [7] R.E. Moore, M e t h o d s a n d A p p l i c a t i o n s o f #lter~'al ana(vsis (SIAM, Philadelphia, 1979). [8] B. Saxen and H. Saxen, NNDT - a neural network development tool, Dept. Chemical Engineering, Abo Akademi Univ., Abo., Finland.