- Email: [email protected]
Identification of fuzzy systems by means of an auto-tuning algorithm and its application to nonlinear systems
Fuzzy Sets and Systems 115 (2000) 205–230
www.elsevier.com/locate/fss
Identi cation of fuzzy systems by means of an auto-tuning algorithm and its application to nonlinear systems Sungkwun Oha , Witold Pedryczb;∗ a Department
of Control and Instrumentation Engineering, Wonkwang University, 344-2, Shinyong-Dong, Iksan, Chon-Buk, 570-749, South Korea b Department of Electrical and Computer Engineering, University of Manitoba, Winnipeg, Manitoba, Canada R3T 2N2 Received October 1997; received in revised form May 1998
Abstract The study concerns a design procedure of rule-based systems. The proposed rule-based fuzzy modeling implements system structure and parameter identi cation in the ecient form of “IF: : : ; THEN: : :” statements, and exploits the theory of system optimization and fuzzy implication rules. Two types of methods for rule-based fuzzy modeling are studied. This classi cation concerns the form of the conclusion part of the rules that can be either constant or formed by some linear functions. Both triangular and Gaussian-like membership function are studied. The optimization hinges on an auto-tuning algorithm that covers a modi ed constrained optimization method known as a complex method. The study introduces a weighted performance index (objective function) that helps achieve a sound balance between the quality of results produced for the training and testing set. This methodology sheds light on the role and impact of di erent parameters of the model on its performance (in-particular, the mapping and predicting capabilities of the rule-based computing). The study is illustrated with the aid of c 2000 Elsevier Science B.V. All rights reserved. several representative numerical examples. Keywords: Identi cation of fuzzy model; Auto-tuning; Weighting factor; Gaussian and triangular membership functions; Optimal fuzzy model
1. Introduction In the early 1980s, the linguistic approach [12,9] and fuzzy relationship equation-based approach [8,3] were proposed as identi cation methods of fuzzy models. In the linguistic approach, Tong identi ed the gas furnace process by means of a logical examination of data [13]. Li et al. obtained good results through ∗ Corresponding author. Tel.: +1 204 474 8380; fax: +1 204 261 4639. E-mail address: [email protected] (W. Pedrycz).
the modi cation of Tong’s method [14] and also proposed the modi ed algorithm of the adaptive model based on a decision table. But the algorithm has some problems due to the computer capacity and computation time which are important, when it is applied to the high-order multivariable systems [15]. Pedrycz analyzed the identi cation of fuzzy systems from the viewpoint of linguistic implication rule modeling, using the referential-fuzzy-set concept [9]. Li et al. presented a self-learning algorithm for the simple SISO fuzzy model [15]. In the fuzzy relationship equation-based approach, Pedrycz identi ed fuzzy
c 2000 Elsevier Science B.V. All rights reserved. 0165-0114/00/$ - see front matter PII: S 0 1 6 5 - 0 1 1 4 ( 9 8 ) 0 0 1 7 4 - 2
206
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
systems, using the referential fuzzy set and Zadeh’s conditional possibility distribution, that is, the new composition rules which were made by the fuzzy relationship equations [8]. Xu constructed and identi ed the fuzzy relationship model using the referential fuzzy set theory and the self-learning algorithm [14,15]. The direct inference utilized by two methods did not perform better than the linear inference. Sugeno identi ed the structure of systems through the standard least-squares methods [11], but the structure of premises of the rules was determined more heuristically through the experience and iterative fuzzy partitioning of the input space. Sugeno also applied his method to the fuzzy identi cation of a gas furnace process, using fuzzy c-means clustering [4,16], but the method did not produce the identi cation of good performance; this could be alleviated to the use of direct linear inference [10]. In this paper, two types of fuzzy inferences are considered, that is, simpli ed (Type 1) and linear (Type 2) reasoning models. In Type 1 we are concerned with consequents regarded as single numerical quantities. Type 2 deals with linear functions viewed as the consequents of the rules. In each fuzzy inference, we consider three types of membership functions, namely Gaussian membership functions with modi able or xed slopes, and triangular membership functions. According to the proposed auto-tuning algorithm – the improved complex method, the parameters of such membership function can be easily adjusted. Furthermore, we introduce an aggregate objective function that deals with training data and testing data, and elaborate on its optimization to produce a meaningful balance between approximation and generalization abilities of the model. The proposed rule-based fuzzy modeling is carried out for time-series data for gas furnace process [2], activated sludge process in sewage treatment system [6] and trac route choices. The performance of the proposed rule-based fuzzy modeling is compared from the viewpoint of the identi cation errors with conventional fuzzy modeling [9,15,13,10,6]. 2. System modeling by means of fuzzy inference The identi cation algorithm of fuzzy model is divided into the identi cation activities of premise and
consequence parts of the rules. The identi cation at the premise level (1) selects the input variables x1 ; x2 ; : : : ; xn of the rules, and (2) determines the fuzzy partitions (Small, Large, etc.) of fuzzy spaces. This means the determination of the number of the optimal fuzzy space partitions, that is, fuzzy subspaces that determinate the number of fuzzy implication rules. The premise identi cation has to determine the membership values of fuzzy variables. The consequence identi cation embraces the following phases: (1) selection of the consequence variables of the fuzzy implication rules, (2) determination of the consequence parameters. In this paper, in order to identify the premise structure and parameters of fuzzy linguistic rules, two essential input variables of process in uenced are considered and the improved complex method which is a powerful auto-tuning algorithm is used. Furthermore, we restrict ourselves to some types of membership function such as Gaussian-like and triangular ones. The parameters of the membership functions are tuned with the help of the auto-tuning method. The parameters of the consequence part of the rules are determined using the standard least-squares method (Gaussian elimination with maximal pivoting algorithm). We also discuss a modi ed performance index (objective function) that aims at achieving a balance between approximation and prediction capabilities of the fuzzy model. 3. An algorithm of fuzzy identiÿcation In this section we elaborate on algorithmic details of the identi cation method discussing the optimization problem to the antecedent (condition part) of the rules as well as enhancements of their conclusions. 3.1. Premise identiÿcation In the premise part of the rules we con ne ourselves to Gaussian-like and triangular-type functions. This selection, even though looks somewhat limited, includes a broad range of cases that could be covered by modifying the parameters of these functions. For the triangular membership functions we have either 2 or 3 parameters, see Fig. 1, whose points can be auto-tuned (adjusted). The Gaussian type of the
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
207
Fig. 1. Triangular membership function. (a) Two fuzzy variables with two modi able parameters. (b) Three fuzzy variables with three modi able parameters.
membership function assumes the form f(x) = e−s(x−a)
2
=b 2
:
Furthermore, we consider Gaussian membership functions involving a xed slope, and assume several levels of their parametric exibility such as – xed slope of the functions, – the slopes are modi ed but kept the same for all the functions, – the slopes of the membership function can vary and will be adjusted (optimized), refer to Fig. 2. In the case of the same slope and di erent slope, these mean the same and di erent slope in each input variable, and the slope parameter of each case is autotuned according to the proposed optimization. 3.2. Consequence identiÿcation The identi cation of the conclusion parts of the rules deals with a selection of their structure (Types 1 and 2) and a determination of the respective parameters of the functions therein.
Fig. 2. Gaussian membership function. (a) Parameters of Gaussian type membership function. (b) Three fuzzy variables with three modi able parameters (s, a, and b).
3.2.1. Type 1 (Constant: consequence part) The consequence part of the simpli ed inference mechanism where the rules have constant conclusion part is given as follows. Ri : If x1 is Ai1 ; : : : ; and xk is Aik ; then y = ai :
(1)
The calculations of the numeric output of the model are carried out in the well-known form, , n ! n n X X X ∗ i ai i = ˆi ai ; y = i=1
i=1
I =1
where Ri is the ith fuzzy rule, xj is input variables, Aij is a membership function of fuzzy sets, ai is a constant, n is the number of the fuzzy rules, y∗ is the infered value, i is the premise tness matching of Ri (activation level) and ˆi is the normalized premise
208
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
tness of Ri . If the input variables of the premise and parameters are given in consequence parameter identi cation, the optimal consequence parameters which minimize the assumed performance index can be determined. In what follows, we de ne the performance index as a sum of squared errors. I = (1=m)
m X
2
(y(k) − y (k)) ;
(2)
where y o is the output of the fuzzy model, k denotes the number of the input variables, and “m” stands for the total number of data. Furthermore x1i ; x2j ; : : : ; xki ; yi (i = 1; 2; : : : ; m) are pairs of input– output data set. The consequence parameters ai can be determined by the standard least-squares method. In the fuzzy model of Type 1, the parameters can be estimated by solving the optimization problem.
(5)
where fi is a linear function of the input variables fi (x1 ; : : : ; xk ) = ai0 + ai1 x1 + · · · + aik xk : The numeric output y∗ is determined in the same way as in the previous approach , n ! n X X ∗ i fi (x1 ; : : : ; xk ) i y = i=1
=
n X
i=1
ˆi fi (x1 ; : : : ; xk ):
i=1
Mina V (a; m);
Again Ri is the ith fuzzy rule, xj is an input variable, Aij are membership functions of fuzzy sets, aij are consequence parameters, n is the number of the fuzzy rules, y∗ is the inferred value, i is the truth value of Ri in the premises and ˆi is the normalized truth value of i . The consequence parameters are produced by the standard least-squares method, that is
where m
m
i=1
j=1
1X 2 1X i = (yi − y o ) 2 2 2 i=1 i=1 2 m m X X 1 yi − = aj uji 2
V (a; m) =
m 1X
2
Ri : If x1 is Ai1 ; : : : ; and xk is Aik then y = fi (x1 ; : : : ; xk );
o
i=1
=
3.2.2. Type 2 (First-order linear equation) The consequence is expressed in the form of a linear relationship. The use of the linear (or complex) inference method gives rise to the expression
i=1
1 [yi − xiT a] 2 = kEk 2 : 2
aˆ = (X T X )−1 X T Y; (3)
where now xiT = [1i ; : : : ; ni ; x1i 1i ; : : : ; x1i ni ; : : : ; xki 1i ; : : : ; xki ni ], aˆT = [a10 ; : : : ; an0 ; a11 ; : : : ; an1 ; a1k ; : : : ; ank ];
(Moreover,
Y = [y1 ; : : : ; ym ]T ;
Aji (x1i ) ∗ · · · ∗ Ajk (xki ) ; j=1; m Aji (x1i ) ∗ · · · ∗ Ajk (xki )
uji = P
X = [x1T ; : : : ; xmT ]T :
3.3. The objective function with weighting factor
where j is the rule no., i the data no., m the total no. of data, and k the no. of input fuzzy variables). Due to the form of the performance index in Eq. (3), the minimal value produced by the leastsquares method is determined as follows: aˆ = (X T X )−1 X T Y;
(6)
(4)
where xiT = [1j ; : : : ; ni ]; aT = [a1 ; : : : ; an ]; Y = [y1 ; : : : ; ym ]T ; E = [1 ; : : : ; m ]T ; X = [x1T ; : : : ; xmT ]T .
We elaborate on the performance index. The objective function for the training data and testing data assumes the form f = (PARA1 × PI + PARA2 × E PI)=2 and is utilized as a cost function of the fuzzy model. Where, PARA1 and PARA2 are two weighting factors for PI and E PI, respectively. PI and E PI denote the values of the performance index for the training data
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
and testing data, respectively. For the purpose of minimization of this objective function, all parameters of the premise membership functions such as Gaussianlike and triangular function are modi ed (optimized). Depending upon the values of the weighting factor, let us discuss several speci c cases of the objective function. 1. If PARA1 = 1 and PARA2 = 0 then the model becomes optimized based on the training set. No testing set is taken into consideration. 2. Both PARA1 = 1 and PARA2 = 1 is the case when the training and testing set are taken into account. 3. The case both PARA1 = and PARA2 = 1 − where ∈ [0; 1] embraces both the cases stated above. The choice of establishes a certain tradeo between the approximation and generalization aspects of the fuzzy model. 4. In general, PARA1 and PARA2 can be selected and adjusted independently. The performance index used in the ensuing numerical experiment will be as Euclidean and Hamming distances, that is, PI =
N 1 X (yi − yˆ i ) 2 ; N
(11)
N 1 X |yi − yˆ i |: N
(12)
i=1
PI =
i=1
The variables of a cost function to be optimized come as the parameters of the membership functions, fuzzy rules, and weighting factors of the performance index. Based upon a selection of sound fuzzy reasoning type, speci cation of the membership function type, and weighting factors we can design an optimal fuzzy model. 3.4. Auto-tuning by improved complex method Usually, by combining these optimization tasks we end up with a problem that is highly nonlinear and may not t well to the domain of gradient-based techniques. To alleviate the problem, we propose to use an autotuning algorithm that is an adaptation of the improved complex method. We realize the algorithm by augmenting the simplex concept to the complex method [9] – constrained optimization technique. The owchart of the proposed
209
optimal auto-tuning algorithm is shown in Fig. 3. In fact, the algorithm known as the improved complex method, is the constrained complex method of the form: Minimize f(x) subject to gi (x)60; Xi(l) 6Xi 6Xi(u) ;
j = 1; 2; : : : ; m;
i = 1; 2; : : : ; n;
where the superscripts l and u denote the lower and upper bound of the corresponding variable. Step 1: The parameters to be optimized include the elements of the fuzzy model. They include the slope and center parameter of Gaussian-type membership function, and each parameter (a and b) of the triangular membership function. They are de ned as Xk = (x1k ; x2k ; : : : ; xnk ; k = 1; 2; : : : ; n; n + 1; : : : ; m) and form the points in an “n”-dimensional space. In general, the value of “m” is selected as being equal to 2n (where n is the number of initial vertices). Step 2: The initial values of ; and is speci ed using the re ection, expansion and contraction of the simplex concept as follows: (i) Re ection: Xr = X0 + (X0 − Xh );
(7)
Xe = X0 + (Xr − X0 );
(8)
(ii) Expansion:
(iii) Contraction: Xc = X0 + (Xh − X0 ):
(9)
Step 3: Xh and Xl are the vertices corresponding to the maximum function value f(Xh ) and the minimum function value f(Xl ). X0 is the centroid of all the points Xi except i = h. The re ection point Xr is given by (7), with PnXh = max f(Xi ); (i = 1; : : : ; k); X0 = (1= (m−1))(( i=1 Xi )−Xh ) and = kXr −X0 k=kXh −X0 k. If Xr does not satisfy the constraints, a new point Xr is generated by Xr = (X0 + Xr )=2. This process is repeated until Xr satis es the constraints. A new simplex is started. Step 4: If a re ection process gives a point Xr for which f(Xr )¡f(Xl ), i.e. if the re ection produces a new minimum, we expand Xr to Xe by (8), with
= kXe − X0 k=kXr − X0 k¿1. If Xe does not satisfy the constraints, a new point Xe is generated by Xe = (X0 + Xe )=2. This process is repeated until Xe satis es the constraints. If f(Xe )¡f(Xl ), we replace the point Xh by Xe and restart the process of re ection. On the other hand,
210
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
Fig. 3. Flowchart of auto-tuning algorithm for premise parameters identi cation.
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
if f(Xe )¿f(Xl ), we replace the point Xh by Xr , and start the re ection process again. Step 5: If the re ection process produces a point Xr for which f(Xr )¿f(Xi ), for all i except i = h and f(Xr )¡f(Xh ), then we replace the point Xh by Xr . In this case, we contract the simplex as in (9), with = kXe − X0 k=kXh − X0 k. If f(Xr )¿f(Xh ), we use Xc without changing the previous point Xh . If Xc does not satisfy the constraints, a new point Xc is generated with Xc = (X0 + Xc )=2. This process is conducted repeatedly until Xe satis es the constraints. If the contraction process produces a point Xc for which f(Xc )¡min[ f(Xh ); f(Xr )], we replace the point Xh by Xc and proceed with the re ection again. On the other hand, if f(Xc )¿= min[f(Xh ); f(Xr )], we replace all Xi by (Xi + Xl )=2, and start the re ection process again. Step 6: The method is assumed to have converged whenever the standard deviation of the function at the vertices of the current simplex is smaller than some prescribed small quantity as follows: )1=2 ( n+1 X [ f(Xi ) − f(X0 )] 2 6: (10) Q= n+1 i=1
If Q does not satisfy (10), we go to step 3.
U denotes the ow rate of methane as an input, the output stands for the carbon dioxide density, i.e., the outlet gas. The structure and parameter identi cation of premise are performed using the improved complex method. The improved complex method extracts the optimal fuzzy rules and upgrades the performance by auto-tuning parameters of premise membership function. The re ection, expansion and contraction coecients which are the initial parameters of the improved complex method are set as = 1; = 2 and = 0:5, respectively. The consequence parts of two kinds of types are used. Tables 1 and 2 show the performance index of the optimal rules obtained using the improved complex method for each fuzzy model consisting of the consequence types of simpli ed and linear inference, and the premise types of the Gaussian-type function with xed slope, same slope and di erent slope versions, and triangular-type function (see Figs. 4 – 8). The identi cation error (or performance index) of each model is compared with other fuzzy modeling methods in Table 3. This case summarizes the values of the performance index for the training data consisting of 296 data points. In other words, the weighting factors PARA1 and PARA2 are set to 1 and 0, respectively. 4.2. Sewage treatment process
4. Experimental studies Once the identi cation methodology has been established, one can proceed with intensive experimental studies. In this section, we report on the experiments using some well-known data sets used in fuzzy modeling. These include gas furnace data, sewage treatment process, and trac control data. 4.1. Gas furnace process In this section, the proposed rule-based fuzzy modeling is applied to the time-series data of a gas furnace utilized by Box and Jenkins [2]. We try to model the gas furnace using 296 pairs of input–output data. The
ow rate of methane gas, Um (t) used in the laboratory changes from −2.5 to 2.5, the control U (t) used in real process, ranges from 0.5 to 0.7 following the expression: U (t) = 0:60 − 0:048Um (t);
211
(13)
Sewage treatment generally uses the activated sludge process which consisted of a sand basin, primary sedimentation basin, aeration tank and nal sedimentation basin (see Fig. 9). The suspended solid included in the sewage is sedimented by gravity in sand and primary sedimentation basins. Air is consecutively absorbed in the sewage in the aeration tank for several hours. Microbe lump (that is called oc or activated sludge) springing naturally, mainly removes the organic matter from the aeration tank. Activated sludge biochemically oxygenates, proliferates and resolves the organic matters into hydrogen and carbon dioxide by metabolism. In the nal sedimentation basin, oc is sedimented, recycled and again used to remove the organic matter and then puri ed water is transported to the tertiary sedimentation basin. The activated sludge process is the process that involves an aeration tank and nal sedimentation. We
212
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
213
214
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
Table 2 Performance index in each fuzzy reasoning method by means of the change in the no. of fuzzy variables (a) Simpli ed fuzzy reasoning method with input variables u(t − 3) and y u(t − 3): No. of fuzzy variables 2 3 4
y: No. of fuzzy variables 2
Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular
5
6
PI
E PI
PI
E PI
PI
E PI
PI
E PI
PI
E PI
0.036 0.035 0.038 0.024
0.274 0.275 0.276 0.328
0.040 0.040 0.040 0.022
0.252 0.252 0.252 0.333
0.092 0.092 0.092 0.022
0.215 0.213 0.213 0.332
0.038 0.037 0.037 0.021
0.261 0.261 0.259 0.327
0.034 0.032 0.036 0.021
0.272 0.267 0.262 0.330
(b) Simpli ed fuzzy reasoning method with input variables u(t − 4) and y u(t − 4): No. of fuzzy variables 2 3
y: No. of fuzzy variables
2
Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope)
4
PI
E PI
PI
E PI
PI
E PI
0.090 0.092 0.101
0.265 0.266 0.259
0.093 0.093 0.093
0.208 0.208 0.209
0.039 0.037 0.037
0.257 0.260 0.260
(c) Linear fuzzy reasoning method with input variables u(t − 3) and y
y: No. of fuzzy variables
2
Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular
u(t − 3): No. of fuzzy variables 2 3
4
PI
E PI
PI
E PI
PI
E PI
PI
E PI
0.020 0.020 0.020 0.022
0.278 0.272 0.271 0.326
0.019 0.023 0.018 0.021
0.270 0.279 0.285 0.364
0.035 0.183 0.021 0.020
0.269 0.331 0.282 0.333
0.029 0.020 0.020 0.019
0.279 0.293 0.294 0.340
5
(d) Linear fuzzy reasoning method with input variables u(t − 4) and y u(t − 4): No. of fuzzy variables 2 3
y: No. of fuzzy variables
2
Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope)
measure the biological oxygen demand (bod) and the concentration of suspended solid (ss) in in uent sewage at the primary sedimentation basin, and euent bod (ebod) and ss (ess) in the euent sewage at the nal sedimentation basin. Because ebod and ess are changed, depending on the bod and ss, dissolved
4
PI
E PI
PI
E PI
PI
E PI
0.072 0.072 0.071
0.206 0.209 0.212
0.071 0.090 0.072
0.334 0.256 0.212
0.069 0.091 0.079
0.314 0.233 0.219
oxygen set-point (dosp) and recycle sludge ratio setpoint (rrsp) are set so that ess and ebod should be kept up less than the prescribed small quantity. Ebod and ess depend on mixed liquid suspended solid (mlss), waste sluge ratio (wsr), rrsp and dosp. Bod has a correlation with ss.
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
215
Fig. 4. Data points induced by I=O data set (u(t−3); y(t−1); y(t)).
Fig. 5. Data points induced by I=O data set (u(t−4); y(t−1); y(t)).
In this paper, a sewage treatment system plant in Seoul, Korea, is chosen as a model. The rule-based fuzzy modeling by two kinds of fuzzy inference is carried out using 52 pairs of input–output data obtained from the activated sludge process. From four input variables, we choose two input variables that minimize the evaluation criterion and fuzzy rule number, and extract more than two fuzzy partitions (Big and Small) from each input–output pair of data. The identi ed parameters of premise and consequence of the optimal fuzzy rules are obtained using the improved complex method. From the two-dimensional plot of the data set shown in Figs. 10 –15, in the case of the training data, the data sets (mlss, dosp, ess) and (rrsp, dosp, ess) exhibit less uniform and more sparse distribution than any other data set. Therefore, we can anticipate that the fuzzy model structure for the fuzzy partition of data set (mlss, dosp, ess) and (rrsp, dosp, ess) could perform
Fig. 6. The comparison of original data and output data for fuzzy model No. 2 (Table 1(a)). (a) In the case of training data. (b) In the case of testing data.
Fig. 7. Convergence procedure to optimal value of PI & E PI for fuzzy model No. 2 (Table 1(a)).
216
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
Fig. 8. Convergence procedure to optimal value of maximal and minimal point of input fuzzy membership functions, (umin (t − 3); umax (t − 3)) and (ymin (t − 1); ymax (t − 1)) for fuzzy model No. 2 (Table 1(a)). (a) In the case of (umin (t − 3); umax (t − 3)). (b) In the case of (ymin (t − 1); ymax (t − 1)).
a little worse than in the remaining scenarios (Fig. 16; Tables 4 and 5). 4.3. Trac route choice process Lately, researchers and ocials who work in the eld of trac planning and engineering have expressed a lot of interest about the assignment of trac load in road network. From 1950s, many highways have been planned and constructed for transportation
of men and materials due to the economic development of the advanced industrial nations such as United States, Europe and Japan. It has triggered an important issue on how to assign trac load and setup e ectively the harmonious transportation system from two roads or a few roads all being in a competitive relationship. To solve such problems, lots of research is being carried out to get the optimal model of trac transition and trac route related to the trac planning.
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
217
Table 3 Comparison of identi cation errors with previous fuzzy models Model name
PI
Total no. of rules
Tong’s model [13] Pedrycz’s model [9] Xu’s model [15] Sugeno’s model [10] Our model Simpli ed
0.469 0.776 0.328 0.355 0.137 0.155 0.180 0.149 0.127 0.123 0.124 0.134
19 20 25 6 4 4 4 6 4 4 4 4
Linear
Gaussian (no-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular
Fig. 9. Con guration of the sewage treatment system.
The selection of a certain trac route is an example of a complex driver’s decision making. Ambiguity (or fuzziness) inherently associated with trac route problem exhibits the implicit meaning such as: (1) ambiguity of model’s input data, (2) fuzziness of human cognition, (3) ambiguity of human decision making. The usage of the knowledge data based on information mentioned above, a method was proposed to make a trac route choice model [1]. Generally, most of these methods perform modeling from valid infor-
mation which are obtained from human knowledge. This method is di erent from some other modeling methods which are primarily based upon statistical approaches. We can think of many cases of route choice model. But, we con ne our interest to the assignment problem between road network and rail system or route choice problem between road and highway. As shown in Fig. 17, Fig. 1 illustrates a simple example of a route structure from O to D. Even though we can
218
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
Fig. 10. Data points induced by I=O data set (mlss, wsr, ess).
Fig. 11. Data points induced by I=O data set (mlss, rrsp, ess).
Fig. 12. Data points induced by I=O data set (mlss, dosp, ess).
consider various types of cost related to many decision criterion for evaluation about each route for generalization, in this paper, we consider only time costs and trac costs as the essential components of the objective function.
Fig. 13. Data points induced by I=O data set (wsr, rrsp, ess).
Fig. 14. Data points induced by I=O data set (wsr, dosp, ess).
Fig. 15. Data points induced by I=O data set (rrsp, dosp, ess).
We consider a simple problem related to trac route choice which is based on two routes mentioned above. The model was considered as a conventional example for the trac route choice model [5]. Then, we assume that the object for trac routes is a driver and our interest is focused on how to express trac behavior at
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
219
R 2 : If u(t − 3) is Small1 & y(t − 1) is Big2 then y(t) = a2 . R 3 : If u(t − 3) is Big1 & y(t − 1) is Small2 then y(t) = a3 . R4 : If u(t − 3) is Big1 & y(t − 1) is Big2 then y(t) = a4 . The initial and nal tuned values of each parameter associated with the fuzzy rules mentioned above are like in Table 10. Those values of Gaussian membership functions of fuzzy input variables are shown in Fig. 21. The dotted line in Fig. 21 represents the nal tuned values.
Fig. 16. Convergence procedure to optimal value of PI & E PI for fuzzy model No. 5 (Table 4(a)).
the individual level. The performance index is de ned as shown in Eq. (12). The inputs T 1 and T 2 denote a trac and time cost associated with routes 1 and 2, respectively. The output P1n is the choice probability ratio of route 1 (Tables 6 and 7). The performance index of each model is compared with other fuzzy modeling methods and the results are summarized in Tables 8 and 9. This comparison reveals that performance index for the training data only; the testing data are not studied. In other words, the weighting factors assume the values PARA1 = 1 and PARA2 = 0, respectively. The analysis of the experimental data allows us to draw some general conclusions: 4.3.1. Gas furnace process The most suitable design option is to choose the simpli ed fuzzy reasoning method, optimize the second weighting parameter (PARA2) of the objective function and use the Gaussian membership function. The best results for the training set can be derived for the fuzzy model with the linear reasoning method. As Table 2 reveals, the increase in the number of the linguistic terms for the input space improves the performance of the model. The optimal fuzzy rules obtained from model 6 with di erent slope in Table 1 are as follows: R1 : If u(t − 3) is Small1 & y(t − 1) is Small2 then y(t) = a1 .
4.3.2. Sewage treatment process As this data set is far smaller than the previous one, the behavior of the model could be very di erent depending on whether we are concerned with the testing or the training set. In this case the PI and E PI make a huge di erence. We may anticipate that equipping the model with modi able membership functions could be bene cial. In fact, this is supported by the completed experiments. For the training and testing data sets, we observe a better performance for the linear and simpli ed reasoning method, respectively. As shown in Table 5 in the case of linear fuzzy reasoning method, the increase in the number of membership function of input fuzzy variables make PI much better, but E PI becomes much worse due to the excessive over tting of the consequence linear equations. The optimal fuzzy rules obtained from model 7 with same slope in Table 5 are as follows: R1 : If wsr is Small1 & dosp is Small2 then ess = a1 . R 2 : If wsr is Small1 & dosp is Big2 then ess = a2 . R 3 : If wsr is Big1 & dosp is Small2 then ess = a3 . R4 : If wsr is Big1 & dosp is Big2 then ess = a4 . The initial and nal tuned values of each parameter associated with the fuzzy rules mentioned above are visualized in Table 11. These values of Gaussian membership functions of fuzzy input variables are shown in Fig. 22. 4.3.3. Trac route choice process As mentioned in the gas furnace and sewage treatment process, the quantity and sparse distribution of data set a ect the structure of the optimal fuzzy model such as the reasoning method, number of membership
220
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
221
222
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
223
Fig. 17. Simple example of route choice.
Table 6 Example data for the logit model No
Route
T1
T2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
2 2 1 2 2 1 1 2 2 2 2 1 1 2 1 1 2 1 1 2 1
52.9 4.1 4.1 56.2 51.8 0.2 27.6 89.9 41.5 95.0 99.1 18.5 82.0 8.6 22.5 51.4 81.0 51.0 62.2 95.1 41.6
4.4 28.5 86.9 31.6 20.2 91.2 79.7 2.2 24.5 43.5 8.4 84.0 38.0 1.6 74.1 83.8 19.2 85.0 90.1 22.2 91.5
functions of input fuzzy variables and membership function type. The optimal fuzzy rules obtained from model 4 with di erent slopes in Table 7a are as follows: R1 : If T1 is Small1 & T2 is Small2 then route = a1 . R 2 : If T1 is Small1 & T2 is Big2 then route = a2 . R 3 : If T1 is Big1 & T2 is Small2 then route = a3 . R4 : If T1 is Big1 & T2 is Big2 then route = a4 . The initial and nal tuned values of each parameter associated with the fuzzy rules mentioned above are like in Table 12. Those values of Gaussian membership functions of fuzzy input variables are shown in Fig. 23. Again, as in the previous experiments we worked with various optimization scenarios utilizing all parameters.
Fig. 18. The comparison of original data and output data for fuzzy model No. 1 (Table 7(a)). (a) In the case of the training data. (b) In the case of the testing data.
When we apply three types of Gaussian functions and triangular function to a premise structure as the membership function of fuzzy input variables, we can compare and analyze the characteristics and the output
224
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
225
226
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
Table 8 Performance index in each fuzzy reasoning method by means of the change in the number of fuzzy variables T 1: No. of fuzzy variables 2 3
(a) Simpli ed fuzzy reasoning method T 2: No. of fuzzy variables 2 Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular 3 Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular 4 Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular (b) Linear fuzzy reasoning method T 2: No. of Fuzzy variables 2
3
4
Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular
4
PI
E PI
PI
E PI
PI
E PI
1.696 1.710 1.690 1.768 1.913 0.00009 0.028 1.466 1.626 0.000004 0.00002 1.603
8.663 8.710 8.655 8.709 8.682 7.000 7.207 9.587 8.388 7.993 7.998 8.792
1.585 1.361 1.488 1.631 0.751 0.002 0.009 1.690 0.098 0.002 0.046 1.179
8.540 7.030 7.254 8.551 7.977 7.003 7.028 8.533 8.629 7.039 7.261 8.797
1.383 1.357 1.422 1.605 0.114 0.107 0.164
7.129 7.020 7.107 7.855 7.314 7.305 7.245
0.0011 0.0002 0.0003
7.333 7.351 7.570
1.477 0.52 1.326 2.016 0.056 0.067 0.052 0.480 0.002 0.0003 0.002 0.001
8.720 7.57 8.469 8.209 8.625 8.715 8.720 8.871 7.977 8.338 8.183 10.270
0.007 0.059 0.003 0.787 0.0001 0.005 0.001 0.00001 0.00004 0.0001 0.001 0.003
8.943 7.404 9.394 7.604 7.971 7.415 7.677 15.878 8.072 8.141 7.938 8.043
0.050 0.244 0.376 0.006 0.010 0.0002 0.00006
8.844 7.741 8.245 8.787 16.958 7.714 8.419
0.0014 0.014 0.00001
8.663 7.604 9.281
Table 9 Comparison of identi cation errors with previous models Model type
PI
Shooting ratio (%)
1) 2) 3) 4)
5.452 2.0 0.497 1.178 0.000002 0.000001 0.000001 0.01 0.006 0.0001 0.013 0.009
90.4 85.7 95.2 90.4 99.9 99.9 99.9 99.7 99.9 99.9 99.7 99.9
BL (Binary Logit) model [5] PS (Production System) model [1] Neural networks model [1] Fuzzy neural networks model [1] Our model Simpli ed
Linear
Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
227
Fig. 19. Convergence procedure to optimal value of PI & E PI for fuzzy model No. 1 (Table 7(a)).
Fig. 21. The initial and nal tuned values of Gaussian-like membership functions for model 6 with di erent slope in Table 1(a)). (a) In the case of membership function of fuzzy variable u(t − 3). (b) In the case of membership function of fuzzy variable y(t − 1).
Fig. 20. Convergence procedure to optimal value of maximal and minimal point of input fuzzy membership functions (T 1min ; T 1max ) and (T 2min ; T 2max ) for fuzzy model No. 1 (Table 7(a)). (a) In the case of T 1min . (b) In the case of T 1max . (c) In the case of (T 2min ; T 2max ).
228
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
Fig. 22. The initial and nal tuned values of Gaussian-like membership functions for model 7 with same slopes in Table 5(a). (a) In the case of membership functions of fuzzy variable wsr. (b) In the case of membership functions of fuzzy variable dosp.
performance of fuzzy model according to each process. As shown in the Tables 1, 2, 4, 5, 7 and 8, the output performance index for testing data in the case of the Gaussian membership function is better than that produced in the case of the triangular membership function. Considering the testing data we obtain better performance for the linear inference method; the improvement occurs on changing the values of the weighting factors in the performance index. By increasing the number of the membership functions in each of input fuzzy variables, in the simpli ed inference method we can nd the optimal fuzzy model with better output performance for both training data and testing data, but in the linear inference method, it is dicult to obtain better output performance from the testing data because of the excessive over tting of consequence linear equations. From the observation and characteristic analysis of two fuzzy in-
Fig. 23. The initial and nal tuned values of Gaussian-like membership functions for model 4 with di erent slopes in Table 7(a). (a) In the case of membership functions of fuzzy variable T 1. (b) In the case of membership functions of fuzzy variable T 2.
ference methods, type of membership functions, no. of membership function of fuzzy input variables, and the weighting factors of the objective function shown in this paper, it is available and feasible to design the optimal fuzzy model structure with a better performance output. 5. Conclusions In this paper, the ecient identi cation technique is presented which automatically extracts the optimal fuzzy rules, using an auto-tuning algorithm and the weighting factors of an object function. The improved complex method, which is a powerful autotuning, is used for auto-tuning of parameters of the premise membership functions in consideration with the overall structure of fuzzy rules. Two types of fuzzy reasoning methods (simpli ed (type 1), linear (type 2)) are used and we consider three types of Gaussian
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
229
230
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
membership functions such as xed, same slope and di erent slope, and the triangular membership function. By means of the adjustment of weighting factors of the objective function for both training and testing data, and the auto-tuning of the parameters of each membership function, we can get a better performance of the fuzzy model. According to the increase in the number of membership functions of each input variable of the process system, generally the PI (performance index) for the fuzzy model using training data is improved, but the PI for the fuzzy model using testing data gets worse. The PI for the fuzzy model by means of the linear inference method using the testing data gets much worse than that of the simpli ed inference method. Comparing the performances of the Gaussian-type membership function with xed slope, same slope, and di erent slope, we nd it depends on the amount of data, a certain degree of nonlinearity and a weighting factor as shown in the previous section. Generally, in the case of same-slope, we can get better performance. The PI for testing data in the case of the simpli ed method is better than that produced in the case of the linear method. Furthermore, in this case of the simpli ed method, the di erence between PI (performance index for training data) and E PI (performance index for testing data) is much smaller. Moreover, the performance of the fuzzy model with the simpli ed inference method using testing data is better than that exploiting the linear type of the inference method. Acknowledgement This paper was supported in part by fund for Postdoctoral program (1996) of Korea Science and Engineering Foundation, the Republic of South Korea. References [1] T. Akiyama, Modeling of route choice behaviour using knowledge based techniques, JSCE 11 (1993) 65–72 (in Japanese).
[2] G.E.P. Box et al., Time Series Analysis, Forecasting and Control, Holden Day, San Francisco, CA. [3] E. Czogola, W. Pedrycz, On identi cation in fuzzy systems and its applications in control problems, Fuzzy Sets and Systems 6 (1981) 73 – 83. [4] M.A. Ismail, Soft clustering algorithm and validity of solutions, in: M.M. Gupta (Ed.), Fuzzy Computing Theory, Hardware and Applications, North-Holland, Amsterdam, 1988, pp. 445– 471. [5] S. Morihayashi, The Forecasting and Test of Disaggregate Model, Text of Infrastructure of Civil Engineering of Japan, vol. 15, JSCE, 1984 (in Japanese). [6] S.K. Oh, H.K. Kim, K.B. Woo, Fuzzy modeling for activated sludge process and control systems, Korean Institute Electrical Engr. 40 (10) (1990) 1072–1083. [7] S.K. Oh, K.B. Woo, Fuzzy identi cation by means of fuzzy inference method and its application to waste water treatment system, J. Korean Institute Telemat. Electronics (KITE) (1994) 43–52. [8] W. Pedrycz, Numerical and application aspects of fuzzy relational equations, Fuzzy Sets and Systems 11 (1983) 1–18. [9] W. Pedrycz, An identi cation algorithm in fuzzy relational system, Fuzzy Sets and Systems 13 (1984) 153–167. [10] M. Sugeno, T. Yasukawa, Linguistic modeling based on numerical data, IFSA’91 Brussels, Computer, Management & System Science, 1991, pp. 264 –267. [11] T. Takagi, M. Sugeno, Fuzzy identi cation of systems and its applications to modeling and control, IEEE Trans. Systems Cybernet. SMC-15 (1) (1985) 116 –132. [12] R.M. Tong, Synthesis of fuzzy models for industrial processes, Int. J. Gen. Systems 4 (1978) 143–162. [13] R.M. Tong, The evaluation of fuzzy models derived from experimental data, Fuzzy Sets and Systems 13 (1980) 1–12. [14] C.W. Xu, Fuzzy system identi cation, IEE Proc. 126 (4) (1989) 146 –150. [15] C.W. Xu, Y. Zailu, Fuzzy model identi cation self-learning for dynamic system, IEEE Trans. Systems Man Cybernet. SMC-17 (4) (1987) 683– 689. [16] L.X. Xuanli, B. Gerado, Validity measure for fuzzy cluster, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-13 (8) (1991) 841– 847.
www.elsevier.com/locate/fss
Identi cation of fuzzy systems by means of an auto-tuning algorithm and its application to nonlinear systems Sungkwun Oha , Witold Pedryczb;∗ a Department
of Control and Instrumentation Engineering, Wonkwang University, 344-2, Shinyong-Dong, Iksan, Chon-Buk, 570-749, South Korea b Department of Electrical and Computer Engineering, University of Manitoba, Winnipeg, Manitoba, Canada R3T 2N2 Received October 1997; received in revised form May 1998
Abstract The study concerns a design procedure of rule-based systems. The proposed rule-based fuzzy modeling implements system structure and parameter identi cation in the ecient form of “IF: : : ; THEN: : :” statements, and exploits the theory of system optimization and fuzzy implication rules. Two types of methods for rule-based fuzzy modeling are studied. This classi cation concerns the form of the conclusion part of the rules that can be either constant or formed by some linear functions. Both triangular and Gaussian-like membership function are studied. The optimization hinges on an auto-tuning algorithm that covers a modi ed constrained optimization method known as a complex method. The study introduces a weighted performance index (objective function) that helps achieve a sound balance between the quality of results produced for the training and testing set. This methodology sheds light on the role and impact of di erent parameters of the model on its performance (in-particular, the mapping and predicting capabilities of the rule-based computing). The study is illustrated with the aid of c 2000 Elsevier Science B.V. All rights reserved. several representative numerical examples. Keywords: Identi cation of fuzzy model; Auto-tuning; Weighting factor; Gaussian and triangular membership functions; Optimal fuzzy model
1. Introduction In the early 1980s, the linguistic approach [12,9] and fuzzy relationship equation-based approach [8,3] were proposed as identi cation methods of fuzzy models. In the linguistic approach, Tong identi ed the gas furnace process by means of a logical examination of data [13]. Li et al. obtained good results through ∗ Corresponding author. Tel.: +1 204 474 8380; fax: +1 204 261 4639. E-mail address: [email protected] (W. Pedrycz).
the modi cation of Tong’s method [14] and also proposed the modi ed algorithm of the adaptive model based on a decision table. But the algorithm has some problems due to the computer capacity and computation time which are important, when it is applied to the high-order multivariable systems [15]. Pedrycz analyzed the identi cation of fuzzy systems from the viewpoint of linguistic implication rule modeling, using the referential-fuzzy-set concept [9]. Li et al. presented a self-learning algorithm for the simple SISO fuzzy model [15]. In the fuzzy relationship equation-based approach, Pedrycz identi ed fuzzy
c 2000 Elsevier Science B.V. All rights reserved. 0165-0114/00/$ - see front matter PII: S 0 1 6 5 - 0 1 1 4 ( 9 8 ) 0 0 1 7 4 - 2
206
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
systems, using the referential fuzzy set and Zadeh’s conditional possibility distribution, that is, the new composition rules which were made by the fuzzy relationship equations [8]. Xu constructed and identi ed the fuzzy relationship model using the referential fuzzy set theory and the self-learning algorithm [14,15]. The direct inference utilized by two methods did not perform better than the linear inference. Sugeno identi ed the structure of systems through the standard least-squares methods [11], but the structure of premises of the rules was determined more heuristically through the experience and iterative fuzzy partitioning of the input space. Sugeno also applied his method to the fuzzy identi cation of a gas furnace process, using fuzzy c-means clustering [4,16], but the method did not produce the identi cation of good performance; this could be alleviated to the use of direct linear inference [10]. In this paper, two types of fuzzy inferences are considered, that is, simpli ed (Type 1) and linear (Type 2) reasoning models. In Type 1 we are concerned with consequents regarded as single numerical quantities. Type 2 deals with linear functions viewed as the consequents of the rules. In each fuzzy inference, we consider three types of membership functions, namely Gaussian membership functions with modi able or xed slopes, and triangular membership functions. According to the proposed auto-tuning algorithm – the improved complex method, the parameters of such membership function can be easily adjusted. Furthermore, we introduce an aggregate objective function that deals with training data and testing data, and elaborate on its optimization to produce a meaningful balance between approximation and generalization abilities of the model. The proposed rule-based fuzzy modeling is carried out for time-series data for gas furnace process [2], activated sludge process in sewage treatment system [6] and trac route choices. The performance of the proposed rule-based fuzzy modeling is compared from the viewpoint of the identi cation errors with conventional fuzzy modeling [9,15,13,10,6]. 2. System modeling by means of fuzzy inference The identi cation algorithm of fuzzy model is divided into the identi cation activities of premise and
consequence parts of the rules. The identi cation at the premise level (1) selects the input variables x1 ; x2 ; : : : ; xn of the rules, and (2) determines the fuzzy partitions (Small, Large, etc.) of fuzzy spaces. This means the determination of the number of the optimal fuzzy space partitions, that is, fuzzy subspaces that determinate the number of fuzzy implication rules. The premise identi cation has to determine the membership values of fuzzy variables. The consequence identi cation embraces the following phases: (1) selection of the consequence variables of the fuzzy implication rules, (2) determination of the consequence parameters. In this paper, in order to identify the premise structure and parameters of fuzzy linguistic rules, two essential input variables of process in uenced are considered and the improved complex method which is a powerful auto-tuning algorithm is used. Furthermore, we restrict ourselves to some types of membership function such as Gaussian-like and triangular ones. The parameters of the membership functions are tuned with the help of the auto-tuning method. The parameters of the consequence part of the rules are determined using the standard least-squares method (Gaussian elimination with maximal pivoting algorithm). We also discuss a modi ed performance index (objective function) that aims at achieving a balance between approximation and prediction capabilities of the fuzzy model. 3. An algorithm of fuzzy identiÿcation In this section we elaborate on algorithmic details of the identi cation method discussing the optimization problem to the antecedent (condition part) of the rules as well as enhancements of their conclusions. 3.1. Premise identiÿcation In the premise part of the rules we con ne ourselves to Gaussian-like and triangular-type functions. This selection, even though looks somewhat limited, includes a broad range of cases that could be covered by modifying the parameters of these functions. For the triangular membership functions we have either 2 or 3 parameters, see Fig. 1, whose points can be auto-tuned (adjusted). The Gaussian type of the
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
207
Fig. 1. Triangular membership function. (a) Two fuzzy variables with two modi able parameters. (b) Three fuzzy variables with three modi able parameters.
membership function assumes the form f(x) = e−s(x−a)
2
=b 2
:
Furthermore, we consider Gaussian membership functions involving a xed slope, and assume several levels of their parametric exibility such as – xed slope of the functions, – the slopes are modi ed but kept the same for all the functions, – the slopes of the membership function can vary and will be adjusted (optimized), refer to Fig. 2. In the case of the same slope and di erent slope, these mean the same and di erent slope in each input variable, and the slope parameter of each case is autotuned according to the proposed optimization. 3.2. Consequence identiÿcation The identi cation of the conclusion parts of the rules deals with a selection of their structure (Types 1 and 2) and a determination of the respective parameters of the functions therein.
Fig. 2. Gaussian membership function. (a) Parameters of Gaussian type membership function. (b) Three fuzzy variables with three modi able parameters (s, a, and b).
3.2.1. Type 1 (Constant: consequence part) The consequence part of the simpli ed inference mechanism where the rules have constant conclusion part is given as follows. Ri : If x1 is Ai1 ; : : : ; and xk is Aik ; then y = ai :
(1)
The calculations of the numeric output of the model are carried out in the well-known form, , n ! n n X X X ∗ i ai i = ˆi ai ; y = i=1
i=1
I =1
where Ri is the ith fuzzy rule, xj is input variables, Aij is a membership function of fuzzy sets, ai is a constant, n is the number of the fuzzy rules, y∗ is the infered value, i is the premise tness matching of Ri (activation level) and ˆi is the normalized premise
208
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
tness of Ri . If the input variables of the premise and parameters are given in consequence parameter identi cation, the optimal consequence parameters which minimize the assumed performance index can be determined. In what follows, we de ne the performance index as a sum of squared errors. I = (1=m)
m X
2
(y(k) − y (k)) ;
(2)
where y o is the output of the fuzzy model, k denotes the number of the input variables, and “m” stands for the total number of data. Furthermore x1i ; x2j ; : : : ; xki ; yi (i = 1; 2; : : : ; m) are pairs of input– output data set. The consequence parameters ai can be determined by the standard least-squares method. In the fuzzy model of Type 1, the parameters can be estimated by solving the optimization problem.
(5)
where fi is a linear function of the input variables fi (x1 ; : : : ; xk ) = ai0 + ai1 x1 + · · · + aik xk : The numeric output y∗ is determined in the same way as in the previous approach , n ! n X X ∗ i fi (x1 ; : : : ; xk ) i y = i=1
=
n X
i=1
ˆi fi (x1 ; : : : ; xk ):
i=1
Mina V (a; m);
Again Ri is the ith fuzzy rule, xj is an input variable, Aij are membership functions of fuzzy sets, aij are consequence parameters, n is the number of the fuzzy rules, y∗ is the inferred value, i is the truth value of Ri in the premises and ˆi is the normalized truth value of i . The consequence parameters are produced by the standard least-squares method, that is
where m
m
i=1
j=1
1X 2 1X i = (yi − y o ) 2 2 2 i=1 i=1 2 m m X X 1 yi − = aj uji 2
V (a; m) =
m 1X
2
Ri : If x1 is Ai1 ; : : : ; and xk is Aik then y = fi (x1 ; : : : ; xk );
o
i=1
=
3.2.2. Type 2 (First-order linear equation) The consequence is expressed in the form of a linear relationship. The use of the linear (or complex) inference method gives rise to the expression
i=1
1 [yi − xiT a] 2 = kEk 2 : 2
aˆ = (X T X )−1 X T Y; (3)
where now xiT = [1i ; : : : ; ni ; x1i 1i ; : : : ; x1i ni ; : : : ; xki 1i ; : : : ; xki ni ], aˆT = [a10 ; : : : ; an0 ; a11 ; : : : ; an1 ; a1k ; : : : ; ank ];
(Moreover,
Y = [y1 ; : : : ; ym ]T ;
Aji (x1i ) ∗ · · · ∗ Ajk (xki ) ; j=1; m Aji (x1i ) ∗ · · · ∗ Ajk (xki )
uji = P
X = [x1T ; : : : ; xmT ]T :
3.3. The objective function with weighting factor
where j is the rule no., i the data no., m the total no. of data, and k the no. of input fuzzy variables). Due to the form of the performance index in Eq. (3), the minimal value produced by the leastsquares method is determined as follows: aˆ = (X T X )−1 X T Y;
(6)
(4)
where xiT = [1j ; : : : ; ni ]; aT = [a1 ; : : : ; an ]; Y = [y1 ; : : : ; ym ]T ; E = [1 ; : : : ; m ]T ; X = [x1T ; : : : ; xmT ]T .
We elaborate on the performance index. The objective function for the training data and testing data assumes the form f = (PARA1 × PI + PARA2 × E PI)=2 and is utilized as a cost function of the fuzzy model. Where, PARA1 and PARA2 are two weighting factors for PI and E PI, respectively. PI and E PI denote the values of the performance index for the training data
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
and testing data, respectively. For the purpose of minimization of this objective function, all parameters of the premise membership functions such as Gaussianlike and triangular function are modi ed (optimized). Depending upon the values of the weighting factor, let us discuss several speci c cases of the objective function. 1. If PARA1 = 1 and PARA2 = 0 then the model becomes optimized based on the training set. No testing set is taken into consideration. 2. Both PARA1 = 1 and PARA2 = 1 is the case when the training and testing set are taken into account. 3. The case both PARA1 = and PARA2 = 1 − where ∈ [0; 1] embraces both the cases stated above. The choice of establishes a certain tradeo between the approximation and generalization aspects of the fuzzy model. 4. In general, PARA1 and PARA2 can be selected and adjusted independently. The performance index used in the ensuing numerical experiment will be as Euclidean and Hamming distances, that is, PI =
N 1 X (yi − yˆ i ) 2 ; N
(11)
N 1 X |yi − yˆ i |: N
(12)
i=1
PI =
i=1
The variables of a cost function to be optimized come as the parameters of the membership functions, fuzzy rules, and weighting factors of the performance index. Based upon a selection of sound fuzzy reasoning type, speci cation of the membership function type, and weighting factors we can design an optimal fuzzy model. 3.4. Auto-tuning by improved complex method Usually, by combining these optimization tasks we end up with a problem that is highly nonlinear and may not t well to the domain of gradient-based techniques. To alleviate the problem, we propose to use an autotuning algorithm that is an adaptation of the improved complex method. We realize the algorithm by augmenting the simplex concept to the complex method [9] – constrained optimization technique. The owchart of the proposed
209
optimal auto-tuning algorithm is shown in Fig. 3. In fact, the algorithm known as the improved complex method, is the constrained complex method of the form: Minimize f(x) subject to gi (x)60; Xi(l) 6Xi 6Xi(u) ;
j = 1; 2; : : : ; m;
i = 1; 2; : : : ; n;
where the superscripts l and u denote the lower and upper bound of the corresponding variable. Step 1: The parameters to be optimized include the elements of the fuzzy model. They include the slope and center parameter of Gaussian-type membership function, and each parameter (a and b) of the triangular membership function. They are de ned as Xk = (x1k ; x2k ; : : : ; xnk ; k = 1; 2; : : : ; n; n + 1; : : : ; m) and form the points in an “n”-dimensional space. In general, the value of “m” is selected as being equal to 2n (where n is the number of initial vertices). Step 2: The initial values of ; and is speci ed using the re ection, expansion and contraction of the simplex concept as follows: (i) Re ection: Xr = X0 + (X0 − Xh );
(7)
Xe = X0 + (Xr − X0 );
(8)
(ii) Expansion:
(iii) Contraction: Xc = X0 + (Xh − X0 ):
(9)
Step 3: Xh and Xl are the vertices corresponding to the maximum function value f(Xh ) and the minimum function value f(Xl ). X0 is the centroid of all the points Xi except i = h. The re ection point Xr is given by (7), with PnXh = max f(Xi ); (i = 1; : : : ; k); X0 = (1= (m−1))(( i=1 Xi )−Xh ) and = kXr −X0 k=kXh −X0 k. If Xr does not satisfy the constraints, a new point Xr is generated by Xr = (X0 + Xr )=2. This process is repeated until Xr satis es the constraints. A new simplex is started. Step 4: If a re ection process gives a point Xr for which f(Xr )¡f(Xl ), i.e. if the re ection produces a new minimum, we expand Xr to Xe by (8), with
= kXe − X0 k=kXr − X0 k¿1. If Xe does not satisfy the constraints, a new point Xe is generated by Xe = (X0 + Xe )=2. This process is repeated until Xe satis es the constraints. If f(Xe )¡f(Xl ), we replace the point Xh by Xe and restart the process of re ection. On the other hand,
210
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
Fig. 3. Flowchart of auto-tuning algorithm for premise parameters identi cation.
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
if f(Xe )¿f(Xl ), we replace the point Xh by Xr , and start the re ection process again. Step 5: If the re ection process produces a point Xr for which f(Xr )¿f(Xi ), for all i except i = h and f(Xr )¡f(Xh ), then we replace the point Xh by Xr . In this case, we contract the simplex as in (9), with = kXe − X0 k=kXh − X0 k. If f(Xr )¿f(Xh ), we use Xc without changing the previous point Xh . If Xc does not satisfy the constraints, a new point Xc is generated with Xc = (X0 + Xc )=2. This process is conducted repeatedly until Xe satis es the constraints. If the contraction process produces a point Xc for which f(Xc )¡min[ f(Xh ); f(Xr )], we replace the point Xh by Xc and proceed with the re ection again. On the other hand, if f(Xc )¿= min[f(Xh ); f(Xr )], we replace all Xi by (Xi + Xl )=2, and start the re ection process again. Step 6: The method is assumed to have converged whenever the standard deviation of the function at the vertices of the current simplex is smaller than some prescribed small quantity as follows: )1=2 ( n+1 X [ f(Xi ) − f(X0 )] 2 6: (10) Q= n+1 i=1
If Q does not satisfy (10), we go to step 3.
U denotes the ow rate of methane as an input, the output stands for the carbon dioxide density, i.e., the outlet gas. The structure and parameter identi cation of premise are performed using the improved complex method. The improved complex method extracts the optimal fuzzy rules and upgrades the performance by auto-tuning parameters of premise membership function. The re ection, expansion and contraction coecients which are the initial parameters of the improved complex method are set as = 1; = 2 and = 0:5, respectively. The consequence parts of two kinds of types are used. Tables 1 and 2 show the performance index of the optimal rules obtained using the improved complex method for each fuzzy model consisting of the consequence types of simpli ed and linear inference, and the premise types of the Gaussian-type function with xed slope, same slope and di erent slope versions, and triangular-type function (see Figs. 4 – 8). The identi cation error (or performance index) of each model is compared with other fuzzy modeling methods in Table 3. This case summarizes the values of the performance index for the training data consisting of 296 data points. In other words, the weighting factors PARA1 and PARA2 are set to 1 and 0, respectively. 4.2. Sewage treatment process
4. Experimental studies Once the identi cation methodology has been established, one can proceed with intensive experimental studies. In this section, we report on the experiments using some well-known data sets used in fuzzy modeling. These include gas furnace data, sewage treatment process, and trac control data. 4.1. Gas furnace process In this section, the proposed rule-based fuzzy modeling is applied to the time-series data of a gas furnace utilized by Box and Jenkins [2]. We try to model the gas furnace using 296 pairs of input–output data. The
ow rate of methane gas, Um (t) used in the laboratory changes from −2.5 to 2.5, the control U (t) used in real process, ranges from 0.5 to 0.7 following the expression: U (t) = 0:60 − 0:048Um (t);
211
(13)
Sewage treatment generally uses the activated sludge process which consisted of a sand basin, primary sedimentation basin, aeration tank and nal sedimentation basin (see Fig. 9). The suspended solid included in the sewage is sedimented by gravity in sand and primary sedimentation basins. Air is consecutively absorbed in the sewage in the aeration tank for several hours. Microbe lump (that is called oc or activated sludge) springing naturally, mainly removes the organic matter from the aeration tank. Activated sludge biochemically oxygenates, proliferates and resolves the organic matters into hydrogen and carbon dioxide by metabolism. In the nal sedimentation basin, oc is sedimented, recycled and again used to remove the organic matter and then puri ed water is transported to the tertiary sedimentation basin. The activated sludge process is the process that involves an aeration tank and nal sedimentation. We
212
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
213
214
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
Table 2 Performance index in each fuzzy reasoning method by means of the change in the no. of fuzzy variables (a) Simpli ed fuzzy reasoning method with input variables u(t − 3) and y u(t − 3): No. of fuzzy variables 2 3 4
y: No. of fuzzy variables 2
Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular
5
6
PI
E PI
PI
E PI
PI
E PI
PI
E PI
PI
E PI
0.036 0.035 0.038 0.024
0.274 0.275 0.276 0.328
0.040 0.040 0.040 0.022
0.252 0.252 0.252 0.333
0.092 0.092 0.092 0.022
0.215 0.213 0.213 0.332
0.038 0.037 0.037 0.021
0.261 0.261 0.259 0.327
0.034 0.032 0.036 0.021
0.272 0.267 0.262 0.330
(b) Simpli ed fuzzy reasoning method with input variables u(t − 4) and y u(t − 4): No. of fuzzy variables 2 3
y: No. of fuzzy variables
2
Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope)
4
PI
E PI
PI
E PI
PI
E PI
0.090 0.092 0.101
0.265 0.266 0.259
0.093 0.093 0.093
0.208 0.208 0.209
0.039 0.037 0.037
0.257 0.260 0.260
(c) Linear fuzzy reasoning method with input variables u(t − 3) and y
y: No. of fuzzy variables
2
Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular
u(t − 3): No. of fuzzy variables 2 3
4
PI
E PI
PI
E PI
PI
E PI
PI
E PI
0.020 0.020 0.020 0.022
0.278 0.272 0.271 0.326
0.019 0.023 0.018 0.021
0.270 0.279 0.285 0.364
0.035 0.183 0.021 0.020
0.269 0.331 0.282 0.333
0.029 0.020 0.020 0.019
0.279 0.293 0.294 0.340
5
(d) Linear fuzzy reasoning method with input variables u(t − 4) and y u(t − 4): No. of fuzzy variables 2 3
y: No. of fuzzy variables
2
Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope)
measure the biological oxygen demand (bod) and the concentration of suspended solid (ss) in in uent sewage at the primary sedimentation basin, and euent bod (ebod) and ss (ess) in the euent sewage at the nal sedimentation basin. Because ebod and ess are changed, depending on the bod and ss, dissolved
4
PI
E PI
PI
E PI
PI
E PI
0.072 0.072 0.071
0.206 0.209 0.212
0.071 0.090 0.072
0.334 0.256 0.212
0.069 0.091 0.079
0.314 0.233 0.219
oxygen set-point (dosp) and recycle sludge ratio setpoint (rrsp) are set so that ess and ebod should be kept up less than the prescribed small quantity. Ebod and ess depend on mixed liquid suspended solid (mlss), waste sluge ratio (wsr), rrsp and dosp. Bod has a correlation with ss.
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
215
Fig. 4. Data points induced by I=O data set (u(t−3); y(t−1); y(t)).
Fig. 5. Data points induced by I=O data set (u(t−4); y(t−1); y(t)).
In this paper, a sewage treatment system plant in Seoul, Korea, is chosen as a model. The rule-based fuzzy modeling by two kinds of fuzzy inference is carried out using 52 pairs of input–output data obtained from the activated sludge process. From four input variables, we choose two input variables that minimize the evaluation criterion and fuzzy rule number, and extract more than two fuzzy partitions (Big and Small) from each input–output pair of data. The identi ed parameters of premise and consequence of the optimal fuzzy rules are obtained using the improved complex method. From the two-dimensional plot of the data set shown in Figs. 10 –15, in the case of the training data, the data sets (mlss, dosp, ess) and (rrsp, dosp, ess) exhibit less uniform and more sparse distribution than any other data set. Therefore, we can anticipate that the fuzzy model structure for the fuzzy partition of data set (mlss, dosp, ess) and (rrsp, dosp, ess) could perform
Fig. 6. The comparison of original data and output data for fuzzy model No. 2 (Table 1(a)). (a) In the case of training data. (b) In the case of testing data.
Fig. 7. Convergence procedure to optimal value of PI & E PI for fuzzy model No. 2 (Table 1(a)).
216
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
Fig. 8. Convergence procedure to optimal value of maximal and minimal point of input fuzzy membership functions, (umin (t − 3); umax (t − 3)) and (ymin (t − 1); ymax (t − 1)) for fuzzy model No. 2 (Table 1(a)). (a) In the case of (umin (t − 3); umax (t − 3)). (b) In the case of (ymin (t − 1); ymax (t − 1)).
a little worse than in the remaining scenarios (Fig. 16; Tables 4 and 5). 4.3. Trac route choice process Lately, researchers and ocials who work in the eld of trac planning and engineering have expressed a lot of interest about the assignment of trac load in road network. From 1950s, many highways have been planned and constructed for transportation
of men and materials due to the economic development of the advanced industrial nations such as United States, Europe and Japan. It has triggered an important issue on how to assign trac load and setup e ectively the harmonious transportation system from two roads or a few roads all being in a competitive relationship. To solve such problems, lots of research is being carried out to get the optimal model of trac transition and trac route related to the trac planning.
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
217
Table 3 Comparison of identi cation errors with previous fuzzy models Model name
PI
Total no. of rules
Tong’s model [13] Pedrycz’s model [9] Xu’s model [15] Sugeno’s model [10] Our model Simpli ed
0.469 0.776 0.328 0.355 0.137 0.155 0.180 0.149 0.127 0.123 0.124 0.134
19 20 25 6 4 4 4 6 4 4 4 4
Linear
Gaussian (no-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular
Fig. 9. Con guration of the sewage treatment system.
The selection of a certain trac route is an example of a complex driver’s decision making. Ambiguity (or fuzziness) inherently associated with trac route problem exhibits the implicit meaning such as: (1) ambiguity of model’s input data, (2) fuzziness of human cognition, (3) ambiguity of human decision making. The usage of the knowledge data based on information mentioned above, a method was proposed to make a trac route choice model [1]. Generally, most of these methods perform modeling from valid infor-
mation which are obtained from human knowledge. This method is di erent from some other modeling methods which are primarily based upon statistical approaches. We can think of many cases of route choice model. But, we con ne our interest to the assignment problem between road network and rail system or route choice problem between road and highway. As shown in Fig. 17, Fig. 1 illustrates a simple example of a route structure from O to D. Even though we can
218
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
Fig. 10. Data points induced by I=O data set (mlss, wsr, ess).
Fig. 11. Data points induced by I=O data set (mlss, rrsp, ess).
Fig. 12. Data points induced by I=O data set (mlss, dosp, ess).
consider various types of cost related to many decision criterion for evaluation about each route for generalization, in this paper, we consider only time costs and trac costs as the essential components of the objective function.
Fig. 13. Data points induced by I=O data set (wsr, rrsp, ess).
Fig. 14. Data points induced by I=O data set (wsr, dosp, ess).
Fig. 15. Data points induced by I=O data set (rrsp, dosp, ess).
We consider a simple problem related to trac route choice which is based on two routes mentioned above. The model was considered as a conventional example for the trac route choice model [5]. Then, we assume that the object for trac routes is a driver and our interest is focused on how to express trac behavior at
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
219
R 2 : If u(t − 3) is Small1 & y(t − 1) is Big2 then y(t) = a2 . R 3 : If u(t − 3) is Big1 & y(t − 1) is Small2 then y(t) = a3 . R4 : If u(t − 3) is Big1 & y(t − 1) is Big2 then y(t) = a4 . The initial and nal tuned values of each parameter associated with the fuzzy rules mentioned above are like in Table 10. Those values of Gaussian membership functions of fuzzy input variables are shown in Fig. 21. The dotted line in Fig. 21 represents the nal tuned values.
Fig. 16. Convergence procedure to optimal value of PI & E PI for fuzzy model No. 5 (Table 4(a)).
the individual level. The performance index is de ned as shown in Eq. (12). The inputs T 1 and T 2 denote a trac and time cost associated with routes 1 and 2, respectively. The output P1n is the choice probability ratio of route 1 (Tables 6 and 7). The performance index of each model is compared with other fuzzy modeling methods and the results are summarized in Tables 8 and 9. This comparison reveals that performance index for the training data only; the testing data are not studied. In other words, the weighting factors assume the values PARA1 = 1 and PARA2 = 0, respectively. The analysis of the experimental data allows us to draw some general conclusions: 4.3.1. Gas furnace process The most suitable design option is to choose the simpli ed fuzzy reasoning method, optimize the second weighting parameter (PARA2) of the objective function and use the Gaussian membership function. The best results for the training set can be derived for the fuzzy model with the linear reasoning method. As Table 2 reveals, the increase in the number of the linguistic terms for the input space improves the performance of the model. The optimal fuzzy rules obtained from model 6 with di erent slope in Table 1 are as follows: R1 : If u(t − 3) is Small1 & y(t − 1) is Small2 then y(t) = a1 .
4.3.2. Sewage treatment process As this data set is far smaller than the previous one, the behavior of the model could be very di erent depending on whether we are concerned with the testing or the training set. In this case the PI and E PI make a huge di erence. We may anticipate that equipping the model with modi able membership functions could be bene cial. In fact, this is supported by the completed experiments. For the training and testing data sets, we observe a better performance for the linear and simpli ed reasoning method, respectively. As shown in Table 5 in the case of linear fuzzy reasoning method, the increase in the number of membership function of input fuzzy variables make PI much better, but E PI becomes much worse due to the excessive over tting of the consequence linear equations. The optimal fuzzy rules obtained from model 7 with same slope in Table 5 are as follows: R1 : If wsr is Small1 & dosp is Small2 then ess = a1 . R 2 : If wsr is Small1 & dosp is Big2 then ess = a2 . R 3 : If wsr is Big1 & dosp is Small2 then ess = a3 . R4 : If wsr is Big1 & dosp is Big2 then ess = a4 . The initial and nal tuned values of each parameter associated with the fuzzy rules mentioned above are visualized in Table 11. These values of Gaussian membership functions of fuzzy input variables are shown in Fig. 22. 4.3.3. Trac route choice process As mentioned in the gas furnace and sewage treatment process, the quantity and sparse distribution of data set a ect the structure of the optimal fuzzy model such as the reasoning method, number of membership
220
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
221
222
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
223
Fig. 17. Simple example of route choice.
Table 6 Example data for the logit model No
Route
T1
T2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
2 2 1 2 2 1 1 2 2 2 2 1 1 2 1 1 2 1 1 2 1
52.9 4.1 4.1 56.2 51.8 0.2 27.6 89.9 41.5 95.0 99.1 18.5 82.0 8.6 22.5 51.4 81.0 51.0 62.2 95.1 41.6
4.4 28.5 86.9 31.6 20.2 91.2 79.7 2.2 24.5 43.5 8.4 84.0 38.0 1.6 74.1 83.8 19.2 85.0 90.1 22.2 91.5
functions of input fuzzy variables and membership function type. The optimal fuzzy rules obtained from model 4 with di erent slopes in Table 7a are as follows: R1 : If T1 is Small1 & T2 is Small2 then route = a1 . R 2 : If T1 is Small1 & T2 is Big2 then route = a2 . R 3 : If T1 is Big1 & T2 is Small2 then route = a3 . R4 : If T1 is Big1 & T2 is Big2 then route = a4 . The initial and nal tuned values of each parameter associated with the fuzzy rules mentioned above are like in Table 12. Those values of Gaussian membership functions of fuzzy input variables are shown in Fig. 23. Again, as in the previous experiments we worked with various optimization scenarios utilizing all parameters.
Fig. 18. The comparison of original data and output data for fuzzy model No. 1 (Table 7(a)). (a) In the case of the training data. (b) In the case of the testing data.
When we apply three types of Gaussian functions and triangular function to a premise structure as the membership function of fuzzy input variables, we can compare and analyze the characteristics and the output
224
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
225
226
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
Table 8 Performance index in each fuzzy reasoning method by means of the change in the number of fuzzy variables T 1: No. of fuzzy variables 2 3
(a) Simpli ed fuzzy reasoning method T 2: No. of fuzzy variables 2 Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular 3 Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular 4 Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular (b) Linear fuzzy reasoning method T 2: No. of Fuzzy variables 2
3
4
Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular
4
PI
E PI
PI
E PI
PI
E PI
1.696 1.710 1.690 1.768 1.913 0.00009 0.028 1.466 1.626 0.000004 0.00002 1.603
8.663 8.710 8.655 8.709 8.682 7.000 7.207 9.587 8.388 7.993 7.998 8.792
1.585 1.361 1.488 1.631 0.751 0.002 0.009 1.690 0.098 0.002 0.046 1.179
8.540 7.030 7.254 8.551 7.977 7.003 7.028 8.533 8.629 7.039 7.261 8.797
1.383 1.357 1.422 1.605 0.114 0.107 0.164
7.129 7.020 7.107 7.855 7.314 7.305 7.245
0.0011 0.0002 0.0003
7.333 7.351 7.570
1.477 0.52 1.326 2.016 0.056 0.067 0.052 0.480 0.002 0.0003 0.002 0.001
8.720 7.57 8.469 8.209 8.625 8.715 8.720 8.871 7.977 8.338 8.183 10.270
0.007 0.059 0.003 0.787 0.0001 0.005 0.001 0.00001 0.00004 0.0001 0.001 0.003
8.943 7.404 9.394 7.604 7.971 7.415 7.677 15.878 8.072 8.141 7.938 8.043
0.050 0.244 0.376 0.006 0.010 0.0002 0.00006
8.844 7.741 8.245 8.787 16.958 7.714 8.419
0.0014 0.014 0.00001
8.663 7.604 9.281
Table 9 Comparison of identi cation errors with previous models Model type
PI
Shooting ratio (%)
1) 2) 3) 4)
5.452 2.0 0.497 1.178 0.000002 0.000001 0.000001 0.01 0.006 0.0001 0.013 0.009
90.4 85.7 95.2 90.4 99.9 99.9 99.9 99.7 99.9 99.9 99.7 99.9
BL (Binary Logit) model [5] PS (Production System) model [1] Neural networks model [1] Fuzzy neural networks model [1] Our model Simpli ed
Linear
Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular Gaussian ( xed-slope) Gaussian (same-slope) Gaussian (di erent-slope) Triangular
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
227
Fig. 19. Convergence procedure to optimal value of PI & E PI for fuzzy model No. 1 (Table 7(a)).
Fig. 21. The initial and nal tuned values of Gaussian-like membership functions for model 6 with di erent slope in Table 1(a)). (a) In the case of membership function of fuzzy variable u(t − 3). (b) In the case of membership function of fuzzy variable y(t − 1).
Fig. 20. Convergence procedure to optimal value of maximal and minimal point of input fuzzy membership functions (T 1min ; T 1max ) and (T 2min ; T 2max ) for fuzzy model No. 1 (Table 7(a)). (a) In the case of T 1min . (b) In the case of T 1max . (c) In the case of (T 2min ; T 2max ).
228
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
Fig. 22. The initial and nal tuned values of Gaussian-like membership functions for model 7 with same slopes in Table 5(a). (a) In the case of membership functions of fuzzy variable wsr. (b) In the case of membership functions of fuzzy variable dosp.
performance of fuzzy model according to each process. As shown in the Tables 1, 2, 4, 5, 7 and 8, the output performance index for testing data in the case of the Gaussian membership function is better than that produced in the case of the triangular membership function. Considering the testing data we obtain better performance for the linear inference method; the improvement occurs on changing the values of the weighting factors in the performance index. By increasing the number of the membership functions in each of input fuzzy variables, in the simpli ed inference method we can nd the optimal fuzzy model with better output performance for both training data and testing data, but in the linear inference method, it is dicult to obtain better output performance from the testing data because of the excessive over tting of consequence linear equations. From the observation and characteristic analysis of two fuzzy in-
Fig. 23. The initial and nal tuned values of Gaussian-like membership functions for model 4 with di erent slopes in Table 7(a). (a) In the case of membership functions of fuzzy variable T 1. (b) In the case of membership functions of fuzzy variable T 2.
ference methods, type of membership functions, no. of membership function of fuzzy input variables, and the weighting factors of the objective function shown in this paper, it is available and feasible to design the optimal fuzzy model structure with a better performance output. 5. Conclusions In this paper, the ecient identi cation technique is presented which automatically extracts the optimal fuzzy rules, using an auto-tuning algorithm and the weighting factors of an object function. The improved complex method, which is a powerful autotuning, is used for auto-tuning of parameters of the premise membership functions in consideration with the overall structure of fuzzy rules. Two types of fuzzy reasoning methods (simpli ed (type 1), linear (type 2)) are used and we consider three types of Gaussian
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
229
230
S. Oh, W. Pedrycz / Fuzzy Sets and Systems 115 (2000) 205–230
membership functions such as xed, same slope and di erent slope, and the triangular membership function. By means of the adjustment of weighting factors of the objective function for both training and testing data, and the auto-tuning of the parameters of each membership function, we can get a better performance of the fuzzy model. According to the increase in the number of membership functions of each input variable of the process system, generally the PI (performance index) for the fuzzy model using training data is improved, but the PI for the fuzzy model using testing data gets worse. The PI for the fuzzy model by means of the linear inference method using the testing data gets much worse than that of the simpli ed inference method. Comparing the performances of the Gaussian-type membership function with xed slope, same slope, and di erent slope, we nd it depends on the amount of data, a certain degree of nonlinearity and a weighting factor as shown in the previous section. Generally, in the case of same-slope, we can get better performance. The PI for testing data in the case of the simpli ed method is better than that produced in the case of the linear method. Furthermore, in this case of the simpli ed method, the di erence between PI (performance index for training data) and E PI (performance index for testing data) is much smaller. Moreover, the performance of the fuzzy model with the simpli ed inference method using testing data is better than that exploiting the linear type of the inference method. Acknowledgement This paper was supported in part by fund for Postdoctoral program (1996) of Korea Science and Engineering Foundation, the Republic of South Korea. References [1] T. Akiyama, Modeling of route choice behaviour using knowledge based techniques, JSCE 11 (1993) 65–72 (in Japanese).
[2] G.E.P. Box et al., Time Series Analysis, Forecasting and Control, Holden Day, San Francisco, CA. [3] E. Czogola, W. Pedrycz, On identi cation in fuzzy systems and its applications in control problems, Fuzzy Sets and Systems 6 (1981) 73 – 83. [4] M.A. Ismail, Soft clustering algorithm and validity of solutions, in: M.M. Gupta (Ed.), Fuzzy Computing Theory, Hardware and Applications, North-Holland, Amsterdam, 1988, pp. 445– 471. [5] S. Morihayashi, The Forecasting and Test of Disaggregate Model, Text of Infrastructure of Civil Engineering of Japan, vol. 15, JSCE, 1984 (in Japanese). [6] S.K. Oh, H.K. Kim, K.B. Woo, Fuzzy modeling for activated sludge process and control systems, Korean Institute Electrical Engr. 40 (10) (1990) 1072–1083. [7] S.K. Oh, K.B. Woo, Fuzzy identi cation by means of fuzzy inference method and its application to waste water treatment system, J. Korean Institute Telemat. Electronics (KITE) (1994) 43–52. [8] W. Pedrycz, Numerical and application aspects of fuzzy relational equations, Fuzzy Sets and Systems 11 (1983) 1–18. [9] W. Pedrycz, An identi cation algorithm in fuzzy relational system, Fuzzy Sets and Systems 13 (1984) 153–167. [10] M. Sugeno, T. Yasukawa, Linguistic modeling based on numerical data, IFSA’91 Brussels, Computer, Management & System Science, 1991, pp. 264 –267. [11] T. Takagi, M. Sugeno, Fuzzy identi cation of systems and its applications to modeling and control, IEEE Trans. Systems Cybernet. SMC-15 (1) (1985) 116 –132. [12] R.M. Tong, Synthesis of fuzzy models for industrial processes, Int. J. Gen. Systems 4 (1978) 143–162. [13] R.M. Tong, The evaluation of fuzzy models derived from experimental data, Fuzzy Sets and Systems 13 (1980) 1–12. [14] C.W. Xu, Fuzzy system identi cation, IEE Proc. 126 (4) (1989) 146 –150. [15] C.W. Xu, Y. Zailu, Fuzzy model identi cation self-learning for dynamic system, IEEE Trans. Systems Man Cybernet. SMC-17 (4) (1987) 683– 689. [16] L.X. Xuanli, B. Gerado, Validity measure for fuzzy cluster, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-13 (8) (1991) 841– 847.