Fuzzy rule base design using tabu search algorithm for nonlinear system modeling

ISA Transactions 47 (2008) 32–44 www.elsevier.com/locate/isatrans

Fuzzy rule base design using tabu search algorithm for nonlinear system modeling Aytekin Bagis ∗ Department of Electrical Electronic Engineering, Erciyes University, 38039 Kayseri, Turkey Received 15 February 2007; received in revised form 29 August 2007; accepted 4 September 2007 Available online 22 October 2007

Abstract This paper presents an approach to fuzzy rule base design using tabu search algorithm (TSA) for nonlinear system modeling. TSA is used to evolve the structure and the parameter of fuzzy rule base. The use of the TSA, in conjunction with a systematic neighbourhood structure for the determination of fuzzy rule base parameters, leads to a significant improvement in the performance of the model. To demonstrate the effectiveness of the presented method, several numerical examples given in the literature are examined. The results obtained by means of the identified fuzzy rule bases are compared with those belonging to other modeling approaches in the literature. The simulation results indicate that the method based on the use of a TSA performs an important and very effective modeling procedure in fuzzy rule base design in the modeling of the nonlinear or complex systems. c 2007, ISA. Published by Elsevier Ltd. All rights reserved.

Keywords: Fuzzy systems; Fuzzy modeling; Fuzzy rule base design; Nonlinear system modeling; Tabu search

1. Introduction Modeling is an essential part of control design as it provides a way of formally proving properties of the closed loop system. There are many reasons why a model of a process may be required. It could, for example, be used to simulate the real process or it could be used to design a controller. If a good model of the system can be obtained by using knowledge and observations, accurate and efficient solutions to engineering problems such as system design and simulation, process analysis, reliability of the measured data, optimization of the systems can be provided. Conventional system theory relies on crisp mathematical models of systems, such as algebraic and differential equations. However, many real world systems are inherently nonlinear and cannot be represented by linear models used in conventional system identification. For a large number of practical problems, the gathering of an acceptable degree of knowledge needed for physical modeling is a difficult, time consuming and expensive or even impossible task. Expert system techniques based on artificial intelligence can provide ∗ Tel.: +90 (0.352) 437 49 37x32255; fax: +90 (0.352) 437 57 84.

E-mail address: [email protected].

a fast and efficient solution to complex or nonlinear system modeling problems. One of the most successful expert system methods is fuzzy logic based modeling approach [1]. Rule based fuzzy modeling is a powerful technique for the modeling of partly known nonlinear systems. In the modeling of complicated and/or ill-defined processes, precise mathematical models may fail to give satisfactory results. In such cases, fuzzy models can effectively integrate information from different sources (such as physical laws, empirical models, and experimental measurements) and they may be used to properly depict the system uncertainty. If fuzzy rule based models can be identified, fuzzy controllers can be obtained from the fuzzy models and genuine control engineering issues can be addressed for fuzzy controllers, namely stability, robustness etc. [2]. In designing fuzzy models, a major difficulty in the identification of an optimized fuzzy rule base is encountered. Traditional fuzzy rule design is based upon a human operator’s experienced knowledge. Not only is this design method time consuming since it uses trial and error to find good fuzzy rules and membership functions, it is also not guaranteed to find optimal or near optimal fuzzy rules and membership functions [3,4].

c 2007, ISA. Published by Elsevier Ltd. All rights reserved. 0019-0578/$ - see front matter doi:10.1016/j.isatra.2007.09.001

A. Bagis / ISA Transactions 47 (2008) 32–44

There are many studies on fuzzy modeling in the literature. Takagi and Sugeno proposed a new type of fuzzy model called “Takagi–Sugeno fuzzy model (T–S fuzzy model)” [5]. Furthermore, this study proposes a procedure to identify the T–S fuzzy model from input–output data of systems. Sugeno and Yasukawa discussed a general approach to qualitative modeling based on fuzzy logic [6]. It proposes the use of a fuzzy clustering method to identify the structure of a fuzzy model. Wang and Lee proposed a method based on the back-propagation algorithm to determine significant variables without the need of human inspection [7]. A novel fuzzy modeling algorithm was presented by Kim et al. [8]. In the study, fuzzy C regression model clustering algorithm and a gradient descent algorithm are used. Dynamic versions of fuzzy logic systems and their nonsingleton generalizations were investigated by Mouzouris and Mendel [9]. They used a dynamic learning algorithm to train the system parameters. Wang and Langari presented a fuzzy logic approach to complex system modeling that is based on fuzzy discretization technique [10]. A great advantage of the presented approach was that it could determine the premise structure of a model off-line. In [11], a method of Takagi–Sugeno–Kang fuzzy model generation using numerical data as a starting point was described by Kukolj. In addition, many other methods such as neural networks based methods were suggested and were used to build fuzzy models [12–15]. Several models for the identification and control of nonlinear dynamic systems were suggested by Narendra and Parthasarathy [13]. These models, which include multilayer neural networks as well as linear dynamics, can be viewed as generalized neural networks. For the adjustment of parameters, static and dynamic back-propagation methods are used. Jang described an adaptive network based fuzzy inference system [12]. The presented method employed a hybrid learning procedure to construct an input–output mapping based on both human knowledge and stipulated input–output data pairs. By using the architecture of the fuzzy-neural network, simple and effective fuzzy rule based models of complex systems from input–output data were developed by Lin and Cunningham [15]. Babuska and Verbruggen presented an overview of neuro-fuzzy modeling methods for nonlinear system identification [14]. Different optimization algorithms were also used for fuzzy system modeling [16–23]. Hanss presented a special fuzzy modeling method for developing multivariable fuzzy models [20]. This fuzzy model identification procedure was carried out by applying a special clustering method. Kilic et al. proposed a fuzzy system modeling approach based on a probabilistic hill-climbing algorithm as a data analysis and approximate reasoning tool [19]. Evsukoff et al. used a least squares error minimization method for nonlinear fuzzy model identification [16]. A genetic based neuro-fuzzy approach for modeling and control of dynamical systems was presented by Farag et al. [23]. The proposed approach combined the merits of the fuzzy logic theory, neural networks, and genetic algorithms. Wu and Yu used a genetic algorithm and gradient descent based approach to optimize the structure of a Takagi–Sugeno–Kang

33

model [17]. An evolutionary design method of fuzzy rule base for nonlinear system modeling and control was presented by Kang et al. [22]. Cordon et al. presented an overview of the representative methods in genetic fuzzy systems proposed in the literature [21]. This paper presents an approach to fuzzy rule base design using a tabu search algorithm (TSA) for nonlinear system modeling. TSA is used to evolve the structure and the parameters of fuzzy rule base. To check the effectiveness of the presented approach, several numerical examples given in the literature are examined. The results obtained by means of the identified fuzzy rule bases are compared with those belonging to other modeling approaches in the literature. The remainder of the paper is organized as follows. In Section 2, a brief review of the TSA is introduced. Structure of the identified fuzzy rule base and design method is developed in Section 3. Simulation results and some conclusions are given in Sections 4 and 5, respectively. 2. Tabu search algorithm (TSA) TSA was proposed by Glover as an intelligent optimization technique to overcome local optimality in solution processes for hard combinatorial optimization problems [24]. This algorithm consists of the systematic prohibition of some solutions to prevent cycling and to avoid the risk of trapping at a local optimum. New solutions are searched in the neighbourhood of the current one. In TSA, there are several important concepts: Neighbourhood: To improve the quality of the solutions visited, the search moves from one solution to another using a neighbourhood structure. The neighbourhood of a solution is the set of all formations that can be arrived at by a move. The move is a process that transforms the current solution to its neighbouring solution. If, after making a move, a solution is found which is better than all solutions found in prior moves than the new solution is saved as the new best solution. This neighbourhood procedure is applied to the only one best solution of each generation. Considering a chromosome with n bits, the number of the neighbour solutions is n, as depicted in Fig. 1. To obtain the neighbour solutions for the best solution, encoding and decoding operations are applied. In this work, using different bit sizes for each of the parameters we have tried to achieve a good balance between the algorithm performance and the computational cost, and we finally adopt a bit size of 8. Therefore, total number of bits required for defining a fuzzy rule base parameter set is [rule number × parameter number × bit number]. According to simulation results, this bit size is suitable for the main process in the algorithm described in this paper. Tabu list: One of the main ideas of TSA is the use of a flexible memory (tabu list). The objective of the tabu list is to exclude moves which would bring the algorithm back to where it was at some previous iteration and keep it trapped in a local minimum. To avoid performing a move returning to a recently visited region, the reverses of last moves are forbidden. In other words, these moves are considered as “tabu” and are recorded in the tabu list. The tabu list stores all tabu moves that are

34

A. Bagis / ISA Transactions 47 (2008) 32–44

Fig. 1. Neighbourhood structure for a solution. Fig. 2. System modeling scheme. Table 1 The values of the control parameters for TSA Tabu search algorithm Recency factor (r ) Frequency factor ( f ) Generation number Tabu conditions Number of variables for a rule Number of neighbourhood per variable Tabu list size

0.2 2.0 2000 (1) Recency (k) > (r × M), (2) Frequency (k) < ( f × avgfreq) 9, for system with 2 inputs and 1 output 13, for system with 3 inputs and 1 output 8 Rule number × number of the variables

not permitted to be applied to the present solution. This list is initialized empty, constructed in consecutive iterations of the search and updated circularly in later iterations. Tabu conditions: The number of the possible moves is determined by the tabu list. If a solution produced by a move that is not on the tabu list is better than all solutions found in prior iterations, then this solution is saved as the best solution. At each of the iterations in the optimization process, candidate solutions are checked with respect to “tabu conditions”, and hence, the next solution is determined depending on evaluation values and tabu conditions. The re-generation of a solution previously obtained is avoided by using the tabu conditions (or restrictions) on the possible moves. The tabu conditions are usually based on two important factors: frequency memory and recency memory. Frequency memory keeps the knowledge of how often the same solutions have been made in the past. Recency memory prevents cycles of length less than or equal to a predetermined number of iterations. The tabu conditions used in this work are as the following: If the element k of a solution vector does not satisfy the conditions (i) recency (k) > r.M, (ii) frequency (k) < f.avgfreq, then it is accepted as tabu and not used to create a neighbour. Here, r and f are recency and frequency factors, M is the number of elements in the solution vector, and avgfreq is the average frequency value. In this work, after some experimentation, the algorithm parameters r and f have been set to the values of 0.2 and 2.0, respectively. The control parameter values for the TS algorithm are given in Table 1. For all of the simulations, the iteration or generation number is taken as 2000. The number of bits per variable is 8. Thus, if a rule has the n parameters, the total number of bits in the string describing each member of the solutions is (n × 8).

Aspiration criterion: TS has another important element called aspiration mechanism. If a move on the tabu list leads to a solution with a value strictly better than the best obtained so far, it is possible to allow this move to get out of the tabu list. This property is used to avoid removing very good moves from consideration and plays an essential role in the search process. In this work, if all available moves are classified as tabu, then a least tabu move is selected for a new solution. The least tabu move means that this solution is changed less recently and frequently among them but it is still classified as tabu. In this work, maximum number of iterations reached is selected as the stopping criterion of the search process. The basic procedure of TSA to be utilized in this paper is described as follows: Initial solution; While predetermined stopping criteria not satisfied; { Create a set of neighbour solutions; Evaluate the neighbour solutions; Choose the best admissible solution; Test the tabu conditions; Perform the aspiration test; Update tabu list; } This procedure is repeated until a satisfactory solution is obtained or predetermined number of generations is elapsed. For all of the modeling examples, the number of the generation was selected as 2000. 3. Structure of the identified fuzzy rule base Fig. 2 shows the system modeling scheme. In this figure, y p (t + 1) is the output of the system to be modeled, and ym (t + 1) is the output of the fuzzy model optimized by the TSA. Input of the TSA is the error value, and it is calculated by {error = y p (t + 1) − ym (t + 1)}. The major objective of the algorithm is to determine the most appropriate fuzzy model parameters and to reduce the error factor. In fuzzy models, structure selection involves the following choices: (a) input and output variables (b) structure of the rules

A. Bagis / ISA Transactions 47 (2008) 32–44

35

Fig. 3. (a) Parameter matrix representing the membership functions, (b) Fuzzy rule base represented by parameter matrices, (c) The rule base coding.

(c) number and type of membership functions for each variable (d) type of the inference mechanism, connective operators, defuzzification method. After the structure is fixed, the performance of a fuzzy model can be fine-tuned by adjusting its parameters. Tunable parameters of linguistic models are the antecedent parameters and consequent membership functions and the rules. Rules in the fuzzy rule base are described by the membership functions. Fuzzy membership functions can have different shapes depending on the designer’s preference or experience. In general, fuzzy engineers have found that triangular and trapezoidal shapes help capture the modeller’s sense of fuzzy numbers and simplify computation. In this study, input variables are characterized by triangular membership functions, and the output variables are characterized by fuzzy singletons. A rule in the rule base has a general form expressed by Ri : If x1 is Ai1 (x1 )

and

x2 is Ai2 (x2 ) then y is Bi

(1)

where Ri is the ith rule, xj is the input variable, and y is the output variable. Aij (xj ) is the fuzzy linguistic value defined as in Eq. (2) and Bi takes the role of fuzzy singleton, i.e., real number. Aij = max(min[(xj − a)/(b − a), (c − xj )/(c − b)], 0).

(2)

In Eq. (2); a, b, and c are the main parameters of a triangular membership function. These parameters represent the locations of starting point, peak point, and the ending point for a triangle shaped membership function, respectively. Deciding the number of fuzzy rules is a very critical issue since it plays key roles in the fuzzy modeling of the systems. Except for some propositions, unfortunately, there is no systematic and efficient procedure for choosing the most

appropriate rule number. A reasonable number of fuzzy rules without losing too much information about the system to be modeled must be carefully obtained. In this work, using different rule sizes we have tried to achieve a good balance between the model performance and rule complexity. Another important element in combining a TSA and a fuzzy model is the encoding method used to represent a fuzzy rule base as a probable solution element on which the TSA operates. Selecting an appropriate encoding procedure is a critical factor that affects the efficiency of the TSA and the final solution. The membership functions of the input variables are characterized by three numerical values. On the other hand, the output variables are defined by singleton numerical values. These numerical values represent the position of the membership function on the universe of discourse. The parameter matrix defining the membership functions and fuzzy rule base represented by the parameter matrices is formulated in Fig. 3. The parameter matrix that contains the parameters for defining the membership functions within the fuzzy rule base will be of one-dimensional matrix form. For each rule in the fuzzy rule base, the adjustment parameters used in the reasoning procedure are also involved in this parameter matrix. In Fig. 3, these parameters are stated by di for each input variable. Thus, the size of matrix is determined by {the number of rules × [(the number of input variables × 3) + (the number of output variables × 1) + (the number of input variables)]}. This value of the matrix is also equal to the size of the tabu list. For a fuzzy model with two inputs and one output, each fuzzy rule is represented by nine parameters. The rule structure coding is organized as shown in Fig. 3(c). For example, if the modeling system has two inputs and one output, 144 bits are used to represent a probable solution with two rules optimized by the algorithm (Fig. 3(c)). Initializing the solution is based

36

A. Bagis / ISA Transactions 47 (2008) 32–44

Table 2 Optimized parameter matrix for fuzzy rule base Rules

Input 1, u(t − 4) ai1 ai2

ai3

Input 2, y(t − 1) bi1 bi2

bi3

Output, y(t) ci

Adjustment parameters di1 di2

Normalized parameters (1) 0.1579 (2) 0.0552 (3) 0.0000 (4) 0.5640

0.1720 0.3294 0.1569 0.7529

0.4845 2.5020 0.8908 1.2549

0.1485 0.0000 0.3476 0.2813

0.3595 0.2667 1.8823 0.4072

1.0196 0.8046 2.7922 0.8314

0.7658 0.0156 0.9412 0.2500

0.4863 0.1489 0.4706 0.2510

0.2813 0.5020 1.1214 0.0784

Denormalized parameters (1) −2.0526 (2) −2.6688 (3) −3.0000 (4) 0.3840

−1.9680 −1.0236 −2.0586 1.5174

−0.0930 12.0120 2.3448 4.5294

46.6730 44.0000 50.2568 49.0634

50.4710 48.8006 77.8814 51.3296

62.3528 58.4828 94.2596 58.9652

57.7844 44.2808 60.9416 48.5000

0.4863 0.1489 0.4706 0.2510

0.2813 0.5020 1.1214 0.0784

on the randomly generated values from the given searching intervals. In this study, the following reasoning procedures are considered: (a) Given input data x1 and x2 , calculate the degree of the fulfillment ωi in the premise for the ith rule as in ωi = Ai1 (x1k ).di1 + Ai2 (x2k ).di2 , k = input data number.

(3)

(b) Calculate the inferred value yi by taking the weighted average of Bi with respect to ωi as in r P

yi =

ωi .Bi

i=1 r P

(4) ωi

i=1

where r is the number of fuzzy rules. To evaluate the performance of a fuzzy model, there are two important performance indices in the literature. These are the mean squared error (MSE) and a performance parameter stated as PI. In the MSE, the error is the difference between the actual output and the estimated output by the fuzzy model. The MSE and PI are calculated from N data point as MSE = s PI =

N 1 X (O d − Ok )2 N k=1 k N P k=1

(5a)

(Okd − Ok )2

N P Od k=1

(5b)

k

where Okd , k = 1, . . . , N are the actual or desired output values and Ok , k = 1, . . . , N are the outputs from the model. These performance indices provide a means for evaluating the performance of a fuzzy model with the selected fuzzy rule base in the process of evolution, so that an optimized model would be developed by the best individual. During the evolution, the quality of the best solution at each generation is calculated by Eq. (5a) or (5b).

4. Simulation results Five numerical examples are provided in this section to illustrate the performance of the presented approach. All these examples have their origin in the existing literature. The datasets used in the first and third examples are derived from practical processes; while the dataset used in the second and fourth examples is generated from the simulation of analytical models. In the first example, the fuzzy modeling of a dynamical process using a famous example of the system identification given by Box and Jenkins [25] is discussed. The third example is used to show how to build a model of a human operator’s control action in a chemical plant. The structure of the human operation is shown in Sugeno and Yasukawa [6]. Furthermore, input–output data for human operation at the chemical process are also listed in [17]. In the fifth example, modeling of a real reservoir system is considered [3,4]. And, the datasets used in this example are real flood hydrograph data of the dynamical system. (a) Box and Jenkins’ gas furnace: The fuzzy model of a dynamic process using a well-known example of the system identification given by Box and Jenkins is considered [6,22,25]. The process is a gas furnace with a single input u(t) and a single output y(t): gas flow rate and CO2 concentration, respectively. The dataset consists of N = 296 pairs of input–output measurements. The inputs to the fuzzy model were selected as u(t −4), y(t −1), respectively. The model output is y(t). For the variables of u(t − 4), y(t − 1), and y(t), normalization intervals were selected as [−3, 3], [44, 62], and [44, 62], respectively. Parameters of the fuzzy model were searched into the interval of [0, 4]. During the evolution, the modeling error of the best individual at each generation was calculated by Eq. (5a). The optimized fuzzy model for a gas furnace and the comparison of model outputs and the original output data are shown in Figs. 4 and 5, and the modeling error is 0.1483. The number of rules in the fuzzy model is only four. Normalized and denormalized parameter matrices of the fuzzy model are also given in Table 2. In Table 3, our fuzzy model is compared with other models identified from the same data. (b) Modeling of a static nonlinear function: In this subsection, double input and single output static function was chosen as

37

A. Bagis / ISA Transactions 47 (2008) 32–44

Fig. 4. Optimized fuzzy rule base for gas furnace. Table 3 Comparison of our model with other models

Fig. 5. Comparison of model outputs and original output data for the gas furnace.

a target system. The function was taken from Narendra and Parthasarathy [13] and represented as y = (1 + x1−2 + x2−1.5 )2 ,

1 ≤ x1 , x2 ≤ 5

(6)

from which 50 data was obtained. From this system equation, 50 input–output data are obtained. In this instance, two random inputs x1 and x2 were inserted to test input identification. Normalization intervals for input–output parameters x1 , x2 , y were selected as [0, 6], [0, 6], [0, 10], respectively. Model parameters were searched in the interval of [0, 4]. During the evolution, the modeling error at each generation was calculated by Eq. (5a). The optimized fuzzy model for nonlinear system and the comparison of model outputs and the original output data are shown in Figs. 6 and 7, respectively, and the modeling error is 0.0019. The rule number of the fuzzy model is

Method

Rule number

MSE

Box and Jenkins (1970) [25] Tong (1978) [26] Pedrycz (1984) [27] Takagi and Sugeno (1985) [5] Xu and Lu (1987) [28] Sugeno and Tanaka (1991) [29] Sugeno and Tanaka (1991) [29] Wang and Langari (1996) [10] Nakoula et al. (1997) [30] Kim et al. (1997) [8] Farag et al. (1998) [23] Kang et al. (2000) [22] Evsukoff et al. (2002) [16] Evsukoff et al. (2002) [16] Our model

– 19 81 6 25 2 2 5 90 2 37 5 36 90 4

0.202 0.469 0.320 0.190 0.328 0.068 0.359 0.158 0.175 0.055 0.111 0.161 0.153 0.090 0.148

five. Normalized and denormalized parameter matrices of the fuzzy model are given in Table 4. Performance indices of the presented method and the other methods in the literature are given in Table 5. To test the validity of the derived model, the later 20 data that are not used in the model derivation are considered by the model, where MSE calculated by (5a) is 0.0043 (Fig. 7(b)). (c) Human operation at a chemical plant: This example deals with the modeling of human operator control in a chemical plant. The plant is for producing a polymer by the polymerization of monomers. Since the start-up of the plant is very complicated, a human operator is used to manually operate the system in this circumstance. The structure of the human operation is given by Sugeno and Yasukawa [6]. Five input candidates, u 1 : the monomer concentration, u 2 : the change of monomer concentration, u 3 : the monomer flow rate, u 4 and u 5 :

38

A. Bagis / ISA Transactions 47 (2008) 32–44

Fig. 6. Optimized fuzzy rule base for a static nonlinear function.

Fig. 7. (a) Comparison of model outputs and original output data for example (b). (b) Test outputs and model outputs for example (b).

the local temperature inside the plant are available, to which the human operator may refer. The output y is the set point for monomer flow rate. There are 70 data points for each of the above six variables (five inputs, and one output). There are different fuzzy models available for modeling this plant. In [6], a six rule qualitative fuzzy model was given by identifying u 1 , u 2 , and u 3 as input variables. In [10], u 1 , u 2 , and u 3 were selected as the input variables, and six clusters from the given data were taken to give six rules and three premise variables of the Takagi–Sugeno–Kang model. In [15], a fuzzy-neural system was used for modeling the plant. The performance measurement was not stated in [6] and [10].

In our approach, the input variables u 1 and u 3 were selected to build the fuzzy model. Normalization intervals for all variables (u 1 , u 3 , y) were selected as [4, 7], [0, 8000], and [0, 8000], respectively. Model parameters were searched in the solution area of [0, 2]. During the evolution, the modeling error at each generation was calculated by Eq. (5b). Optimized model parameters and optimum rule base are given in Table 6 and Fig. 8, respectively. Optimum rule base has only three rules. The comparison of model outputs and the original output data are shown in Fig. 9. In Table 7, our fuzzy model is compared with the other models in the literature.

39

A. Bagis / ISA Transactions 47 (2008) 32–44 Table 4 Optimized parameter matrix for fuzzy rule base Rules

Input 1, x1 ai1

ai2

ai3

Input 2, x2 bi1

bi2

bi3

Output, y ci

Adjustment parameters di1 di2

Normalized parameters (1) 0.5348 (2) 0.2285 (3) 0.1661 (4) 0.1661 (5) 0.0938

0.7867 0.2286 0.2247 0.2516 0.0956

0.7887 0.5411 3.1662 0.2657 0.2657

0.3438 0.0000 0.1641 0.0938 0.2130

0.7735 0.0728 0.3141 0.5781 0.3439

0.8282 0.3282 0.6661 1.9137 0.3594

0.1730 1.0000 0.2328 0.0313 0.9705

0.0625 0.1279 0.2731 0.3516 1.7887

1.4922 0.2500 1.0490 0.5742 0.0509

Denormalized parameters (1) 3.2088 (2) 1.3710 (3) 0.9966 (4) 0.9966 (5) 0.5628

4.7202 1.3716 1.3482 1.5096 0.5736

4.7322 3.2466 18.9972 1.5942 1.5942

2.0628 0.0000 0.9846 0.5628 1.2780

4.6410 0.4368 1.8846 3.4686 2.0634

4.9692 1.9692 3.9966 11.4822 2.1564

1.7300 10.000 2.3280 0.3130 9.7050

0.0625 0.1279 0.2731 0.3516 1.7887

1.4922 0.2500 1.0490 0.5742 0.0509

Fig. 8. Optimized fuzzy rule base for human operation at a chemical plant. Table 5 Comparison of different models for example (b) Method

Rule number

MSE

Sugeno and Yasukawa (1993) [6] Lin and Cunningham (1995) [15] Kim et al. (1997) [8] Our model

6 3 3 5

0.0790 0.0035 0.0197 0.0019

(d) Nonlinear differential equation: This example is taken from Narendra and Parthasarathy [13] in which the plant to be identified is given by the second-order highly nonlinear difference equation y(k) =

y(k − 1).y(k − 2).(y(k − 1) + 2.5) + u(k). 1 + y 2 (k − 1) + y 2 (k − 2)

(7)

Training data of 500 points are generated from the plant model, assuming a random input signal u(k) uniformly distributed in the interval [−2, 2]. This unpredictable input “u(k)” is randomly generated and is inserted into the system. The first 500 data points are produced from (7) for model identification. To demonstrate the performance of the fuzzy

Fig. 9. Comparison of model outputs and original output data for the chemical plant.

model, a different type sinusoidal input data u(k) is inserted into the system y(k) (Fig. 10). The model has three inputs u(k), y(k − 1), and y(k − 2), and a single output y(k). Normalization intervals for the inputs

40

A. Bagis / ISA Transactions 47 (2008) 32–44

Table 6 Optimized parameter matrix for fuzzy rule base Rules

Input 1, u1 ai1

ai2

ai3

Input 2, u3 bi1

bi2

bi3

Output, y ci

Adjustment parameters di1 di2

Normalized parameters (1) 0.0000 (2) 0.1719 (3) 0.2537

0.3130 0.4840 0.3387

1.1006 0.7141 0.6829

0.5938 0.4527 0.0430

1.4525 0.4844 0.0489

1.4642 0.8908 0.5266

0.8827 0.4172 0.0110

0.2544 0.0774 0.0351

1.4486 0.3157 0.8387

Denormalized parameters (1) 4.0000 (2) 4.5157 (3) 4.7611

4.9390 5.4520 5.0161

7.3018 6.1423 6.0487

4750.40 3621.60 344.00

11 620.00 3 875.20 391.20

11 713.60 7 126.40 4 212.80

7061.60 3337.60 88.00

0.2544 0.0774 0.0351

1.4486 0.3157 0.8387

Table 7 Comparison of different models for human operation at a chemical plant Method

Model inputs

Rule number

PI

Sugeno and Yasukawa (1993) [6] Lin and Cunningham (1995) [15] Wu and Yu (2000) [17] Our model

u1, u2, u3

6

Not stated

u1, u2, u3

7

0.002245

u3, u4 u1, u3

4 3

0.002000 0.001700

were selected as [−2, 2], [−5, 5], and [−5, 5], respectively. The search interval of all the variables is [0, 4]. After the learning process is finished, the model is tested by applying a sinusoidal input data u(k) = sin(2πk/25) to the fuzzy model. The optimized fuzzy model is given in Fig. 11. The rule number of the fuzzy model is four. Normalized and denormalized parameter matrices of the fuzzy model are also given in Table 8. The output of both the fuzzy model and the actual model are shown in Fig. 12. The fuzzy model has a good match with the actual model with a MSE of 0.0378. Table 9 compares our modeling approach with the other approaches in the literature. The models are obtained from the previously generated 500 data pairs and tested by applying a sinusoidal input signal u(k) = sin(2πk/25). (e) Modeling of a real reservoir system: This example includes the modeling of the nonlinear relationship among the main variables of the reservoir in a reservoir management system.

Fig. 10. Train and test inputs for a nonlinear differential equation. Table 9 Comparison of different models for the nonlinear differential equation Method

Rule number

Sugeno’s model (1991, 1993) [6,29] Farag’s model (1998) [23] Wang’s model (1999) [7] Evsukoff’s model (2002) [16] Our model

Learning MSE

Testing MSE

12

0.5072

0.2447

75 8 100 4

0.0374 0.6184 0.1577 0.0341

0.0403 0.2037 0.0185 0.0378

Table 8 Optimized parameter matrix for fuzzy rule base Rules

Input 1, u(k) ai1 ai2

ai3

Input 2, y(k − 1) bi1 bi2

bi3

Input 3, y(k − 2) ci1 ci2

ci3

Output, y(k) di

Adjustment parameters ei1 ei2 ei3

Normalized parameters (1) 0.0000 0.0001 (2) 0.0000 0.0000 (3) 0.3752 1.0179 (4) 0.1255 0.9849

0.8363 0.3381 1.9853 3.8359

0.1212 0.1250 0.5814 0.4385

0.4045 0.1760 0.9993 0.8799

0.6214 0.5904 1.1009 3.3353

0.0899 0.0977 0.9223 0.5178

0.2462 0.1055 0.9231 0.6566

1.0005 1.9072 0.9243 0.9385

0.5226 0.1055 1.0924 1.3333

1.3420 0.7062 1.3952 0.4279

2.5905 0.4260 1.0081 0.6614

1.2435 1.4667 0.4905 0.3088

Denormalized parameters (1) −2.000 −1.999 (2) −2.000 −2.000 (3) −0.499 2.072 (4) −1.498 1.940

1.345 −0.648 5.941 13.344

−3.788 −3.750 0.814 −0.615

−0.955 −3.240 4.993 3.799

1.214 0.904 6.009 28.35

−4.101 −4.023 4.223 0.178

−2.538 −3.945 4.231 1.566

5.005 14.07 4.243 4.385

0.2260 −3.9450 5.9240 8.3330

1.3420 0.7062 1.3952 0.4279

2.5905 0.4260 1.0081 0.6614

1.2435 1.4667 0.4905 0.3088

A. Bagis / ISA Transactions 47 (2008) 32–44

41

Fig. 11. Optimized fuzzy rule base for the nonlinear differential equation.

Fig. 12. Comparison of model outputs and original output data for the nonlinear differential equation.

Reservoir management system is a control system that manages a spillway gate in a dam to increase or decrease the amount of discharge water [3,4,31]. Reservoir management is a complex, nonlinear, nonstationary control process because: (a) The hydrological conditions have a nonlinear behavior, (b) Determination of the inflow hydrograph can be extremely difficult, (c) Construction of a precise model of this process is very difficult, (d) Owing to human factor, accurate and efficient control of the process is quite difficult. The main variables of a reservoir are reservoir lake level (H , m), spillway gate opening (d, m) and released water amount (Q, m3 /s) [3,4,31]. The fuzzy model is used to model the nonlinear relationship among these variables. Lake level (H ) and spillway gate opening (d) are the inputs of the model. The output variable of the fuzzy model is the released water amount (Q). In this work, the reservoir system of Adana Catalan Dam in Turkey is considered, and the real flood hydrograph data for

Catalan Dam are used [31]. To generate comparable results to those already published; the 548 real data points were used to generate a model using the TSA based method. At the end of the training process, the resultant model was applied to the validation dataset, which was the dataset of 25 unseen data. All of the above information is obtained from the file containing the final project details of Catalan Dam [31]. Normalization intervals for input–output parameters H, d, Q were selected as [117, 131], [0, 12], [0, 10 000], respectively. Model parameters were searched in the interval of [0, 10]. Optimized model parameters and optimum rule base are given in Table 10 and Fig. 13, respectively. The PI was calculated for the model predictions of the validation data and the results have been given in Table 11. The optimum rule base has only five rules. The comparison of model outputs and the original output data is shown in Fig. 14 as three dimensions. In Fig. 15, process outputs for train and test data are given as two dimensions. The results show that the fuzzy rule structure optimized by the TSA can yield a better model than the model which has the structure of a neural network, improving the validation data model prediction PI. Because it did not seem to be the most commonly used parameter in the literature to be considered, the time parameter was not selected as a comparison parameter. The values of the computation time were not stated mostly in the literature to be compared. However, to indicate the computational performance of the TSA, the computation time per evaluation of the algorithm for each problem is given in Table 12. In the simulations, Matlab programming package and Pentium IV 3600 MHz computer were used. It is clear from the figures that fuzzy modeling based TSA is a powerful tool for modeling of nonlinear systems. The approach is easy to use and there is no difficulty in writing its algorithm based modeling procedure.

42

A. Bagis / ISA Transactions 47 (2008) 32–44

Table 10 Optimized parameter matrix for fuzzy rule base Rules

Input 1, H ai1

ai2

ai3

Input 2, d bi1

bi2

bi3

Output, Q ci

Adjustment parameters di1 di2

Normalized parameters (1) 0.0566 (2) 1.3333 (3) 0.1604 (4) 0.4688 (5) 0.0802

0.0661 4.0000 0.1631 0.6421 0.2444

0.6299 4.3175 1.3449 0.9029 0.4727

0.0000 0.5402 0.0059 0.7512 0.0056

0.0000 0.8887 0.5761 0.8905 0.3711

0.5368 2.8233 0.9376 0.9922 0.8422

0.0049 0.8105 0.5102 0.0000 0.4859

1.7488 0.0000 0.0293 0.2344 0.0510

2.4610 1.2308 1.1965 0.0778 1.1429

Denormalized parameters (1) 117.79 (2) 135.67 (3) 119.25 (4) 123.56 (5) 118.12

117.93 173.00 119.28 125.99 120.42

125.82 177.45 135.83 129.64 123.62

0.0000 6.4824 0.0708 9.0144 0.0672

0.0000 10.6644 6.9132 10.6860 4.4532

6.4416 33.8796 11.2512 11.9064 10.1064

49.00 8105 5102 0.00 4859

1.7488 0.0000 0.0293 0.2344 0.0510

2.4610 1.2308 1.1965 0.0778 1.1429

Fig. 13. Optimized fuzzy rule base for a reservoir system.

Fig. 14. Comparison with three dimensions of the outputs for (a) train operation and (b) test operation in the reservoir system.

43

A. Bagis / ISA Transactions 47 (2008) 32–44

Fig. 15. Comparison with two dimensions of the outputs for (a) train operation and (b) test operation in the reservoir system. Table 11 Comparison of different models for a nonlinear reservoir system Method

Model inputs

PI (train)

PI (test)

Neural model [31] Our model

H, d H, d

0.00170 0.00158

0.0056 0.0065

When the model performance and rule size are considered, the resulting outcome is quite exciting. It is clear from the tables that a striking reduction in the value of the performance index (MSE or PI) is obtained without the need of a large number of rules. The low rule size is a significant advantage of the model. The studies in the literature show that TSA is an effective optimization method in the designing of fuzzy systems [3, 4]. By using the control factors such as the initial solution, type of move, size of neighbourhood, tabu list size, aspiration criterion and stopping criterion, the performance of the algorithm can be significantly improved. And, system models with high quality can be obtained. Three important conclusions can be made from the details of the numerical results: First, the requirement for using the large number of rules is successfully removed by employing the TSA. Tabu algorithm enables us to obtain better results when the number of membership functions and rules is as small as possible to solve the problem at hand, a realistic case from an industrial point of view. With tabu search, it is indeed possible to optimally tune the structure of the fuzzy model. Second, even though the presented approach does not show the best performance for all of the problem types considered, it obviously exhibits a competitive performance for the modeling of the nonlinear or complex systems. It has the ability of getting out of local minima and finding global optimal solutions

for multimodal problems in a reasonable time although the traditional optimization algorithms fail to produce global optimal solution for such problems. Thus, it is verified that the presented method is capable of defining the nonlinearities or uncertainties of the system conditions without requiring any expert knowledge or experience about the system to be considered. Third, the numerical results show that the modeling approach based on the TSA has high accuracy (low error) and good generalization ability. Moreover, it can start to learn a process from a very low number of initial data samples and improve the performance of the initial model. Therefore, it can be used effectively for various engineering problems in different areas with the nonlinear behaviors. And, accurate, fast, and reliable fuzzy models can be developed from measured/simulated data. When the process is highly nonlinear, time varying, or when it depends on a lot of parameters, the cost for getting an accurate model is often estimated as being too high. In this case, fuzzy modeling approach based on the use of TSA can be viewed as a useful alternative method. 5. Conclusion In this paper, an efficient approach to fuzzy rule base design using the tabu search algorithm for nonlinear system modeling is presented. To demonstrate the effectiveness of the presented approach, several numerical examples given in the literature are examined. Simulation results show that the fuzzy modeling approach presented here is a powerful tool for the modeling of nonlinear systems. As compared with the other modeling methods, the described approach has the advantages of simplicity, flexibility, and high accuracy. The method is

Table 12 Computational performance of the TSA for different problems Problem

Data number

Rule number

Parameter number per rule

Total parameter number

Neighbourhood number

Computation time per iteration (s)

Box–Jenkins Static nonlinear function Chemical plant Nonlinear differential equation Reservoir operation

296 50 70 500

4 5 3 4

9 9 9 13

36 45 27 52

288 360 216 416

9.18 (for 288 evaluation) 2.37 (for 360 evaluation) 1.27 (for 216 evaluation) 31.27 (for 416 evaluation)

548

5

9

45

360

24.13 (for 360 evaluation)

44

A. Bagis / ISA Transactions 47 (2008) 32–44

easy to use and there is no difficulty in writing its automatic modeling procedure. Especially, when the model performance and rule size are considered, the method based on the use of a tabu search algorithm performs an important and very effective modeling procedure to fuzzy rule base design in the modeling of nonlinear or complex systems. Acknowledgements The author is grateful to the reviewers, and Prof. Dervis Karaboga at Erciyes University in Turkey for their valuable comments and suggestions. References [1] Babuska R. Fuzzy modeling for control. Kluwer Academic Pub.; 1998. [2] Nguyen HT, Sugeno M. Fuzzy systems: Modeling and control. Kluwer Academic Pub.; 1998. [3] Bagis A. Fuzzy and PD controller based intelligent control of spillway gates of dams. J Intell Fuzzy Syst 2003;14(1):25–36. [4] Bagis A. Determining fuzzy membership functions with tabu search—An application to control. Fuzzy Sets and Systems 2003;139:209–25. [5] Takagi T, Sugeno M. Fuzzy identification of systems and its applications to modeling and control. IEEE Trans Syst Man Cybern 1985;15:116–32. [6] Sugeno M, Yasukawa T. A fuzzy logic based approach to qualitative modeling. IEEE Trans Fuzzy Syst 1993;1(1):7–31. [7] Wang S, Lee C. Fuzzy system modeling using linear distance rules. Fuzzy Sets and Systems 1999;108:179–91. [8] Kim E, Park M, Ji S, Park M. A new approach to fuzzy modeling. IEEE Trans Fuzzy Syst 1997;5(3):328–37. [9] Mouzouris GC, Mendel JM. Dynamic non-singleton fuzzy logic systems for nonlinear modeling. IEEE Trans Fuzzy Syst 1997;5(2):199–208. [10] Wang L, Langari R. Complex systems modeling via fuzzy logic. IEEE Trans Syst Man Cybern-Part B: Cybern 1996;26(1):100–6. [11] Kukolj D. Design of adaptive Takagi–Sugeno–Kang fuzzy models. Appl Soft Comput 2002;2:89–103. [12] Jang JR. ANFIS: adaptive network based fuzzy inference system. IEEE Trans Syst Man Cybern 1993;23(3):665–85. [13] Narendra KS, Parthasarathy K. Identification and control of dynamical systems using neural networks. IEEE Trans Neural Netw 1990;1(1):4–27. [14] Babuska R, Verbruggen H. Neuro-fuzzy methods for nonlinear system identification. Ann Rev Control 2003;27:73–85.

[15] Lin Y, Cunningham GA. A new approach to fuzzy-neural system modeling. IEEE Trans Fuzzy Syst 1995;3(2):190–8. [16] Evsukoff A, Branco ACS, Galichet S. Structure identification and parameter optimization for non-linear fuzzy modeling. Fuzzy Sets and Systems 2002;132:173–88. [17] Wu B, Yu X. Fuzzy modeling and identification with genetic algorithm based learning. Fuzzy Sets and Systems 2000;113:351–65. [18] Pham DT, Castellani M. Evolutionary learning of fuzzy models. Eng Appl Artif Intell 2006;19:583–92. [19] Kilic K, Sproule BA, Turksen IB, Naranjo CA. A fuzzy system modeling algorithm for data analysis and approximate reasoning. Robot Auton Syst 2004;49(3–4):173–80. [20] Hanss M. Identification of enhanced fuzzy models with special membership functions and fuzzy rule bases. Eng Appl Artif Intell 1999; 12:309–19. [21] Cordon O, Herrera F, Hoffman F, Magdalena L. Genetic fuzzy systems: Evolutionary tuning and learning of fuzzy knowledge bases. Singapore, New Jersey, London, Hong Kong: World Scientific; 2001. [22] Kang S, Woo C, Hwang H, Woo KB. Evolutionary design of fuzzy rule base for nonlinear system modeling and control. IEEE Trans Fuzzy Syst 2000;8(1):37–45. [23] Farag WA, Quintana VH, Torres GL. A genetic based neuro-fuzzy approach for modeling and control of dynamical systems. IEEE Trans Neural Netw 1998;9(5):756–67. [24] Pham DT, Karaboga D. Intelligent optimisation techniques: Genetic algorithms, tabu search, simulated annealing and neural networks. Springer-Verlag; 2000. [25] Box GEP, Jenkins GM. Time series analysis, forecasting, and control. San Francisco (CA): Holden Day; 1970. [26] Tong RM. Synthesis of fuzzy models for industrial processes-some recent results. Int J General Syst 1978;(4):143–62. [27] Pedrycz W. An identification algorithm in fuzzy relational systems. Fuzzy Sets and Systems 1984;(13):153–67. [28] Xu CW, Lu YZ. Fuzzy model identification and self-learning for dynamic systems. IEEE Trans Syst Man Cybern 1987;(17):683–9. [29] Sugeno M, Tanaka K. Successive identification of a fuzzy model and its application to prediction of a complex system. Fuzzy Sets and Systems 1991;(42):315–34. [30] Nakoula Y, Galichet S, Foulloy L. A learning method for structure and parameter identification of fuzzy linguistic models. In: Hellendoorn H, Driankov D, editors. Selected approaches to fuzzy model identification. Berlin: Springer; 1997. p. 281–319. [31] Bagis A, Karaboga D. Artificial neural networks and fuzzy logic based control of spillway gates of dams. Hydrol Process 2004;18:2485–501.