Fuzzy-tree model and its applications to complex system modeling

FUZZY-TREE MODEL AND ITS APPLICATIONS TO COMPLEX S...

14th World Congress of IFAC

H-Ja-U4-4

Copyright© 1999 IFAC 14th Triennial World Congress, Beijing, P.R. China

FUZZY-TREE MODEL AND ITS APPLICATIONS TO COMPLEX SYSTEM MODELING 1

Jiangang Zbang" Jianqin Mao· Jiyang Dar Kehui Wet*

* The Seventh Research Division,

Beijing University ofAeronautics & Astronau(ics} 10083, Beijing China. E-mail: [email protected] ** P.D.Box 517, Academe ofEngineering PhySics} 610003. Chengdu~ China t

Abstract: Based on binary tree structure and fuzzy logic theory, a Fuzzy... Tree Model applied to complex system modeling is proposed in this paper. Linear models and fuzzy sets are arranged in a tree structure. A training algorithm is used to update linear mode) coefficients and membership functions of the fuzzy sets. Compared with some other modeling methods, such as ANFIS, Fuzzy-Tree Model is of less computation, higher accuracy, and insensitivity to the dimension of the input space. Simulation results give a description of the advantages of this approach. Copyright @ 1999lFAC Keywords: Adaptive filter, tree structures, fuzzy models, nonlinear identification, and fuzzy modeling

1.. INTRODUCTION

In recent years, complex system modeling has become one of the most active areas for research in control theory. Some new theories and methods, such as artificial neural network (Hunt, et al. , 1992; Narendra, and Parthasarathy,. 1990), fuzzy logic theory (Takagi, and Sugeno, 1985; Jang!l 1993), wavelet theory (Sjoberg, et al., 1995) etc., have been applied to complex system modeling, which exhibit great potential in their applications. It is rather difficult to find a gJobaJ function to describe a complex system. Therefore, it is a very natural thought to divide the input space into a few

J This research is supported by National Nature Science Foundation of China(NO. 69434012)

subspaces, in each of which the local mapping are relatively simple~ thus can be described with a simple function, such as linear function. It was on the basis of this idea that Takagi and Sugeno model (T-S Model) (Takagi, and Sugeno, 1985) was presented, which partitioned the input space through searching optimal premise structure. However the searching process is extremely slowly for high dimension data. Adaptive-Network-Based Fuzzy Inference System (ANFIS) (Jang~ 1993) has the advantage of being significantly faster and more accurate than T·S Model, but high dimension data is still difficult to process. On the other hand, tree-structured piecewise linear adaptive filter (Gelfand, and Ravishankar, 1993) gives another type of partition of the input space by means of a tree structure, whose advantage consists in its insensitivity to the dimension of the input space. However, there will be spikes or broken-points in

3891

Copyright 1999 IFAC

ISBN: 0 08 043248 4

FUZZY-TREE MODEL AND ITS APPLICATIONS TO COMPLEX S...

intersections of linear filters associated with tenninal nodes, which results in the unsmoothness of the approximated function. The proposed Fuzzy-Tree Model not only overcomes the complexity of dividing input space in T-S model, but also realizes the smooth transition of piecewise function at the points of intersections~ thus a better approximation of complex system.

Section 2 gives an introduction to Fuzzy-Tree Model. In section 3, the training algorithm of building Fuzzy- Tree Model is discussed. Section 4 gives the simulation results, and makes a comparison with some famous modeling methods. A summary of the Fuzzy-Tree approach is given in section 5 ~

2. FUZZY-TREE MODEL

14th World Congress of IFAC

node t is not a parent node, then it is called terminal node, that is let) = r(t) =o. T denotes the set of T 'Is terminal nodes. A Fuzzy-Tree Model is associated with a binary tree T. For each node t ET, we define a tap weight vector C t == [Cl(O), .. ~,Ct(n)]T and a fuzzy set N t ~ whose membership function defined as J...I. t (x): x ---=). [0,1]. Thus, each node t ET is associated with a linear model (2)

The fuzzy set N t associated with node described a fuzzy subspace Xt • The input space is partitioned into fuzzy subspaces associated with terminal nodes~ Then we give the rules for the Fuzzy-Tree Model as following

This paper is mainly concerned with the modeling of the following complex system y = f(x)

(1)

XnY

n 1 where x = [1 xl E R + is input vector, y E R is the output. Since a multi-output problem can be transformed into several single-output ones, this paper will only focus on single-output case.

Because binary tree plays an important role in FuzzyTree Model, some basic definitions and terminologies about it are discussed as following. A binary tree is defined as a finite nonempty set T of positive integers and two functions 1(.) and r(.) : T ~ TU {O}, properti es:

which

satisfy

or

foJ1owing

l(t»t,

two

r(t»t

(1)

'fteT, hold;

(2)

"\It ET, except the smallest integer in T, there must exist only one s, which satisfies t = I(s)

or t

l(t)=r(t)=O

the

If x is Nt(tET),then

Yt =C'[X.

The output of Fuzzy-Tree Model YT is defined as the weighted average of the linear model output of all terminal nodes (3)

The family of Fuzzy-Tree Model associated with binary tree have the same structure. A Fuzzy-Tree Model, which include a binary tree T with five nodes and three terminal nodes, tap weight vector c t == [et (O)!l ~ .. ,c t (n)]T and a fuzzy set N, for each node t ET, is illustrated in Fig. 1. Here, the input vector x:::: [x 1 X 2 ]. Fig. (b) describes the process of dividing the input space into three fuzzy subspaces, with the oblique Hne representing the fuzzy belt between two fuzzy subspaces. Xt' t ET, represent the subspace corresponding to N t •

= res) .

For each node t ET, let) and r(t) represent respectively the left and right nodes which originate from t. The minimum element is named root node, denoted as root(T). If s, t ET, and t = l(s) or t = res) ,then s is called parent node of t, t child

=

node of s ~ denoted as s == p(t). If s pet) s == p(p(t») or ... , then s is called ancestor of and t is descendent of s. The depth of node refers to the number of t 's ancestors. The depth tree T is the maximum depth of the nodes of T.

l1'

m.

ex

7

cs

x

. Y,= .03X

or

1

+.21Xz -.36

p.,(xll=L

Y,=.Slxl-.33x!+.02

x

t,

(a)

t

of ]f

3892

Copyright 1999 IFAC

ISBN: 0 08 043248 4

FUZZY-TREE MODEL AND ITS APPLICATIONS TO COMPLEX S...

14th World Congress of IFAC

The value of the tap weight vector c t = [c t (0),··· ,c t (n)]T can be obtained by minimizing (4) with respect to c t ' t LS solution is obtained by

eT.

Thus) an

(5) ~

where

$

~'1;J x.

(OIJ)

x == (Xl... XL Y=(yl

(1.1)

~

It

~00

$

~*

(0,0)

---T X

::::

yL]T,

...

[llk(X T ) 4J.lr(X

Xl

r, X

T

~k"'P(XT)

iT

.4J..lrCX

)

tET

(b)

T

X

i'r]T

)

t£T

C~+p]T,and k, ... k+peT.

Cy=[C[

Fig. 1 (a) Fuzzy-Tree Model (b) Partition of input

space corresponding to (a) Compared with Tree-Structured Piecewise Linear Adaptive Filter, Fuzzy-Tree Model is smooth in the

intersection of piecewise function; Compared with that of T -S model, which is described in Fig. 2) the partition of input space of Fuzzy-Tree Model is different. Because T-S model divides input space for every input variable, the computation will increase with the increase of the input dimension. In the case of Fuzzy-Tree Model, however, this case will not happen, for it is insensitive to input dimension.

=:

1."

f U U ff"~ IIf If f 11

11

.= Xs

IIIIIIIIIE 11111111

Z:J

=:

(6)

Then the membership of the fuzzy sets of other nodes (except the root node) is defined as the following

deduced formula

weight vector.

~

(0,0)

In this paper, the instrumental membership IlI(t)(X) and

Fig. 2 Partition of input space ofT-S model

J!r(l)

(x) of child node l(t) and r(t) is defined by

_ IJ-l(t)

(

x)

3. THE TRAINING ALGORITHM OF FUZZYTREE MODEL ~

J.1 r(t) (x)

Building a Fuzzy-Tree Model can be described as following: Given training data (Xi ~ yi), Xi t; R n+l ,

i = 1)2, ... ,L, generate a binary tree T associated with tap weight vector and fuzzy sets, whose output -y r can minimize yi ER,

e

L

1

(1)

fl t (x) = Jl p{t) (x)J1 t (x)

The shape of the instrumental membership function ,J t (x) is of many choices. Different shapes of the instrumental function will influence that of the membership function, thus the fmal result of the tap

(1,1)

%1

Xl

For the convenience of calculating III (X), an instrumental membership function ~t (x) is defined for each node t ET, except root node. The fuzzy set of the root node is defined as

.

.

=L-[Y~ ~y~]

2

et

=

1

(8)

T

1 + exp[cx](l) (et x - 8](t»)]

I

(9)

T

1 + exp[ -ar(t) (et x - 8 r (1»)]

has multiple selection. To make the tree balance,

here we se] ect L

eI(t) == ar(t) ==

(4)

..

~Jlt(XI)y~

1=:1 ~L---.-

(10)

~}.ll(xl )

i=12

The binary tree grows from its root node. Therefore~ only parameters of the terminal nodes need to be

=

i=1

The influence of different selection of

(X.'(t)

and

calculated each time.

3893

Copyright 1999 IFAC

ISBN: 0 08 043248 4

FUZZY-TREE MODEL AND ITS APPLICATIONS TO COMPLEX S...

14th World Congress ofIFAC

For every node t ET, node error is defined as

aT(t) can be made out from fig. 3.

(16)

y-----------------. Partition node error is defined as

52

e~

L

.,

= L[j.1t (Xl )(yl

ii l(.) (Xi )Y:(~l + iir(.) (Xi )Y~(')12

i:::::1

where c jUP

x

(0,0)

Fig. 3 The influence of different selection of

iiJ(t)

cr(t)

(x~ ) +

(17)

fl r(t) (x' )

can be obtained by minimizing

ul(l)

If

and a ..(t) on approximation

(18)

In fig. 3, 12 and 13 are used to approximate the given curve. Curve sI and s2 describe the different case corresponding with a)(t} and ur(t) of different values. Hence, it is necessary to obtain optimal values of cx1(t) and ur(t) by training Fuzzy-Tree

Model. Next, we will deduce a gradient algorithm to training at ~ t E T ~ Firstly, we derme

e

i

= 2"1 [Yi

... i ]2 -Yr

(11)

Then, the derivative of e i with respect to a. ean be expressed

ae = 8e j

oCt s

i

ay~

ay~ BlJ. s (Xi)

oJ.!s(x

l

8~s(x;)

)

OJJs (Xi)

9 ,

SE

l-

The Training algorithm is described as following (1) Initialize root node of Fuzzy-Tree - calculate weight Cl' where }ll (x) = 1. Select initialized

value for at' t eT, not including root node. (2) Calculate et and e~ of each terminal node using (16) and (17) respectively. Use (18) to find out the node that can be divided. If it exists, continue; otherwise, end the algorithm. (3) Divide every clivi sion node. Calculate et ~ t E T of the newly formed using (10)~ then calculate Cl'

t

E

T using (5).

(12) (4) Train a p t (14), (15).

OOs

s ;: l(p(s))

(13)

for s == r(p(s))

(14)

Thus, n~p SET can be optimized using gradient algorithm as following (X.(i+l)=a.(i)-y : ;

holds, node t would be divided.

E

T of all

terminal nodes using (13),

(5) Calculate membership function of terminal node using (7). Update weights Ct~ t E T using (5). Calculate the norm of error between real output and Fuzzy-Tree Model output. If e is decreased, continue; otherwise, end the algorithm..

4. SIMULATION RESULTS

Example I-Function Approximation: For illustration, we consider the simple problem of modeling the nonlinear function

(15)

y(k + 1)

s

= O.3y(k) + O.6y(k -

1) + g[u(k)]

(19)

where y is a learning rate and satisfies that y ER,y >0.

where the unknown function g(u) = Q.6 sin(1tu)+ 0.3 sin(3xu) + 0.1 sin(51tU), and input u(k) = sin(2'Jtk /250). Let

Now tree-growing problem will be discussed.

x

= [1

y(k)

y(k -1)

u(k)]T,

Y = y(k + 1).

The

3894

Copyright 1999 IFAC

ISBN: 0 08 043248 4

FUZZY-TREE MODEL AND ITS APPLICATIONS TO COMPLEX S...

first 250 data are used as training data and ex. t , t E T are initialized as 0.0 I . The Fuzzy- Tree Model generates 3 nodes. Fig. 4 shows the training data and Fuzzy-Tree Model output. Figure 5 shows the output error between training data and Fuzzy-Tree Model output.

6

r----__---.-----r----.. . . . . -~........,

14th World Congress of IFAC

polynomial-fitting methods reported by Crowder (1990). For the Mackey-Glass time series problem, we use the same data set as that used in (Jang, 1993), which consisted of 1000 data points extracted from t = 118 to t = 1117. Fig. 6 is parts of Mackey-G lass chaotic time series. What is different from other methods is that we use first 100 data points for training the model. We initialize O:p t ET with 0.05. FuzzyTree Model generates 3 nodes. Fig. 7 is the comparison of checking data with Fuzzy-Tree Model output. Fig~ 8 is the error between checking data and Fuzzy-Tree Model output

_6'---------"'O'---'---'------"------....L.....------' 400 o 200 600 800 1000

Fig~

4 Training data and Fuzzy-Tree Model Output

::

0.2,...----,----............------.----......,....----.........

O_le)

~-~:::-[::---- --,-- ----- -:-------- 1---- -- --, -- -- --200

400

600

800

1000

1200

Fig. 6 Parts of Mackey-Glass chaotic time series.

-0.1 ' -_ _ o 200

~

400

............_ _......o.-_ _- - ' 600 aDo 1000

Fig. 5 Predicted error of Fuzzy- Tree Model

Example 2-Chaotic Time Series Prediction: In this example:- Fuzzy-Tree Model will be employed to predict future values of Mackey-Glass chaotic time series, which is a benchmark problem in system modeling. It is generated by chaotic Mackey-Glass differential delay equation (Mackey, and Glass, 1977) defined below: _dC_x_(t_)) = O.2x(t - 't) _ O~lx(t) dt I + X 10 (t - 't)

Fig. 7 Comparison of checking data with Fuzzy-Tree Model output.

(20)

• - - ... r ........ _ ...

- - - _.

- -

•

+- -

-

-

.. -

~

~

-

-- -

-

-

--

I

When 1" > 17, (20) shows chaotic character. The lager is 't, the more chaotic is (20). To allow Comparison with the published results of other methods, we use x

T=17, n

= 10,

y

==[1

x(t-I)

x(t-2)

...

x(t-n)]T,

= x(t) . 400

This problem has been considered by a lot of connectionist researchers. The modeling performance of the proposed method will be compared with that of the ANFIS as well as other neural network based and

600

800

1000

1200

Fig. 8 Error bet:v.;een checking data and Fuzzy-Tree Model output.

3895

Copyright 1999 IFAC

ISBN: 0 08 043248 4

14th World Congress of IFAC

FUZZY-TREE MODEL AND ITS APPLICATIONS TO COMPLEX S...

Table lists other methods' generalization capabilities which are measured by using each method to predict 500 points immediately following the training set, where Fuzzy-Tree Model uses the last 500 points. Here the nondimensional error index (NDEI) is defined as the root mean square error divided by the standard deviation of the target series. It is clear from Table I that Fuzzy-Tree Model is more accurate than other methods. Next to Fuzzy-

Tree Model, ANFIS model referenced here is more accurate than others. We compare Fuzzy-Tree Model with ANFIS in Table If. Here the root mean square error (RMSE) is used.

tremendously long, while it does not matter at all for Fuzzy-Tree ModeL

s~

This paper has presented a Fuzzy-Tree Model for complex system modeling~ Through the simulation, we can see that Fuzzy-Tree Model is very efficient for nonlinear system, time series modeling, etc. Compared with some other models, such ANFIS, Fuzzy-Tree Model is of less computation, higher accuracy, and insensitivity to the dimension of the input space.

Table J Generalization Result Comparisons

Method

Training Cases

NDEI

Fuzzy-Tree Model

100

0.0006

ANFIS

500

0.007

Cluster Estimation-Based

500

0.014

Auto-Regressive Model

500

0.19

Cascaded-Correlation NN

500

0.06

Back-Prop NN

500

0.02

Sixth-order Polynomial

500

0.04

Linear Predictive Method

2000

0.55

Rows 2 and 4 are from (Jang, 1993); row 3 is from (Chiu:t 1994); the last 4 rows are from (Crowder, 1990). Table 2 ComparisoD of Fuz?'y-Tree MQdel with

AmlS ANFIS

Fuzzy- Tree Model

Training Cases

500

100

Input dimension

4

]0

RMSE tm

0.0020

O~OOO15

RMSE chk

0.0019

0.000165

Rules

16

2

CONCLUSION

REFERENCES Chiu, S. (1994). Fuzzy model identification based on cluster estimationa Journal of Intelligent and Fuzzy Systems, 2, 267-278 Crowder, R.S~ (1990). Predicting the Mackey-Glass time series with cascade-correlation learning. In Proc. J 990 Connectionist Models Summer School Carnegie Me/Ion University, 117-123. Gelfand, S.B. and C.S. Ravishankar (1993). A treestructured piecewise linear adaptive filter. IEEE Transaction on Information Theory, 39, 19071922. Hunt, K.J.~ D. Sbarbaro, R. Zbikowski and P.l. Gawthrop (1992). NeruaI networks for control systems-a survey. AUfomatica, 28, 1083-1112. Jang, J.R. (1993). ANFIS: Adaptive-Network-Based fuzzy inference system. IEEE Transaction On Systems Man, and Cybernetics. 23, 665..685. Mackey M~ and L. Glass (1977), Oscillation and chaos in physiological control systems, Scjence~ 197,287..289. Narendra~ K.S. and K. Parthasarathy (1990). Identification and control of dynamical systems using neural networks. IEEE Transaction Oft Neural Networks, 1, 4-27. Sjoberg, J., Q. Zhang, L. Ljung, A. Benveniste, B. Delyon~ P.Y. Glorennec, H. Hjalmarsson, A. Juditsky (1995). Nonlinear black-box modeling in system identification: a unified overview. Automatica, 31, 1691-1724. Takagi, T. and M. Sugeno (1985). Fuzzy identi fication of systems and applications to

modelling and control. IEEE Transaction On Systems Man, and Cybernetics) 15, 116-132.

Besides all the items in Table 2, the ANFIS algorithm (coded in C) required 1.5 hours on an Apollo 700 series workstation, while The Fuzzy-Tree algorithm (coded in Matlab M script file) can identify the model within 2 minutes on a pentium 586/166. Thus Fuzzy-Tree Model is insensitive to input space dimensiona When input space dimension is higher than 7, the training time of ANFIS model is

3896

Copyright 1999 IFAC

ISBN: 0 08 043248 4