Small Dynamic Universe Algorithm of Fuzzy Neural Network Identifier

ELSEVIER

Copyright © IF AC New Technologies for Automation of Metallurgical Industry, Shanghai, P.R . China, 2003

IFAC PUBUCATIONS www.e1sevier.comllocatelifac

SMALL DYNAMIC UNIVERSE ALGORITHM OF FUZZY NEURAL NETWORK IDENTIFIER Xu Da-Pin(

Yang Xi-Yun a

Dong Pingb

Zbang Bine

• Department of automation, North China Electric Power University , Beijing 102206,People Republic of China bBeijing Hollysys Ltd Co, Beijing. 100096, People Republic ofChina t:Beijing Huaneng automation incorporation, Beijing, 100020, People Republic of China

Abstract: This paper represents a small dynamic universe algorithm for achieving both high precision and real-time performance of nonlinear identification by fuzzy neural network. Two influence factors of identification precision, membership function type and numbers of fuzzy subset in the input space, are analysed with given network structure. Then a small dynamic universe algorithm is derived. Different with common fuzzification procedure, algorithm employs methods of shifting a small operating window in the input workspace to generate the initial values of premise fuzzification parameters and reducing the numbers of fuzzy subsets in the windows. So a contradiction between the high precision and real-time performance is avoided. Simulation results of nonlinear identification demonstrate effectiveness of proposed algorithm. Copyright © 2003 IFAC Keywords: fuzzy neural network; identification; nonlinear; real-time; small dynamic universe.

After analysing effects of membership function type and numbers of fuzzy subset in the input space, small dynamic universe algorithm is created, which shifts small universe operation window during training of fuzzy neural network. Only two fuzzy subsets of every input are divided in operating window so that real-time performance of algorithm can be guaranteed. And due to input workspace relatively reduced, high precision is also obtained from fewer fuzzy rules. The algorithm gives a bright promise for on-line fuzzy neural network identifier to complex metallurgical process.

1. INTRODUCTION

Fuzzy neural network with ability of approximating a nonlinear function in any precision (Wang, 1992), is a useful tool replacing traditional modelling theory for complex metallurgical industrial process with uncertain and nonlinear characteristics (Zou and Xia, 1997). However how to ensure both real-time performance and high precIsIon for on-line metallurgical process identification must be considered. In general, fuzzy subsets increasing in the input workspace achieve both high precision and expansion of fuzzy rules that slows the algorithm speeds with more learning parameters. Approaches to solve above problem have been discussed, such as the rule combination (Lin, 1995). Whether or not high precisions really depend on a large numbers of learning parameters accompanied abundance fuzzy rules? The small dynamic universe algorithm proposed in the paper solves a contradiction between the high precision and fuzzy rules expansion.

The remainder of this paper is organized as follows. Takagi-Sugeno fuzzy model (Takagi and Sugeno, 1985) is chose to construct a fuzzy neural structure in section 2. Section 3 introduces a hybrid method of separating premise and consequence parameters for network training (Jang, 1993). In section 4, firstly two influence factors to identification precision, the membership function type and numbers of fuzzy subsets divided in the input space, are studied in

287

section 4.1. Then an algorithm of the small dynamic universe is deduced in section 4.2. In Section 5, simulation results of nonlinear function identification prove that the fuzzy neural network identifier has high precision and good real-time performance. Finally some conclusions are presented in section 6.

by backpropagation method, and linear consequence parameters ( P;I ) are determined by least squares method.

Taken membership of Gaussian function as example, premise parameters cij and O'ij is adjust as: Cost function is: E = ~(Yd 2

2 .FUZZY NEURAL NETWORK BASED ON TAKAGI-SUGENO FUZZY MODEL

y)2 .

aE

cij(k + 1) = cij(k) - p~ Ij

A fuzzy neural network identifier based on TakagiSugeno fuzzy model is shown in figure 1. Given a Takagi-Sugeno fuzzy model with n input, one output and m rules, the jth rule is defined as: R) : if(x1 is An and . .. and (x" is

O'ij(k + 1) = O'ij(k) -

,=)

A/

~ and

An Then

ocij

Written" =lP\o Pt 1 .•• PIn .. . pw{) P,m .. . Pm,,] And P)i is updated using least squares solution: B=(XTXrIXTy,

Where X =[lij alxl ... alx" . .. am amXl ... amx,,]

The output of the fuzzy system with m rules is aggregated as:

= fa)y)

)=1

are not listed in detail since

is a fuzzy subset

the fuzzy subset (i=I,2, ... ,n; j=I,2, .. . mi) .

)=1

aE

aO'ij

backpropagation algorithm is widely used, found in (Jang, 1993) if needed.

defined on the universe of input Xi accordingly PA/(Xi) is degree of membership mapped

Y = Ia)Y)/Ia)

P aO'

Ij

y) = PlO + P)lxl + ... + p)"x"

j=I,2 ... , m (m ~nm), where

aE

The strategy not only escapes to local minima of backpropagation algorithm to achieve high precision, but also cut down the convergence time efficiently.

)=1

Where a) =a;(fa) )=1

a; = PAJ (XI)PAJ (X2) •.. P AJ (x,,) I

a)

2

4 . ALGORITHM OF SMALL DYNAMIC UNIVERSE

•

is the degree of fulfilment of rule jth.

4.1 Two influence factors of identification precision: membership function type and numbers offuzzy subset in the input space.

Vl 1

:==~~~~~~~~~ __ r-:__ c:~~:::::~~~~-t1

Used different type of membership function in above network, simulation result with different precision is shown in Fig.2. Input workspace [0 10] is divided averagely 5 fuzzy subsets. Training data pairs are sampled for every 0.1. Nonlinear function identified is y = sin(2t)exp(-O.2t). From observation, it can be conclusion that using a narrow smooth shape membership function, such as Gaussian function, can improve identification precision.

y - Consequence ~.a... ___network Premise network

Xr!.-- ....A. Fig. 1. Fuzzy neural network structure ofT-S fuzzy model

1.-------~-----__,

3.HYBRID LEARNING METHOD OF SEPARATING PREMISE AND CONSEQUENCE PARAMETERS

0 .5

o -0 .5

A hybrid algorithm combining the backpropagation method with least squared method (Jang, 1993) is employed for training the network, where nonlinear premise parameters implied with PAl (xi) are updated

-1 ~-----------~ 10 5 o

a identification curve with gaussian membership function ( real model - identatification--)

288

Consequently an algorithm of small dynamic universe theory is developed. It implements fuzzification in the small dynamic universe instead of in the overall input spaces. Only by using two fuzzy subsets, it can produce the same or more numbers of fuzzy subsets in the unit region as that of common fuzzification. It is very efficient to reducing the gaps between the precision and real-time performance. The detailed step is followed as: b membership function curve (after traininginitiaI-) (I) Convert the fixed universe into the dynamic universe. For nonlinear function in Fig.3, the overall input space [0 30] is divided into many small universes and the distance of every small universe is described as a characteristic region [0 a]. That means if a small universe is region [7.4 7.8], the characteristic region is [00.4]. a represents width of operating small universe window, which is either constant or variable depending on the curvature of identification curve and identification precision achieved. Assumed sample point with equal spacing, a is set to small value under the steep curvature or high precision, such as assigned distance between sample point k and sample point k+ I . In another case, value of a will be large for a smooth curve, maybe assigned distance between sample point k+2 and sample point k+ 1O. It must be emphasized that the small universes is a sliding operating window, where every region a is dynamically freshened with sample point varying and covers different input space. So this method is an on-line algorithm in nature. Further the weights parameters of the current small universe do not depend on the parameters of precious small universe, whose initial parameters of the membership function (cij and (j ij ) are always generated by

0 -0.5 'r

-1

0

"

5

10

c identification curve with triangle membership function (real model- identification -)

d membership function curve (after traininginitiaI-) Fig.2 Simulation with different type of membership function

average partition method. Of course, the end-point of the precious small window is also the start-point of

Another factor to influence preCISion, numbers of fuzzy subsets in the input space, is also tested. As we known, the numbers of fuzzy subsets are often increased to satisfy precision in the case of the input space larger or steeper curve. Where the input workspace extends [0 30] and nonlinear function becomes y = sin(4t)exp(--{}.5t) , Fig.3 shows worse identification precision on 9 fuzzy subsets case. For achieving high precision, 20 subsets should be required. However number of fuzzy subsets in practice is commonly limited to 3 to 7 for store spaces of rules and algorithm real-time performance. So it is necessary to find a new idea instead of infinitely increasing fuzzy labels for higher identification precision. Small dynamic universe theory is developed to solve the question.

0.5 0 -0.5

-1

~ 30 0 10 20 a Simulation result(reaI model- identification-)

4.2 Algorithm ofsmall dynamic universe Analysing examples in section 4.1, it can conclude that a crucial reason that fuzzy subsets numbers affects precision under the same membership function is dominated by numbers of fuzzy subsets contained in every unit region of input space.

b membership

function curve (after training - initial-)

Fig.3 Simulation curve with 9 fuzzy subsets

289

current window for keeping universe consistence so that the operating small universe window covers everywhere while sample point moves in overall the workspace [0 30).

so that such subtle turning guarantees the high identification precision. Table 1 shows the weights parameters of network on the operating universe [0.1 0.2]. It is very difficult to imagine that above membership function optimization parameters is regulated only by expert experience.

(2) Reduce the numbers of fuzzy subsets to 2 and the type of membership function is Gaussian function. Since the range of small universe is very small, two fuzzy subsets are densely partitioned in this universe. It is essence that the innumerable fuzzy labels are divided in overall input workspace with sliding small universe window, which lead to higher precision than convention fuzzy set divided finite fuzzy labels. It is noted that each input only with two fuzzy subsets significantly speed up algorithm, especially for MIMO system. So the method provides a good strategy to reconcile the contradiction between the high precision and a good real-time performance.

0.5 y.:sin (4xt) exp (-0. Sxt)

o -0.5 -1~----~------~------~

o

10

30

20

a simulation result(real modelIdentification- universe ro 301 )

(3) Using a recursive least-squares algorithm to replace least squares method. Because region [0 a] contain less train data pairs, a recursive least squares with no exponential forgettiilg algorithm can be applied. P(k)X(k + I)(yd(k + I) - X(k + Illj;(k»

0.5

P;(k + I) = P;(k) +----...:.....-:;:-----"'-.I J I+X(k+llp(k)X(k+l)

0.12

0.14

0.16

0.18

0.2

b on the universe [0.1 0.2J membership function curve (after training-- initial-)

P(k + I) = P(k) _ P(k)X(k + I)X(k + I l P(k) 1+ X(k + I l P(k)X(k + I)

The following steps summarize the procedure: (1) Determine an operating small universe window [0 a] for current sample point. (2) Choose · the- train data pairs on current usmall universe window. (3) Divide the current small universe into two fuzzy subsets by method of average partition. (4) Keep training network parameters with separating premise and consequence parameters until the precision is achieved, where consequence parameters training uses recursive least-squares algorithm. (5) Get a next sample point. Judge whether all the input space has been covered. No, go to 1. Yes, end.

0.5

o~--~--~----~--~--~

27.9

27.92

27.94

27.96

27.98

28

c on the universe [27.9 28] membership function curve (after training - initial-) Fig.4. A group of curve of simulated fig.3 by small dynamic universe Tablel simulation parameters in universe [0.1 0.2] Initial value

5.SIMULAnON RESULT

Clj

Example 1:

(Jlj

Final value after learning Clj

(Jlj

0.1000 0.0425 0.0796 0.0215 0.2000 0.0425 0.1734 0.0827

Fig.4 shows the result simulated Fig.3 again with small dynamic universe and a high precision is observed, where a is assigned constant 0.1. Modified membership function shapes are significant different between Fig4.b and Fig4.c with small dynamic universe theory. Parameters of membership function (Cij and uij ) in every operating window can be

PjO

0.0875 0.1159

Pjl

0.0087 0.6259

Example2: Another identification of complex non linear function carried out is shown in Fig.5, where the non linear function is governed by the following equation: y(k + I ) = O.3y(k) + 0.6y(k -I) + g[u(k)]

adaptively updated with backpropagation algorithm

290

= 0.6sin(1/U) + 0.3sin(31/U) + 0.lsin(51/U)

g[u(k)]

16,-.----~----~----~

u(k) = sin(2nfc/250)

t/h

From observation, the identification curve maps the real curve very precisely and error is less than 1.5 xl 0.... A modification of membership function curve in [0.685 0.72] is also shown in Fig.5.c. This nonlinear function is also found in (Jang, 1993) that uses at least 7 fuzzy subsets with on-line training. Thus simulating with 2 fuzzy subsets in small dynamic universe significantly improves the realtime performance.

Bbt

12

8 4

o

50

25

75 t/h

Bj

10~----~------~----~

Fig.6. Bb( -Bj response curve (model output network output 0)

5

o

6. CONCLUSIONS

100

This paper provides an on-line algorithm named dynamic small universe. By using technology of shifting operating universe and reducing fuzzy subsets to 2, algorithm of small dynamic universe can achieve both high precision and good real performance for nonlinear identification. It will give a new approach for on-line modelling complex metallurgical process.

200

.a simulation result ( real model- identification-)

X 10-4 1.B'---~--~------~----~

1

o -1 -1. BL -_ _~_ _---,.!.o-_ ___.J

o

100

REFERENCES Jang, J.R. (1993). ANFIS: Adaptive-Networkbased Fuzzy Inference System. IEEE Trans on Syst. Man and Cerby. 23(2), 666-684. Lin, C.T. (1995). A neural fuzzy control system with structure and parameter learning. Fuzzy sets and system, 70,183-212 Takagi, I. and Sugeno, M. (1985). Fuzzy identification of systems and its application to modeling and control. IEEE Transactions on System Man and Cybernetics, 15,116-132. Wang, L.X. and Mendel, J.M. (1992). Fuzzy basic function, universal approximation and orthogol1al least squares learning. IEEE Transactions on Neural Networks, 3(5), 807-814. Wang, D.F. (2000). Study on multi variable intelligent control and its application to ball mill coal pulverized system. PHD, North China Electric Power University, Baoding. Zou, T. P. and Xia J.B (1997). Coal powder flow rate mode ling using fuzzy neural network. Information and control, 26(4), 306-310.

200

b error curve

1 ~,.........."""",,--.---........- - -

0.5

o ---

.. -

0.685 0.69

",

0.7

0.71

0.72

c on the universe [0.685 0.72] membership function curve (after training- initial-) Fig. 5. A complex nonlinear function simulation by small dynamic universe

Example3: Ball mill system of blast furnace is a multi variable system with nonlinear, large dead-time and time-vary characteristic. Storage load of ball mill ( Bbt ) is difficult to analyse, because it is affected by fed-coal (Bj ), heat air flow (G rk ), recycle air flow (Gzx) and temperature of mill (Im ). Nonlinear Function expression B bt = f(Bj,Grt,Gzx,l m ) and response curve to every variable are found in (Wang, 2000). Simulation with network identifying the function Bbt = f(Bj,Grt,Gzx,l m ) is carried out and one of the curves is shown in Fig.6.

291