A piecewise bivariate interpolation approach to GMDH

14th World Congress ofIFAC

A PIECEWISE BIVARIATE INTERPOLATION APPROACH TO GM...

H-3a-13-4

Copyright © 1999 IFAC 14th Triennial World Congress, Beijing, P.R. China

A PIECEWISE DIVARIATE INTERPOLATION APPROACH TO GMDH

Wei

Liu

Department ofAutomaton Engineering, Hebei Institute ofTeclmology. Tangshan City, Hebei 063009 PR. China. E-mail: hjtswliu@public,tsptt.he.en 1

Abstract: To improve the weakness of basic GMDH (Group Method of Data Handling) in the choice of partial models~ we present a piecewise bivariate interpolation polynomialbased GMDH. The original motivation behind the development is to replace the fixed partial model with 6 parameters in the convention GMDH algorithm so that it can be widely applied. The necessary and sufficient condition of the unique existence of the partial model is proven. To illustrate the efficiency of the approach, two numerical examples are given. Copyright (91999 IFAC Keywords: Interpolation approximation, Modeling) Data handling, Piecewise linear, Parameter estimation, Complex systems, GMDH

1. INTRODUCTION

using

equations matrix X T X. They discussed the estimation of the coefficients of polynomials in GMDH by using improved instrumental variable method. Kondo et al. (1993) described a multinput multioutput type GMDH algorithm using regression principal component analysis. The optimal partial polynomials are constructed by combining characteristic variables. Nishikawa and Shimizu ( 1993) discussed the characteristics of a biased estimator applied to the adaptive GMDH. A large number of application have been shown such as the model of British Economy, Long-term Prediction of Demand for Consumer Goods, Modeling and Control of Hot Strip Mill and other industrial processes:, and Mode]jng and Control of Hydroelectric Power Station Water Reservoir. Hihi and Richard (] 993) studied the problems in the use of GMDH. They pointed out that the GMDH seems to be particularly weak in the choice of partial models that the simple polynomial models are not suitable for many applications.

Orthogonal polynomial and Splinear for the partial models of GMDH. Ivaklmenk and Zholnarskvy (1992) demonstrate that the error in estimating the coefficients of a linear regression depends on the magnitude of the conditionaHy index of norma)

This paper presents using a piecewise bivariate interpolation polynomial for partial models of GMDH (PBIP-GMDH), and gives a necessary and sufficiency condition that the partial model at any

The Group Method of Data Handling (GMDH) was advanced by Ivakhnenk and his coworkers (1970), which is a kind method for modeling complex system based on the principle of self-organization in biology control theory. If consider a nonlinear system Y = !(X1 ,·· ·~xn) where Endenote the input variables and y the output variable, the basic GMDH is to fit the function y;;;: [(Xl'· .. , X n ) by a recursive quadratic polynomial in two variables. Each layer of GMDH consists of a bank of Quadratic Polynomial Functions with inputs from the previous layer having been passed through a Selection Layer. Farlow (1984) systematically discussed the GMDH algorithms. Since the method was presented, it has been improved and developed from many approaches. Duffy

and

Franklin

(1975)

suggested

4213

Copyright 1999 IFAC

ISBN: 0 08 043248 4

A PIECEWISE BIVARIATE INTERPOLATION APPROACH TO GM...

14th World Congress of IFAC

layer is unique existence. In the application, the existence is used to select and treat the obtained data.

PI (x~ .. x/ )

P1 (x~ , xl )

Pn(X;, x{)

A = PI (x~ , xl)

pz (x~ , x~' )

p" (x; , xt) ~ 0

(1 )

2. ELECTING AND HANDLING THE OBTAINED DATA Meanwhile, if assume that there is a nonzero polynomial p(x i , x j ) E H TJ which makes En be on

Assuming:

p(x

1). System equation as:

= f(X),

y

X

E

R

P2 (Xi, xi), .. ',

15i,j~m,

,

x

j

)

= 0,

there

then,

must

be

p(x~,xt)=o (k=1,2'l···,n). As En is a well

m

posed point set, p(x i , xi)

2). There is a group ofbivariate polynomials with real coefficients and liner independence, which is ~ (Xi, xi),

i

=0

must be held, and it

conflicts with assumption which the p(x i , xi) is a nonzero polynomial. The necessary has been proven.

Pn (Xi, xl),

On the other hand, if En is not a posed point set of H n , i.e. A = 0, which illustrates that the polynomial

i*J·

vector group H" denotes a real linear space which bases are composed of these polynomials; 3). Input data point set of the system is

El

= {(x: ,xl), (x;~x~)~· .. , (x~,x~)},

1-::; i, j ~ rn,

i

*j

IS linear dependence. nonzero vector

Therefore~

there must be a [ c t ' c z , "., e" ] which

orthogonalizes with the polynomial vector group {Pl(x~,Xf),P2(xi'lx~),,·.,Pn(x~,xt)}, i.e. En falls on the algebraic curve of H n:

Output data points of system is Y k ~ 1::; k :::; m ; 4). S(xi;o x j polynomial, if

)

as

a

bivariate

interpolation and only if En and Nil are give,

and the interpolation satisfied.

condition: S(Xk, xl) = Yk is

Different from the single variable interpolation, the multiple interpolation polynomial is not always unique existence. In order to hold the uniqueness of the partial model, the following defme and theorem are given.

Defmition: Assuming that the fundamental functions PI (x' ,xJ), P2(X i ,xi),···.,Pn(.,\:i ,x J ), input point set ER' and output data Yk have been given, if a unique Sex' , x J ) E Rn exists and makes S(x~, xl) = Yk held, then E" is said a well posed point set of H

The sufficiency is proven. The signature of the theorem is to translate the uniqueness of bivariate interpoJation into a geometric problem. Coronary: rf assuming that

i). The composition of the point set En is that there are six points in any element of Ell' which three points are on a liner and other three points neither on this line nor any common line. ii). H n represents all sets which are formed from the square polynomial, that is, as fonn as:

PI'

Theorem: En is a well posed point set of H n, ifand only if En is not on any algebraic curve of H,._ Proof: It is obvious that if EH is a well posed point set, then:

Then) under the point set En ) the interpolation

polynomial of H

n

is existence and unique existence.

Example: when the interpolation is done in triangle domain T, if the set of point En consists of three angle points and three central points of each side. then the interpolation exists and uniquely exists.

4214

Copyright 1999 IFAC

ISBN: 0 08 043248 4

A PIECEWISE BIVARIATE INTERPOLATION APPROACH TO GM...

14th World Congress of IFAC

As the practice data are usually disordered, the dissection and optimum in any triangle domain must be done before using the method, so as to obtain a well posed points set En' A realizing method is given by Tian and Hu (1988).

4.1. Creating Input and Output Data

The input data are obtained from a uniform. random generator. The output data are calculated based on (5). We get totally 150 samples, and use 120 samples for identification and 30 for checking the models.

3. THE STRUCTURE OF BIVARIATE AND QUADRATIC INTERPOLATION POLYNOMIAL

4.2 Selecting Criterion

We use BIC Criterion defined as follow to select the intennediate variables.

We assume that the input data of system have been dissected in triangle domains as T == {to, I],· ~., 1,.-1}. In each triangle domain t f ( 0 S f ~ n - 1 ) , six points have been obtained which are three angle points and three central points of each side, as Fig.I. Then, a quadratic surface is constructed in the triangle domain) that is:

BIC(r)

s(r) per) N - r Nlog--+rlog---N-r S(r) r

=

(6)

where r is the number of parameters in the model N is the number of data point, per) is the swn-ofsquares of the model fit and S(r) is the sum-ofsq uares of the residuals.

on the triangle and in ) == (J f (Xi' x) the inner points. Then, a bivariate quadratic interpolation polynomial is constructed by piecewise surfaceS(x;, x j ) on T. If let P(x;, x j ) = S(xi , x j ) in the whole domain, the partial models will be obtained. Let S(x s " x j

4.3 Estimating Results

The followings show the result using convention GMDH algorithm.

The fIfst layer:

f. = 2 .. 42+1.26X J /2 = 2.17 +0. 96x: + O. 'Ix:

(7)

f3 ;::: 1.83+ 2.18x; + 3. 27x.x 3

/4

__- - - - -... tfS.------~_'tt-4

= 3.14+ 1. 25x j + O.37x4+ 1.43x]x 4

The second layer:

Fig. 1 An example ofdata dissection

4. NUMERICAL EXAMPLES In this section two numerical examples are given to illustrate the efficiency of the approach. One is the identifying result using basic GMDH algorithm and

= 13.8 + 8.53x l + O.75x 4

+3.7x

+3.12x; +37.1x;

g3

= 1.. 69 + 1.17f .. + O. 42f/

3.21 + 2. 23f2 + 3. 43flf3

l'-t == 3.1S+3.13g, +O.23gl g 3 hz

(8)

(9)

= 2.. 84 + 4.. 56gz

y = 1.57 + 1.16h. + 1.42h2

+ 12.5x: + 4.2x~ + 2.4x)x 3 + 4.8x 1X 4 2 I X4

=

The fourth layer:

+ 8.58xt + 6x;

+ 35.8x t x: + O~27xlX~ + 16.6xt x 3

= 2.46 + 3.41);. + 2 .. 46f4

The third layer:

the other is the identifying result using the PBIP GMDH. Consider the following equation: y

gl

g2

(5)

(10)

The final model:

+o.8x1

+ 36.4xlX~ + 8.3xf x; + O. 7 x~ xi + 2. 8x i x;

4215

Copyright 1999 IFAC

ISBN: 0 08 043248 4

A PIECEWISE BIVARIATE INTERPOLATION APPROACH TO GM...

y

14th World Congress of IFAC

28t

14~ 17 + 9.03x] + O.67x4 + 8.21x1

1

;::=

+ 6.13x: + 13.24x; + 4.73x; + 3. 19x]x3 2

+ 4. 52xt x 4 + 34.63x1x; + 16.93x1 x 3

24

(11)

3 ~ 14x{x4 + +0. 14x.x; + 3 ~21x~

~:t

+ 34.62x; +O.78x: + 37.34xTxi 2 + 9.64x12 X; + O.84x1 X; + 3.03x;x~ +O.13x4

1

•

Estimated by basic GMDH Estimated by PBIP GMDH Observed values

x1 +O.03x x;x; +O.llxlx~x:

t

1

12T

The followings show the result using the PBIP GMDH algorithm

8

t

--+-f--------~t____+__-+---+---

o

4

2

-----+----+ 6

~

8

1·--+------,- I- +-

10

12

14

-

No.

The first layer: /1

Fig.2 Estimation curves by basic GMDH and by the PBIPGMDH

= 0.351 + 1.182x1 + 2. 463x; + 1.178xt x 3

f2 = 1.373+0.921xi + O.627x;

(12)

f3 =3.314+0.487x. +O.157x 4 +1.121x."\:4

2. 40!

The second layer:

g.

= 0.02+ O.162f. + 1.173// + 0.527/,,2

Estimation error by PBIP GMDH

1.60

o. Bot

(13)

g2 = O.315f.2+ 0 .. 679fJ +0.234/3'2

i\ I

O'OOf

The third layer:

y = 4.571g1 + 2.694g 2

\

\

(14)

-}. 60t

= 13.56+8.21xl + O.7Ix4 + 8. 826xl2

\

,

I

I

\

\

, "---

~

0

2

4

6

8

I

I

10

12

14 No.

Fig.3 The estimation error curves by basic GMDH and by the PBIP GMDH

+6.09x~ +12.41x; + 4.18x; + 2. 18xt x 3

+S.12x1x 4 + 36.14x1X; +O.22xlX~

I

-0.80

The final model: y

Estimation error by basic GMDH

(15)

+ 17.29x11 X3 + 3.45x12 X 4 + 3.01x~ + 37.67x; +O.94x: + 36.03x1xi + 8.62x11 X; + O.79x~x; + 2.17xix;

REFERENCE Duffy, J. J. and M. A. Franklin (1975). A learning Identification Algorithm and Its Application to an Environmental System. IEEE Trans. SMC-5. 226-240. Farlow, S. J. (1984). Self-Organizing Methods in Modeling --GMDH Type Algorithms, Marcel Dekker, New York. Hihi, J. and A. Richard (1993). Problems in the Use of GMDH, Improvment. Proceedings of lAfACSllFAC Second Symposium on: Mathematical and Intelligent Models in System Simul ation~ Brussels~ Belgium, 1, 115-119. Ivakhnenko, A.G. (1970). Heuristic Self-organization in problem of Engineering Cybemetics~ Automatic, 6,207-219.

The sample data are in table 1. Estimation and error curves are shown in Fig. 2 and Fig. 3.

5. CONCLUSION

In this paper~ we proposed a piecewise bivariate interpolation polynomial approach to GMDH. The original motivation behind the development was to replace the fixed partial model with 6 parameters in the convention GMDH algorithm so that it can be widely applied.

4216

Copyright 1999 IFAC

ISBN: 0 08 043248 4

14th World Congress ofIFAC

A PIECEWISE BIVARIATE INTERPOLATION APPROACH TO GM...

Nishikawa, T and S. Shimzu (1993). The Characteristics of A Biased Estimator Applied to the Adative GMDH. Math. Comput. Model. 17, 37-48. Tian , C. S. and J. W. Hu (1988). A Algorithm of Making Triangle Dissection on Plane Surface. Computer and Numerical Calculation {in Chinese). 9, 144-152.

Ivakhnenko, A.G. and A. A. Zholnarsking (1992). Estimating the Coefficients of Polynomials in Parametric GMDH Algorithms by the Improved Instrumental Variables Method. J. Autom. Inf Set.. 25. Kondo, T. (1993). Mutinput - Mutoutput Type GMDH Algorithm Using Regression Principal Component Analysis. Inst. Syst. Control Inf. Eng. (in Japan). 6~ 520-529.

Table 1 Sample data for checking of the GMDH

1

0,34

0.064

2 3 4 5 6

-3.72 0.160 -0.140 -0.940 ..0.330

-0.016 -0.028 -0.333 0.228

7 8 9

-2.35 -2.230 -0.200

10

-0.69

11

-1.000

-O~065

12

O~47

-0.371

13 14

-0.880 -0.060 0.150

-0.165

15

-0.413 -0.435 -0.448 0.412

0.004

-0.208 -0.096

0.171 -0.445 -0.363 0.490 0.092 -0.113 -0.41 -0.438 0.074

Ow209 -Ow099 0.174 -0.100 0.419 -0.198

0.202 -0.138

-0.017 0.362 -0.05] 0.446 0.176 0.463 0.148 0.263 -0.272 -0.397 0.138 0.409 0.353

ez

Yo

Yl

Y2

19.775 10.328 17.865 17.505 14.722 13.730 11.709

19.462

20.31

0.3]3

11.171

8.887

17.595 17.308 15.]03 13.519

18.286

-0.844 0.269 0.197 -0.381 0.211 0.189 0.777 0.171 -0.094 -0.407 0.347 -0.084 0.242 0.283

11.520

13.735

12~958

13.681 12.880 12.805 21.760 11.943 17.292 17.000

13.511 12.974 13.212 21.414 12.027 17.050 16.717

17.886 14.303 14.073 10.434 12.596 13.964 12.746 12.551 22.600 11.642 17.746 17.445

-0.534 1.44 -0.421 -0.382 0.418 -0.344

1.280 1.138 -0.282 0.134 0.254 -0.840 0.300 -0.454 -0.445

Notes: YO: Observed values; YI : Estimated by PBIP GMDH; Y2: Estimated by basic GMDH, et = YO-YI; e2=YO-Y2

4217

Copyright 1999 IFAC

ISBN: 0 08 043248 4