Adaptive Structural Identification

ADAPTIVE STRUCTURAL IDENTIFICATION Dobrin Burev Institute of Engineering Cybernetics, Sofia 111 J, Bulgaria

ABSTRACT

In this paper the synthesis of mathematical models in terms of parametric, argument and structural indefiniteness of plants with known laws of noises' probability distribution is discuss3d. As an optimality criterion of the structures the proposed method for adaptive structural identification uses the verifiCation of the statistical hypothesis for the insignificance of the correlation ratio between the structures and the remainders, obtained after the parametric identification with the mean-root-square method. Increasing the complexity of the model structure does not lead to increasing the dimension of identified parameters. This method is applied for the syntheSiS of mathematical models in a class of transport processes. n~TRODUCTION

The syntheSiS of mathematical models of plants with incomplete information is one of the central problems in contemporary modeling theory, identification and estimation. According to the character of the incomplete information, these models Can be: a) with parametric indefiniteness of a vector with finite number of unknown parameters; b) with argument indefiniteness of probability distributions and statistical characteristics of the observable model variables; c) with noise indefiniteness of the values, probability distributions and statistical characteristics of the unobservable variables, and also of the joint with observables distributions and characteristics; d) with structural indefiniteness of the functional link between the variables (the functional form of the models being unknown). ;Iioreover, all kinds of indefiniteness mentioned may be either stationary, or instationary, either functions in(m+C·dimensional coordinate space (distributed indefiniteness), or not. The paper discusses the syntheSis of parametriC models in terms of parametric and structural indefiniteness of stationary plants, in Cases of: a) known probability distributions of observable variables(ar~ent definiteness); b) unknown, reconstructed by current observations probability distributions of observable variables (argument indefinite65

D. Bur ev

66

ness). It is assumed that the probability distributions of noises and their correlational properties, together with the observable variables, are known_ FORMULATION OF THE PROBLEM OF STRUCTURAL IDENTIFICATION

A class of stochastic multidimensional plants are treated: (1)

Y

=

F (X) + Cc) ~

where Y is random output observable variable (process) with density of probability distribution !,y(Y) , X is 11 -dimensional vector of input observable random variables (processes) with densities of probability distributions~(~, ~ is noise determined by the influence of the input unobservable variables with Markovian properties, normally distributed with zero mean and finite variance, ~ is a si~ple non-linear operator, determined in a class of operators QF- The real values of Y and X are connected with the observable values JJ and X by the correlations: (2)

II = 'Y + W'!f;

)( =

X +

"';:x:.

;I

where the noises of the observation CV,:!! and cLJx have the same properties aB these of C4.-' • The modeling of plant (1) with a parametric model

0)

I (X ,e)

11 =

+

e ~

I; F f ~ 52.Fj

)

J2F

' F

are unknown ,

is carried out under conditions in which the operator of model ~ is a simple parametric function of unknown parameters C ,defined in a given class of functions A Case will be treated when f!J{Y} and are known, and also a Case when P!/V) and Px(x.) are unknown and are being reconstructed on the basis of current, discrete and independent observations and X •

PJ%)

5'2/_

y

As is known [1] , a basic criterion for correctly chosen structure in parametric identification of model (3) in the classical formulation of this problem (for example, by means of the mean-root-square method) is the lack ofoorrelation between the remainder Eo and y./((,9. The result of the verification of the statistical hypothesis for the insignificance of the correlation ratio between e and )' may be used as a formal criterion for optimBlity of structural identification: Ry 2. IV 2 -1 (4)

~2

cE,Y

:

EP,[mECY'&.)- mE] • [~ (ea 311lf

i=t

- me)] :: s2. €.y

IS'Eo'

where: ~

(5)

c 3 = :13

(6)

LEe me = IV 8=1 a

- y~ -= 3,

fexJ,C). 3:

1,#;

IV ~

rne(~) is an estimation of the relative mathematical expectation r1{elYj,

Adapti ve struc tur a l ident if i ca t io n

67

which is defined as a mean value of E. t to which correspond values of ~ , referring to i-th order of the domain ofY,i.",1,,\; Ry is the number of ranks of Y ; IV is the volume of accesses from values of )( and}J , which .... are used for the estimations C by the mean-root-square method of parameters C of model (J). It will be noted, that ~2 fo[O.~ is a me a sure of the non-linear statistical link between f and Y ~'YFrom 't2,.,:0 it dOes not follow 2 E.l that ~£,y 0 , bu; the re;erse is true C'Z.6,y is the correlation coefficient of €. and Y ; ~E\V ~ ~4i.Y ) [2] • ~he virification of the statistical hypothesis for t h e insigni f icance of '2E,Y( ~~.Y <~(1'1) is carried out at a confidential level ~ = 5% with t h e h elp of the inequa lity [2] :

=

(7)

, ! 2..

en

f ...

"eE'Yl <

{+

'2e 1 v

If (7 ) i s n o1,; satisfied, then with an error of 5% the hypothesis for insignifiCance of '2f.v is rejected, and, conse quently, the structure is admitted to be incorrectly chosen. The utilization of (7) as an optimality criterion for structural identific a tion is c arried out considering the following t wo assumptions: a ) rel a tionship (7), generally valid for ~e,Y, is used to estimate ~E,Y , since ~ b) the problem of estimating "l: in mathematical statistics is developed only in the case of variables with normal distribution [3] . If t he ve rification of th e statistical lqpothco i o :;:"or n ormality of the empirical distributions of E and Y is with negative result, a normalization of and e is necessary via approximation of their distributions with distribution families of JOhnson [4], and the utilization of the corresponding reverse transformations.

I

-et,y 'C.:'v ;

Y

opt

The class of procedures for choosing an optimal structure)' of model (3), (7) being satisfied, can be written in the form of the following recursive functional equation: (B)

YK

= Y.tc-f (x)

J<.

= ~. 2) ... ~

I( x, Co) .

Vc, =

(9)

lJ; (~_, (xL )c lele ) ,

+

The particular choice of the functional operators~ determines a given iterative algorithm for a generation of a new model structure (for example,

no

(Ba, 9a)

no

k

y~ .. ~ C~ 'Pi. (x) + ~~f

Yk-i (

~ C~+i '"" (x) -+ 4) j ~=I

or, in the most simple case ~: ~ ; /(: 0,1, ••. ( J( is the number of iteratlo~s of structural identifiCation process model (3». 1~e optimal estimations Care found as a result of the solution of the following problem: I(

(10)

SSMIlS:

f

N[

A

A.

~

NE YIc,(xq,CO,C' .... Qk)3='

d

2

-:13 1 --

l;sl

m"i.n C"

;

1<=

fJ,J••..•

We shall note, that with ,to each k-th iteration of the structural identification procOBB (K ... f/~ ••• ) , the dimension of the vector of identifiable parameters C is constant (n 1(r:1,2, ••• ) and is equal to the num-

l(-1:"

c

n;

D. Burev

68

f

ber of the parameters of the initial function with (n+1) variables (~+,~ Y,,_,) (for (8a,9a) ne#(= 2IJo.l<=f,<,.~. Consequently, the increased complexity of model structure (3), in this case does not lead to an increase of the number of estimated parameters, which is of particular importance for the practical realization of the method. At the k-thA iteration, the parametric values Ci ( i = 0,11.- f) are constant and equal to Cl.. CI:: 0,1<-1), determined correspondingly at 0,1, ••• ,k-1 -th iterations of the process (8),(9) in the solution of (10). Process (8) can be treated as a process of minimization of the function: (11)

~Z

G E,Y

I Y [ d' c ex, d)JJ -

with constraints

)C,

~~~

.mi.n.. ~ Y(x,JI)

and criterion for stopping

STRUCTURAL IDENTIFICATION ALGORITHMS

First, the procedure shall be treated of the structural identification of model 0) of plant (1), with the help of the parametrio functions ~(c:.)E52j'"(e) ( ~e)iS a given set of functions ~(c) with known distributions ~(y) and &:(x) with the help of finite access from values of x and ':J with volume N. 1. Assign 1(=0 • Select Y.=Cf;(x,c). 2. With the help of type-procedure for solution of problem (10) (for example, optimization algorithm for unconditional minimization [5] we estimate

" C~.

Z

3. The values of ~"X",=f,#, ~oIC. and Sc!'oIC. are oomputed by (5) and (6). 4. Estimate ~..;" and X,,,,AX in the access i yAC }, 3=I,IV', and RYic. by the 3 proportion: (12 )

and YKi min and YJ(imAX (i: ~ Rv) (YI(. • and'r,. are the bounds of the i-th '1(. " ' " '" " 'mAx rank of values of VII. ,i= 1, Ry.IC. ). 5. Assign tj:::-O; Pi: 0) i= 1~Ryl(.; "'Jc(. (3):= (J ,i: f/RYJ<· 6. Assign 9: 8+ 1. 1. Assign i: = 0 . 8. Assign £: i + -f . 9. Check the relationship:

=

=

=

(13)

If (13) is satisfied, assign P:=p;'+1 and estimate &

(14)

Mlti (3) = ~.

[(Pi - 1)

ml
(3- f) + ~, ] ~

I.

(ml(t" is an estimation

Ofl1?!("ri)

and go to 10. If (13) is not satisfied, assign MJ(;C3)zI1Jti (a-') and go to 8. C 10. It ,
69

Adaptive stru c tur a l id en t i fi cat ion

of convergence of process (8) «")-(20». Assign Xc,::X_/~(1..t\)C.e,,) and go to 2. 13. End. Now, we shall treat the procedure for adaptive structural identification of model (3) of plant (1) with the help of structures f.:E ~ at discrete input

d

values of Xi and ~3 t = f" ... ~ ••.• 1. Assign 'iJ:: O. 2. Assign = i+ f. C;-( · I<. = 0 an d se 1 ec t Jr'Jo x, Co) ... ~o . A 3 • Ass~gn 4. For estimation of values Co the algorithm of stochastic approximation is used:

a:

(15) C IeC3]= CIcC3-1]

-J! C3J[ ~[3]- ¥c.,ca J - t(~-t[' JI ><[,], c. J(f:-,))]~lCc7i [3].,

2 where ~['}iS determined by the condition [!I[,]-"(c"r'])]-~1,}bY the algorithm for one-dimensional optimization [5J. If~<4 go to 2, else go to 5. 5. At Ck::Ckr,)~d N=CJ steps 3 - 10 of the above-mentioned procedure are realized and ~ek.~~[~l is determined. Condition (7) is checked at JV=~ If (7) satisfied, assign J(=k.(a), and go to 7, if not go to 6. 6. Assign K=k+ I and select ~ E:.Jl~ according to conditions (Z") (in a particular Case ~:: roJN= ,,~ ...). Assign ~ = Y.._,+~
•

K(aJ= K(g- e)

e=

%(a- RJ =~f8,,1< =1" 2

",K!a4i.

e:o,i..s;

kfg-e),. e., the input of LS successive observations JJ and X do not lead to a change of the model structure YI<. ) (At ~(3-eJ..q:; only condition Kra):: kfi-e)~ €= "L.s ' is ohecked). If it is satisfied assign K·= k(l) and go to 8, else go to 2. 8. Check the criterion for stopping parametrio identifioation: t

I,Ls ;

1 ",;

Y.ca ':

(16)

mA)( ":AX It #Cs

le

"I<. J= ',ne.

J

Cl(~

( ne

-

is the dimension of vector C else go to 2.

C It

(le

~(~- Lp) 1<

E

)

€ >0 i

= (J K.. ) ). If it is satisfied, go to 9,

9. End. The utilization of the above prooedure "on-line" would allow the modeling of plant. with instationary indefiniteness. In this case, the parametric identifioation of the models is carried out by the modified criterion for optimali ty of C" by mean-root-square method: (17)

where

S$~RS B

= N'

~ ~N-~ [ 8='

YN, ()(,. Ck,")

2

- ~~] -

mcf~

ie the weight of observations in acoordance with their validity.

ON WE CONVERGENCE OF STRUCTURAL IDENTIFICATION PROCESSES

In case of known distributions }'~(~and hb(~, the necessary condition the convergence of the functional. determinated sequence the torm:

{'2; ,y()( .v..~=f'1iS I(

k' a

of

70

D. Burev

N

J(-t

/C.-I

(18) ( Zol.q . I{q ) (

)

3: I " ~ J 1(., Y.

19 ~Q

o

=

(f

I<-f I( 2 Ne /(-, I< 2 III N-I c AI J(, lr'" le-I ~ IN ~ 2 .~q +~,,). ~ HQ -~)-~9 j·E(1{ "')t~{<,·4!)~,-2 i >q 3-' (J 0 3- t ; (f 3-(" 3=1; r ' (f (f"3 ~ 4.t!.v.. ~I<. Cl"" ~ ~a- D 1e-4 .f " ~-,

Z N

(I

31(-' - 'N":

J-I

'01(-1' Oq :

<1

(J

~I(Q-;jL. ~~ d

3s ,

)

d

= ~q- N-~tt -eta . g:l(/d

"'et 0

(1

'J.'he sufficient condition may be written in the form:

[ .; (ol ;'-'+cr,")(R;' - cY'"; ) ] Z N''''' 2 N le .::. E (0/ /C.-I + 8';") Z(R '!..~I()2. -

(20)

,_, ,.

',=,

a s

I< J<.:f,c, .. . )

a

y

0 Efo,1 1 . ".:.J

.Ie shall no t e, t i l e'.t GCl tisfying the condition for decreasing of the variance

of

eJ<.

(21)

is not sufficient to guarantee the convergence of the process. In c a se of unknown distributions Py(y) and &C=9the weakest condition for convergence of t h e probaoili ty functional sequcclc e J ~:J('" (a)}, f,c) ... , is a result of the condition for local improvement of estimations

3:

(22 )

) \lv ~: y.(~) > < Cif::C) I
> 0,

J<

= f,~~ ...

A TEST .3AliIPLE Ne shall discuss the application of the method for structural identification to the solution of the following simplest test sample. Let us synthesize the mathematical model of plant:

(23)

/I::

X

2

- 2)( + CV

by accesses of values of X and :J (the values of /J are determined by given values of X , generated values of W (non-correlated with X) and (23) Table 1). The functional form of plant (23) is assumed to be unknown and the initial structure of the synthesized model is determined in the class of linear functions:

Using values of)c and /I (Table 1) as well aB the mean-root-square method . 0 y. "0 allow the estimat~on for C, 0: ' " C t = -1,97437. The values o=c,·x and eo: ~- Yo are given on Table 1. G o = 11,07585; Z:(E~-Eo)2.", 803,12598. In order to determine '2:o,Yo the range of change of Yo is divided into 4 ({(Y. '" 4) intervals with bounds [-10,-5), [-5,0),[0,5),[5,10]. v'le have:f".CYot)= 16,72915, p,=3;€oCV.J= 2,42595, P2=2;Eo("~)'" 2,36589, P3=2;~(Yo,,)= 16,99592, P4=3. By (4) ~t,yo = 0,62555. The check of condition (7) with H=10 is with negative resul t:

7J

Adaptiv e struct ur al id entification

'fABLE 1

Results of J tru.ctural Identification of Plant (23 )

N!'

X

c.u

Y

Yo

1 2 3 4 5 6 7 8 9 10

1 -1 2 -2 3 -3 4 -4 5 -5

-0,17086 -0,19807 0,09965 -0,14744 - 2 ,15893 C.34340 2,42071 0,63981 0,23324 -0,30301

-1,17086 2 , 60193 0,09965 7, 852 56 0,84107 15,34340 10 ,42071 24,63981 15,23324 34,69699

-1,97437 1, 97437 -3,94874 3,94874 -5,92311 5,92311 -7, 8 9742 7,89746 -9, 87185 9,87185

(25)

l

'2E.,Yo 2Ien f1+ ~_

~o~

I

I -1, 074041

=

Y-

Eo 0, 80351 0,82756 4,04839 3,90}82 6 ,76418 9,42029 12 , 31319 16,74233 25,10509 24,82514

-0,95888 2,98986 0 ,11321 8,01069 3,21627 15 ,06249 8 , 35031 24,14527 15,51533 35 ,25903

>

(;!;

= 1,07404

1,96

£. -0,21198 -0,18664 - O, (J13 jG

- 0 ,15 813 -2,37520 0 ,28091 2,07040 0,49453 - 0 , 28209 -0,56204

- - = 0 ,74073. ~-3

',Vi th

the help of the i terati ve process ( Sa,9a) we go to the second iteration of the process of structural identification of plant (23): (26)

Using the values of Eo" ~- Yo, x ,Yo from Table 1 a llo\'ls the estimations for a.."ld C~ to be obtained: C~ = -0,51433; c3~ = -1, 82 .10- 7 • On Table 1 AI A, Y are given the values of Y.=Yo..... C, 'r;,x+c.~x and E,= ~- f • et =-0,0 3438 ; ~(€., - g,)'::: 10,66297. In order to deter.nine ~~"Y1 the range of change of Y, is divided into 4 (Ry , =4) intervals with bou..."lds [-1,9),[9,19),[19,29), [29,39]. Estimate: £,(Y..)= - 0 ,14585, P1=6; ~,(Ycz) =-0,00059, P2=2; E,(Vc2.)= 0,49453, P3=1;e,(y,lt) = - 0 ,56204, P4=1. By (4) ~:,'Y,= 0,05618. 'l'he check of condition (7) with N=10 is with positive result:

C;

(27 )

lRn /- ~E"Y, l

f

+ ~EIJY,

1-0 ,241541 = 0,24154 <:

=

1,96

VfO - 3 -f

':::

0 ,74073.

The process of structural identification of plant (23) is ended. APPLICA'U ON OF 'mE METHOD

'£he method for adaptive structural identification was used in the synthesis of a mathematical model of a transport process in an open-pit mine, carried out by transport vehicles of one type with capacity R. The model is defined in the form: (28)

Y,

t

= U 1 .X 1

lr2 = U2.~ (t - transpor ting),

'

where ~1 and lV2 are the times between two successive indications of one and the same transport vehicle before the dispatching stations for loading and unloading; (29) I.S. -F

l

u11

U1

=

.

~1_ • ~

= 1,n;

U2

r12] , . =

= L~

J

1,m ;

D. Burev

72

if the vehicle is addressed to the i-th loading station, i=1 , n; u i1 = 0, if the vehicle is not addressed to the i-th loadinr: station,

e'

=~',

if the vehicle is addressed to the j-th unloading station, j=1,m; if the vehicle is not addressed to the j-th unloading statio~, n is the number of loading stations; m is the number of unloading stations; u j2

X1 =

CP:~:l~::::::~;i·~:~r~-r ~. ~:~~!~:::::~!~:~E~ I ~

(x1n ,···,xM+5

n'Co~

~

(X1m,···,X5m,vo:j

p2(C o ) is the initial structure ~ ,defined as a quadratic polynomial of X1 (X2); X~i and X~j is planned productivity of i-th loading station (i=1,n) and of j-th unloading station (j=1,m); xJi and ~j are time intervals of the last vehicles addressings towards i-th loading station (i=1,n) and j-th unloading station (j=1,m); x1i and X3j are the densities of transport flows towards i-th loading station (i=1,n) and j-th unloading station (j=1,m); xli and X~j are the numbers of transport vehicles waiting in front of i-th loading station (i=1,n) and j-th unloading station (j=1 ,m); X~i =n/N ; X;j ::m/lI ; U is the num ber of working transport vehicles; X~+5 i are indices of the quality of production loaded at the i-th loading station (i=1,n ; r=1,M); M is the number of quality indices ( X~i - average content in percents of S in the ore, xii average content in percents of Cu in the ore, i=1,n; M=2); C~ and C~ - identifiable parameters. Model (28) is used for transport process control by addressing a transport vehicle towards stations with numbers (i)opt and~)opt, for which the times ~1i and 'V 2j ' computed with the help of the model (28), are minimal among all i=1,n (j=1,m). The model synthesis is carried out under the following conditions: 1) Accesses of values of X~} , ~ x~ U~}, ~U~} 1g } {'V 2g} are constant, with capacity N=100 (g=1, 100). 2) The class of definition of the structure of model ~ ie the class of polynomial functions. ~ C?; (p2(X, Co) , i.e., the iterative algorithm for generation of the model structure is defined in the form:

I

1, l

,pr

=

DO)

1\

of

C

3) Since YI<. CC") is a linear function in respect to Ck ' for estimation a standard programme for solving the system of linear equations is used:

k

where S<5 MRS is estimated according to (10) and (30). Q is estimated as follows: a) for the model of the loading process Q=36,k=0 and Q=45,k=1,2, ••• ; b) for the model of the unloading process Q=21,k=0 and Q=28,k=1,2, •••• As a result of the performed structural identification the models of loading and unloading processes were obtained of 8-th power in respect to X (P8(X», i.e., k=0,1,2 (Table 2). We shall note, that the synthesis of models of 8-th power in respect to X, without the method used here, would lead to solutions

Adapti v e s t r uc tura l i den t ifica tion

73

of problems of type (10), where t :1.e nU.'Jlber of yariables C is up to 5435. TABLE 2

Results of a Structural Identification of a 'I'ransport Process

k=O Model of loading

0,79588

k=1

k=2

0,32830

0,03009

..-

l)Io del of ,nloading

0, 85402

0,26507

_.__ ._ - -- 0,04312

CONCWSION A method is proposed for adaptive structural identification of parametric models, wnich uses as an optimality criterion for model structures tne result of the verification of the statistical hypothesis for insignificance of the correlation ratio between structures and remainders, obtained after the parametric identification of the structures by the mean-root-square method. It is s n own, that with the help of this method the model structures, in previously defined, suffiCiently general cl ass of parametric functions, can be estimated. An increase in the complexity of the structural models is not connected with an increase in the dimensi on of the vector of identifiable parameters, which considerably decreases computation expenses in parametric identification. REFERENCES 1.

A.~peHKenl> (1972) "U8Te"aTMtJeCKile MeT02Ui 8HanU38 1(MH8NMKM 11 nooI'-

H08MPOB8W1B npOM3BOilUITenl>BOCTM TOYA8", H31(. "SKoBoMMKa", M.

2. f.Kpallep, (1975),"i1laTellaTMQeCKMe lleT01Uil CTaTMCTMKM~ nep. C aHr.,

MlSll. "Jl4i2P", lIII. 3·A.K.UMTpOnOnl>cKHA (1971) "TexHMK8 CT8TMCTMQeCKMX BlllqMlCneW!I4'~, "Mi2pn

4. Jobnson N.L. Systems of Frequenoy Curves Generated by Methods of Translation, Biometrika, 36, 149 (1949). 5. D.M. Himmelblau. (1972) Applied Nonlinear PrOgramming, McGraw-Hill, new York. 6.UtinKMB H.S., KannMHCKilH A.H., Kp8CBeKep A.C., MeTOZ(lll nOK8nl>Horo ynYQWeHMR B 38Z(8Q8X CT8TMC~MqeCKO~ OnTMllMa8~MM. TeXHMQeCK8R KMOepHeTMK8 ~

6,

CTp.S - 11, 1975.