- Email: [email protected]
Decision and Approximation Based Algorithms for Identification with Alternative Models
3a-094
Copyright © 1996 IFAC 13th Triennial World Congress . San Francisco. US A
DECISION AND APPROXIMATION BASED ALGORITHMS FOR Il)ENTIFICATION WITH ALTERNATIVE MODELS 1 V. H avlen a , .. '.... J.
Stecha,·~
a nd F . .1.
KralJ s ·"~
• Honc ywdl Tcd m ology Cente7' Prag ue, Hone yu;eJl In c., Pod vod{l. n~ fl.sko u veZf 4, 182 08 Pmha 8, Czech Republic, E-mail haviena@utio.(:os.cz •• Fuc:ully of Eh'CIrical En.qin eering, Czech Ter.hn ind Universit y, /(a1'lovo namesti 18 . J:.!! :J/) Pm h a 2/ Czech Republic, E-mail ..Jtecha(Q: con t7.01.fclk .(;tju i. cz _a . .4 utomatic ('ont rol I.aboro tory, S wiss Federal Institute of Ter.hn%gy. CIf-S09E Zll rir.h, S witzerland, E -mail k ra.U .~ .g:a.ut.ef:. e th z. ch
Abs tract. An a.pp roar:h to t he t r;u;king of t ime va rying para mete rs o f a d y na mic sys tem l>a.sed on the bayesia n view po in t is prcsc nt.cd . A set of a lter nati ve models of poss ihk parru ll et er development can be used to incorporate the prior knowl edge about. t.h e possible pa ramet er changes . Based on th is set. of models a tree of possi ble '; paramctcr hist.or i (~s :' is built. To achive reasona ble computationa l c':J mpiexity and memory requir(, Tllf! nt.s so me pruning of t he t ree m ust be ll s~d . Dec is io n-t heoretical a nd ap proxi mat ion-based pr u ning md.hods a re disclIssed K e ywu rds . Para meter es tim at ion , time-va ryi ng pa ra meters, mi xt ures of proba bili ty density functions , statistical decision . a.pproxilnatio n .
I INT ROllCCTI0 N
proves t he tracking qua li ty considerably (Hav lena , 1993 ), (Hav len. a nd St.echa , 1993), (Siech. et al., 1993) .
T he paper solves t he pro blem of trackin g of t ime-varying pa ra meters of a dynamica l sys tem if a finite number of models of JJossil>le parameter changes is availahle . To i:lchicvc tb e paramet er t racking ability in the. r:ase of time··-varyin g pa.ramete rs , some form of obsolet.e information for gettin g must be incorporated in the identificat io n algori t bJI\ . Va.rious modifi cati ons of expo nential forge tting based on di rec t inc r('asing t.h e pa ramcin UllC.e rta int.), (Peterka, 1981) MO' 11 >. <1 (Fortesc ue el ai, 198 1), (K ulh avy , 1993) . The forgetti ng ('a n be a lso indu r:ed hy a stochast ic mode l of p aralltet.e r develo pment inco rporat cd In t he prob lelll formu latioll ( Kraus, 1986) . Ofte n the utilizati on of t.he p rior informati on ill t his form J[Il-
1 T h ill r ~ lIc ar(' h w ;\ :<; l'I upp o rtcd b y t h e ( ; r a n t Age n cy u £ tllt: C'LCI:h repuhlic. g rant nu mber 102/ 94 / UU 4.
4202
If sevNal al tern ative hy p o th ~es a bo ut t he po~s i b l e parameter changes a rc a va il a l>le, a procedure fo r compu (ing th e probabi li ty d istribution o ver t he t ree of pos.<; ible pa ramete r develop ment h istories can be developed. Thp reason for building th e trt ~r. is the fad that a ll the informatio n about a pa ramete r change at time t is cont. ai fled in t he fut urE data only. T he t ree st ructu re is a convenicnt, tool fo r forma l desc ript io n of th e accumulat ion of t. h(' lllfo rruatio n co ntained in this oat.a . Tree s t ruc tu res are a pplica ble to m any s im il a r problem s , see e.g . (V iterbi , 191;7) . Usi ng the tree: th e proba bility distributi on over the set. hypothe~c~ about the parameter change at ti me t co nditi oned by t. he data up to time t + d call be rcu lfs ively updated. Note t.hat t he root of t he t.ree
or poss ible
co rrespondi ng t o time '- is d step delayed wit.h res pect to Cllfr ~ nt (re.a l) time t + d . To (:01)(' wit.h IIH' inneatiillg JilJlt'lIsionality of the problelll , a IHCdlOd fo r tree pruning mus t be incorpo rated in the algorithm . Ei t her a decision theory based approa.t:h can be used to select the >
2.2 Appli r. ati()fI lo ARX modeJ
Consider t ht' problem of identification of t.he paramel.ers of a.n A RX lIIodel t!
y(t) =
,~
L a,(t)y(l-i) + L bi(t)U(t-i) + e(t ) ! ::
1
i= O
= ,T(t)O(t) + e(l)
(7)
where the parameter vect or is
and the regresor is a nnit,! subset of (pt-I , u(t)}
2. BAYESIA'I APPROACH TO IOENTIFICATIO'l z(t) = [y(I - I), . .. ,y( l-n) , u(t) "
u(l-n)f
:t . l /v{()(ld , m odel structure and I!ammeters
The kn owl ~dgf" of the proccss based o n a given se t of input and o utput. dat.a up to time l-I denoted by 1>(- 1 (ut 1), y( 1),. .. u{l - 1), y{l - I) } "an be desc ribed by pred ictive (·.p .d.f.
=
p(y(I) IP'-' , u(t)i
for
t
= I ,.
model ca n be parametrized hy a finite number of paramet.ers
p(8(I)lP'- ' )
mu ~ t
Usin g ( I) . (3) and natllral cotldition of control (Peterka, 19RI) p(8(l)lp'-1 ,,(I )) = I,(O(/)IP'-') the incorp orat ion of the informat.ion cont aiIH'd in a new pair of data {u(l),y(l)) into (:1) (data-update step) is described '"
p(811)IP'):x p(~(t)lpt-', u(II, 11ft)) p(O(I)IP'- ')(1) where C( is used to denote proportlonalit.y Hp to a scaling facLo r. The time-update step is given a..'J
=
J
p(8(1 + I)1P' , 8(1)) = N(9(1). u;Q(t))
(9)
(10)
Suppose that. the prior distribution is normal
As during the st.eps dur:ed it is suffici ent tics of the (11). The well known formulas
(4) and (5) the normality is reprot o o perat e on the sufficient statisdata-upd at e step (4) results in t.he (Petnka l L981) fo r the conditiol1f'd
rllean
(I(t:t)
= (I(llt +
(5)
l) + C(t ll -I )%(1) 1 + zT(I)C(t ll-I)% (t) e(l lt-l)
(12)
a.nd cova.riaJlCe matri x
p(8(1 + I)[P'. 8 (1)) p(O(t )[P') d8(1)
and requires a pararneler df' velopment model
1'(8(1+ 1) f'D' , fI(I))
= N(.T(t)8(1), u;)
and t.he parameter devdupment mod el (8) defines the
(3)
out kno wledge abou !. the unknow n parambe al so propagall'd ill time.
,,(8(1 + 1)i'D' ) =
The ARX model (7) defin e, th e c.p,d.f. (2) which is
c. p.<1. f. (6) as
re pres~ nting
ete rs
(8)
whpre v(t) ,..... A((O, u;Q(t)) is a whit e noise sequence independcnt on the past data .
1'(y(t)I'D'-' , u(t), 8(t))
=
oe: often modelled
8 (1+1) = 8 (t) + v (l)
(2)
To ohtain t.he prerlict.ivc c.pd.f. (I) p(y(t)IPI-l , u(t) ) p(O(I) IPt-l) d8 (1 ) the c.p.d.f.
f p(y(I)fV'-'. u(t ), 0(1))
The ex peeled parameter "hanges can by a random walk
( I)
If a nnite. suffi cient statistic for t.his c.p .d.f. exists this
p(y(l)IP"', u(1),8(t))
a nd e(t) is a no rma1 whi te noise independ ellt of the data (D'-', ,,(I )) with known \'a riance e(t ) - N(O , u;') .
(6)
4203
C (t lt ) = C(I II - 1) C(t It - 1) %( t)zT(I)C( lit - I ) 1 + zT(t)C (t ll - I).(t)
(1 :\)
wh ere
(14 ) i~
t he output pred ict. ioIl {' rror. Th e t.ime-u pdate step (5 ) g l\'cs
9( 1+ Ill ) = 0(111 )
(1 5 )
C (t+ lit ) = C(l lf ) + Q (t)
(16)
The forgettin g d efIn ed by (lii)
IS
After the tree has hr.cll bllilt for given d epth d the po:::;t eri o r distributi on ( 18) c;tn be obtained as a marginal d istrihution
p(Ci " (t) IV"")
=:L P(c;, (t) , ... , c;, (t+d )IV"") (21)
{ i 1,,, . •I rl }
T his m a.rginal distrihu t io ll describes the pro ba bility distri b utio n over th e h + 1 h ran che~ from th e root node of the tree conditioned by t.h r. data up to t im e t+ d.
called lin ear forgetting. 3.2 E ualuation of Ihe tN" f or ARX model If th e parameter develop m ent mod els differ o nly in tlH' ('Qva ri a nce m atrices o f para meter changes, the c.p .dJ. (1 0 ) i, replaced by a set
3. A.LTERNATlVE PARAMET E R DEVE LOPMENT MODEl.S
p(O(/+ 1)I 'D', lI( t), Cl)
.3.1 Tn:e of po.')siblf: paramfler df1!l',:l opm cnt histories Suppo:-;e a set. o f It + 1 alternat.ive parameter development m orl el, 1'(11 (1+ I )IV', II(I. ). ,·; (I )) panune trized by thf' events Cl' i 0 ... . , h is givclI . Note that all the information a bo ut 1 he parameter cha nge a t t ime l i~ cont ain ed in the <' fu t ure" data . Th e rt ~ fnrc t.o co llect enough infor mrtt io ll fo r th e d ec ision abo ut th e tru e hypothesi s corretipond ing to t,he paramclcr chan ge at t ime t, it is necessary to upd a te the prior probability distribution over the set of possi ble event.s
=
p {c;(1) IV' )
(17 )
=
by the d ata D:r,' {u{l +l ). y(t+ I ), .. . , u( t+d) , y(t+d)} to the d-st ep Jap; posteri o r rl istr ibuti ll n ( 18) To d o t. his . th e t ree of all p u ssibl (~ ';parameter development h ls to ries" p(Ci,( t) . Ci , (I+I) . . . c;.( I+k)IV'-t« ) for k = 0 .. . . , d a nd i . E {O . .. It} mu st he propagated . The data-upd ate step
p(c;,in.c,, (I+ l ), . . . , C,~,(t+k .. I)IV'-t«) x p(!I(i +k) IV""'- ' , u(/+k),
= N(IJ(I J, Q(i) (I ))
ilnd the time-updat.e step (16) condit.ioned by the i-th bypoth esis
To evaluate t he posterior probability of a pat h (19) the predi ctive c.p.d.f. has t.o be evaluted for given dat a y(t+ k) . TillS c.p_d.f. is normal
p(y(t+k)(V'-tk- 1 , u( t +k) , c;,(/ ), . . .)
=
(22)
=N (zT(I + k) 8(t + kll+k~l , C;" . . . ) ; ",;' ( 1 + z T(t+ k )C (l+ k l l+k~I , Ci"
..
}zT(t+k )))
where lI(t+k It+k~l. c;" . .) and C(I+klt+k~I , co" ... ) arc th e st.atistic~ corresp onding to the final no rl p. o f the path cor responding to para met er change sequ ence cio (l },. C'I<_I (l + k-I ) fro m th e ro')t of the tree .
1 DECISION RA SE D AKD APPROXIMATION RASED TR.EE PRUNIN G
(IY)
cdn . .., C;~, (t+k ~ 1))
x p(ci , (L) . ci,(t+ l ), results fr om t.he 8 aycs formula . III 1.lle t ime- update step wc SIIPpOse th at ltit"' prio r distriblltio n over the pOf,sihle changes at l im p. 1+ k is itHlepeJl dt"I1t OIl t he previo us c hanges , i. e. for all k
Two a.pproaches how to keep the tree depth within a gi .... en limit can be used . Onc possibility is to select the hr'l nch of th e tree co rresponding to "op t im31" change c;o(t ) a nd c ut off o th er b ran ches or th e t. ree. Th en th e root o f the tree is s hifted I,ll •.he nex t nud e u n the se lected b ranch and the t ree is e:
p( ,,;,(I) ,<:i, (/+l ), . . . ci.it+k)ID'-t« ) =
= ph, (t + k) Iv '-t« }p(Ci" (I. ),
(20) .. Ci,_, (I + k ~ I) Iv'-t« )
4204
ill t he first Hoor o f t.h e t. ree by another c. p .d.f. from a give n class and use this approximat.ion as t he root of a new tree. This approach is ca lled approxima tion- based.
data {u(l), y(t)}
data {u(I+l) ,M(t +I)}
Oi,(t)
Ci , (1 +
I)
p(8(1+ 1) IV', "i, (1))
Fig . I . Part of th e tree of alternative hypotlwses fot' dept.h 2. Si rlLiiar approaches c a.n be fo und in the ext.p. nsiv(' li t.erat.ure o n non-lin ea r fi lteri llg from the 1960 's (sce the TeJcrcnccs in (Tittcring to tl. 198 ~))) . The basic differenr.e is th at o nly the fiit,ert!d values p(cdt) lvt) based on a single dat.a pair V; : :;: : {l/(l).y{l.)} were used. Ho wever , the (:ollclitioning by et ridlc r set. of t he data V:+d is crutial fo r t he illgo rit h m performance
1 . 1 Decision based appmQch Th e tr('e pruning cltn he fnrmuiflteu as a standard sequenti a l statistical decision prohl f'tn . The corresponding underlying event at time t is th e j -th possible parameter change cJ(i) E C = {II . . ,h } . The decision rI; E V corresponds to a("t :c pti H~ th e i-t ll dlange hypothesis and pruning the tree c orr p.s pnllllill~l y .
Introducing a loss funct io n J.(c}. d i ) the expected posteri or los::> (risk) for th e decision cl) ta.ken at. time t based on da t a ly-tJ can b(' eval uated as e(l'c(l ll+d),d] (tj) =
, = I: L(c, (t). d)(I)) p( cdtl IV '+')
(23)
..... here Pc{t lt + d) is anot her nOI.at.i o n for t he margina l (2 1). The Hu mber of e vents (changes) and number of decisi on:; are finite. T herefore the Bayes decision fu nction can be easily evaluat. ed and th e o ptimal de:cisio n (an Iw sel t'f:tcd n:-
loss functions sho uld s aLi~fy L(cord o) = '~ (cl , d d = 0 and /, (co , dd < rh , do I ., the decision d 1 ( "apply furgetting correspondi ng to Cl n) ror the "no cha.nge" event Co result s (unde r t.llC sufficient excitation conditiou s) only in the increa.<;cd sensitivity to noise while the decisio n do ("do not apply fo rgett.ing" ) for th e cha.n ge Cl ma.y disahle the parameter t. racking capabili ty of the algorithm. The set, D call al so contain m ore complex decisions , for exa mple to build a no ther Hoo r of the t ree. This enable1j liS to IJ se the tree with vari able depth . The optimal stoppi n ~ ru le should minimize th e expL'Cted total l o&~ whi ch consists of th e cost of t he tree update (d ependin g 0 11 the current. tree dcpth ) and the expected loss [ {e( p, (1 It+d + 1), dJ (I)) II+d} b o;;ed o n th e updated distribut loll .
4 .2 Approxim ation based approa ch
Instead o f t rel ~ pruning approach the dimenSio n of the tree can h~ reduced by approximation. The availabl e information in th ~ k-th flo,')r of the tree can be d e~ t: ribed by a rnixturt, of c. p .d.f.'s
I:
di~lriLutio!l
Lsing i'l sy mmct ri ~:al loss function L( Cj, dJ ) = 0 for i = j , {,( Ci , dj ) = 1 for i i j t h" MAP (max imum aposterior pro habiht.y ) decisio n-making algorit.hm is obtained _ Howev{'r, the ImilS fum:t ion ror pa rameter t racki ng is typically non symmctrir:al. Sup pose C(I describes "no change" event and Cl describ es i'L given dli.l.nge . Then reasonable
4205
{in
,i ~- l
p(9(t+k) !V'''c,, (t), .. .)
(25)
}
?'lot·e t ha t th e dist rihu tio n over the set of ch:mge sequence8 Cill (t ), . .. !ci ~ _,(t + k- l ) is conditionro by all the data '[) H-d whil e t.he filtered paramet ers estimatetl (condit ioned hy th E': data Dttk) are used. The mixturf' disl ribution (25) can be ap proxim a.ted by a c. p .d .L p.(9 (t+k) ) from a given c1ass_ Csin" this approximat io n 0 11 t.he fir~t floor of th e t rc.c, a new prior di stributio n p(9(t+l) IV') = pJ(/I(t+I)) can he used as a root of a newly built tree.
In the case o f the ARX model a ll the c. p .dJ.'s in the t ree ar(' lLorrnal
p(II (t+k)lv'+k . (',,(t ).
=.q
)=
,
(A(pUy(I)
1.5 .
0.5
(26)
0
9(I+k lt + k . C'" .. . ).
.,
~
0
20
10
"
and approximati oll o f t he mixt llrt'! (25) by a norfllal p.d.f. with matching tirst and f,('(":ond moments i:; the simple.st selection.
Fig . 2. Inpllt/out put data
L p(II(I+d)IV~ c;,(t) .... ) (27)
{'O,
, ',j _
10
80
"
00
6;(t+ I)
= &;(1) + ",b,(t),
lIi(t+ 1)
= "i(t)
L6b; (t ) = 0
I:
Both these ca....,es can be characterized by t.h e covari ance mat r io~s of t.h(' parameter increments of th e form
x p( c",(t), ... IV'-+
Qb =
nnc.~
;, EXA\IPLE The dec is io n- based approaeh will be illustrated by the tracking of the parameters of an A RX sys;t.cm with ab rupt changes o f the gain a.nd tim€' delay. Suppose a first-order system
COy
= 0.2 s. For o < Td < 'l7 ~ the s truClure of t he dicrete · time model will be (usin g ::-1 for a dd a:~' o pe rato r)
2
2
= O' L1 + (0'/
2
O' L)1I
-
.,.
= cov {t>a(t)) =0 variance {l"f in the direction " II'II = 1, and l1"i in all directions orthogonal to l.
(30)
vari-
Consider t.he hypot.h eses Co (no change), Cl (cha.nge in d elay) and C2 (change in gain). In t he case of no cha ngf' the corresponding covariarKe m atrix Q bO) O. In t he
=
case o f gain chall~c the c()vari ance matr ix Qil) wi ll be given by (30) wit.h the direction of possibl~ ch an get; given by the current parrlmet. er value (approxim ated by the bc,t ava ilable estimate b(tlt))
[&0 b,
1-
= 1 s. is sampled will) sa mpling period TIJ
{~b(I Jl
Qa
with
T
"
InpUu(l)
Some approxima ti on may also bl' necessary if the estimated paramet.ers are to be used for controller design . \Vhilc tbe dcci::;iofl abou t the clJanges (tree p ru ning) tCLkes plCLct' CLl LI([)(' I, the "real lime" when the latest items of dat a are meas ured is 1+ d . The informa.tion a bout the paranwfc: rs for co ntro l d esign is given as
p(IJ(t +d )lv'-kI) =
.
~,
b,iT
, 0"/
- J b6 +bl~ '
2
> 0,
fT.l
=0
In the case or del ay change, from the condition
L i Llb i =
o the covariance m atrix Qf) will he given by (30) wit.h th e d irection of "forbidden" cha.nges given by
where 11 (1) = [6 0 (1) 6,(t) b,(t) a,(t)JT are unknow n parameters. If the gain o r t.irrw-dday is changed, the expected cha nge of {h p pa r amc{j~ r :"- can be d e&:ribed In a relatively simple way as follows: - the cha nge or t.he gain l( ill (28j r~s u ll.~ in bi(l+ I) = bit/) a;(1+ li
I
= ~" .j3 .
0'/'
= 0, 0 ~
i.e. on ly changes o rthogonal to l a re possible. Data correspoll uifl g to tht ~ fo ll owing paranle:f.cl"s - initial i'ictti ng J{ = 1, 1J = 0.1 1) Td 0 .25 - change Cl at time t 2r.'~ T{ - change C2 at tirm: t 1n, I{ = 2, Td = 0.25 - change Cl at t.inw t 60, I.... = 2, Td 0.1 \\'ith sufficient excitati o n hy a pscudora ndo m input werf'
= = =
+ "'bdl). 6b i (t) oc bolt)
= a,(t)
- the change of the delay Td in (28 ) res ul ts in
4206
=
= =
P IlfIllMlm" bO
--- ---,--,.- - --
.~-
- - ,.. _ - , - - -
~-
0.2
2.'
0.'
lJ
0
,.
0
.~~~~~~~ ~~··~,y_m,~' ~oo~~~~~~~~,~~~~
,
'"
, 30
,.
0."
o:r-
. ,.~
0.'
0.0.5
~.
"
60
"
V-'" 3
••
a nd t heir probabili t ies
used (see Fi g. 2) . Th e ba..<;ic \1 A P i\pp ruach to t ree pr u ning wa.s used with tree de pth d .::;: 1. and fixed pr ior probabilities of t he hypo t heses p( c, (I ) 'V') (0.9 0.05 0.0 5). The se lected hyp ot heses a.nd t heir probabi li tic.s arc d epictrxl in Fig :~ . Est im ated parameters a and b a re shown in Fig. 4. f'u tt· 1 hat in th iS case t he t ree dept h d = 2 i:-; s ufficient fo r c.o r ree l oN('d ion af a ll changes.
=
08
0.7
Anderoon, 13 D. O. and .I. B. Moore (1979). Op timal Filt t: T"ing. Prenti r.e Ha ll, En glf'wood C liffs. Fortesque, T . H.. I.. S. Kersclwnhaum a nd B. F . Ydst ie (1981) . Implementation ofst' lf-Lunin g reg ulato rs with vari able fo rgett in g facto rs. A Ill omatic(J : 17: 83 1 835. Hilv[ena , V. (19 93) . S im ultaneolls pa rarn ct.p. r track ing a nd IjtaLe es timation in a l i n l~ a r ~ys t e m. Automat.ica, 29 . 1041 1052 . Hav lena, V. and.l . Stecha (In9:)) . Fault Detection Based on Alt ern at ive .\ l odels of Pa rarnctr.r Developm ent. In: Pro ceedin gs of the Int ernat iona l vVor'kshop on .!lpp liuf / \ ut omatic Control IltAAC J93, pp. 44-47 , Pr ague.
4207
60
50
70
"
~
1",-\
li. CO N(,[XSI ON
7. REFE ltE\CES
"
P;II'M'OeIerb2
j
0.'
0: /\ novel approad l to Lime- varying parameter t racking based o n t he baycs iau vir:w point was prese nted . In many pract ical s it uatio ns a set o f a ltc l"llative mod els o f possible para llleter cha nges is avai lahl e. Rased o n t hi s set o f mod els a t rL'~; of possible "paranlet(!r hi storif'~'i" can be buil t. T ,vo basic: lllf!1.hods for pruuin g of this tree using decision-t heoreti ca l a.n d a. ppro x itll ati o n ~ based approach were suggest ed .
~
~-
0.2
0
"
IW
o.
SclCd ~d h ypothes(~
,.
I
0.'
'2'·~_~--e'. o 10 20
60
P.ametef bl
0.2 .
0.6
Fig. 3.
t
PM:wncIer 31
l
;J
V
__ . .l-... ___ _ -"
10
W
•
~
~
ro
ro
j
ro
F ig . 4. True a nd o:.st imated par ameters
Ku lhavy, R. and M . Zarrop (1993). On agclleral concept of fo rgetting. 1nl. lo ,moal of Conlrol, 5 8 , 4, 905- 924. Kraus F. J. (1 986). Das V"p.rgcsscn in n:kursiven Paramd f;T"sdHllzvcr/afiren . PhD. thesis, ETU Zu rich. Lew is . F. L. (1986) . Oplima l Estimation , J ohn W iley, New York . Papoulis, A . ( 1965). Probubiiily, Random \iariables, lIlId Sl ochasl ic ProceSM?5. J\.kG raw Hill , f'ew Yo rk. Peterka, V. (198 1). Raycsian Approach to Syst< m Id entification_ In : Tre nd." nnd Progress in Sys t em Idenf-ljic(ltion ( P. Eykhoff, 8d. ), Perg amon P ress, Oxford . Stecha, J., V HavleM and O. Vasicek (1 99:1) . Smoothing wit. h Alte rnat ive Mod els of Param et er Developme nt. In: Procer;dirlgs of the J2th lASTED 1nl. Conlerello ' " Modelling, Identificat ion and Control", l nn s hruck. Austria, Vol. I, pp. 179- 182. Tit,t.cr ill gto n , D. M .. A . F. M . Sm ith and 11 . E. \ \ akov ( 1985). Stat istical Analy.,>i.r; of Finit e lH il:lure Distf'ibtJ.iiofls . \V iley, New York. Viterbi, A. ,I. (1 967) . Ort hogonal Tree Codes for Commu nication in t he Prcsl:nce of \Vhite Gaus..c:;ian '\~o i s e . I E I ,' /<, [mn". Comm. Tech., COM 15 , pp. 238- 242 .
Copyright © 1996 IFAC 13th Triennial World Congress . San Francisco. US A
DECISION AND APPROXIMATION BASED ALGORITHMS FOR Il)ENTIFICATION WITH ALTERNATIVE MODELS 1 V. H avlen a , .. '.... J.
Stecha,·~
a nd F . .1.
KralJ s ·"~
• Honc ywdl Tcd m ology Cente7' Prag ue, Hone yu;eJl In c., Pod vod{l. n~ fl.sko u veZf 4, 182 08 Pmha 8, Czech Republic, E-mail haviena@utio.(:os.cz •• Fuc:ully of Eh'CIrical En.qin eering, Czech Ter.hn ind Universit y, /(a1'lovo namesti 18 . J:.!! :J/) Pm h a 2/ Czech Republic, E-mail ..Jtecha(Q: con t7.01.fclk .(;tju i. cz _a . .4 utomatic ('ont rol I.aboro tory, S wiss Federal Institute of Ter.hn%gy. CIf-S09E Zll rir.h, S witzerland, E -mail k ra.U .~ .g:a.ut.ef:. e th z. ch
Abs tract. An a.pp roar:h to t he t r;u;king of t ime va rying para mete rs o f a d y na mic sys tem l>a.sed on the bayesia n view po in t is prcsc nt.cd . A set of a lter nati ve models of poss ihk parru ll et er development can be used to incorporate the prior knowl edge about. t.h e possible pa ramet er changes . Based on th is set. of models a tree of possi ble '; paramctcr hist.or i (~s :' is built. To achive reasona ble computationa l c':J mpiexity and memory requir(, Tllf! nt.s so me pruning of t he t ree m ust be ll s~d . Dec is io n-t heoretical a nd ap proxi mat ion-based pr u ning md.hods a re disclIssed K e ywu rds . Para meter es tim at ion , time-va ryi ng pa ra meters, mi xt ures of proba bili ty density functions , statistical decision . a.pproxilnatio n .
I INT ROllCCTI0 N
proves t he tracking qua li ty considerably (Hav lena , 1993 ), (Hav len. a nd St.echa , 1993), (Siech. et al., 1993) .
T he paper solves t he pro blem of trackin g of t ime-varying pa ra meters of a dynamica l sys tem if a finite number of models of JJossil>le parameter changes is availahle . To i:lchicvc tb e paramet er t racking ability in the. r:ase of time··-varyin g pa.ramete rs , some form of obsolet.e information for gettin g must be incorporated in the identificat io n algori t bJI\ . Va.rious modifi cati ons of expo nential forge tting based on di rec t inc r('asing t.h e pa ramcin UllC.e rta int.), (Peterka, 1981) MO' 11 >. <1 (Fortesc ue el ai, 198 1), (K ulh avy , 1993) . The forgetti ng ('a n be a lso indu r:ed hy a stochast ic mode l of p aralltet.e r develo pment inco rporat cd In t he prob lelll formu latioll ( Kraus, 1986) . Ofte n the utilizati on of t.he p rior informati on ill t his form J[Il-
1 T h ill r ~ lIc ar(' h w ;\ :<; l'I upp o rtcd b y t h e ( ; r a n t Age n cy u £ tllt: C'LCI:h repuhlic. g rant nu mber 102/ 94 / UU 4.
4202
If sevNal al tern ative hy p o th ~es a bo ut t he po~s i b l e parameter changes a rc a va il a l>le, a procedure fo r compu (ing th e probabi li ty d istribution o ver t he t ree of pos.<; ible pa ramete r develop ment h istories can be developed. Thp reason for building th e trt ~r. is the fad that a ll the informatio n about a pa ramete r change at time t is cont. ai fled in t he fut urE data only. T he t ree st ructu re is a convenicnt, tool fo r forma l desc ript io n of th e accumulat ion of t. h(' lllfo rruatio n co ntained in this oat.a . Tree s t ruc tu res are a pplica ble to m any s im il a r problem s , see e.g . (V iterbi , 191;7) . Usi ng the tree: th e proba bility distributi on over the set. hypothe~c~ about the parameter change at ti me t co nditi oned by t. he data up to time t + d call be rcu lfs ively updated. Note t.hat t he root of t he t.ree
or poss ible
co rrespondi ng t o time '- is d step delayed wit.h res pect to Cllfr ~ nt (re.a l) time t + d . To (:01)(' wit.h IIH' inneatiillg JilJlt'lIsionality of the problelll , a IHCdlOd fo r tree pruning mus t be incorpo rated in the algorithm . Ei t her a decision theory based approa.t:h can be used to select the >
2.2 Appli r. ati()fI lo ARX modeJ
Consider t ht' problem of identification of t.he paramel.ers of a.n A RX lIIodel t!
y(t) =
,~
L a,(t)y(l-i) + L bi(t)U(t-i) + e(t ) ! ::
1
i= O
= ,T(t)O(t) + e(l)
(7)
where the parameter vect or is
and the regresor is a nnit,! subset of (pt-I , u(t)}
2. BAYESIA'I APPROACH TO IOENTIFICATIO'l z(t) = [y(I - I), . .. ,y( l-n) , u(t) "
u(l-n)f
:t . l /v{()(ld , m odel structure and I!ammeters
The kn owl ~dgf" of the proccss based o n a given se t of input and o utput. dat.a up to time l-I denoted by 1>(- 1 (ut 1), y( 1),. .. u{l - 1), y{l - I) } "an be desc ribed by pred ictive (·.p .d.f.
=
p(y(I) IP'-' , u(t)i
for
t
= I ,.
model ca n be parametrized hy a finite number of paramet.ers
p(8(I)lP'- ' )
mu ~ t
Usin g ( I) . (3) and natllral cotldition of control (Peterka, 19RI) p(8(l)lp'-1 ,,(I )) = I,(O(/)IP'-') the incorp orat ion of the informat.ion cont aiIH'd in a new pair of data {u(l),y(l)) into (:1) (data-update step) is described '"
p(811)IP'):x p(~(t)lpt-', u(II, 11ft)) p(O(I)IP'- ')(1) where C( is used to denote proportlonalit.y Hp to a scaling facLo r. The time-update step is given a..'J
=
J
p(8(1 + I)1P' , 8(1)) = N(9(1). u;Q(t))
(9)
(10)
Suppose that. the prior distribution is normal
As during the st.eps dur:ed it is suffici ent tics of the (11). The well known formulas
(4) and (5) the normality is reprot o o perat e on the sufficient statisdata-upd at e step (4) results in t.he (Petnka l L981) fo r the conditiol1f'd
rllean
(I(t:t)
= (I(llt +
(5)
l) + C(t ll -I )%(1) 1 + zT(I)C(t ll-I)% (t) e(l lt-l)
(12)
a.nd cova.riaJlCe matri x
p(8(1 + I)[P'. 8 (1)) p(O(t )[P') d8(1)
and requires a pararneler df' velopment model
1'(8(1+ 1) f'D' , fI(I))
= N(.T(t)8(1), u;)
and t.he parameter devdupment mod el (8) defines the
(3)
out kno wledge abou !. the unknow n parambe al so propagall'd ill time.
,,(8(1 + 1)i'D' ) =
The ARX model (7) defin e, th e c.p,d.f. (2) which is
c. p.<1. f. (6) as
re pres~ nting
ete rs
(8)
whpre v(t) ,..... A((O, u;Q(t)) is a whit e noise sequence independcnt on the past data .
1'(y(t)I'D'-' , u(t), 8(t))
=
oe: often modelled
8 (1+1) = 8 (t) + v (l)
(2)
To ohtain t.he prerlict.ivc c.pd.f. (I) p(y(t)IPI-l , u(t) ) p(O(I) IPt-l) d8 (1 ) the c.p.d.f.
f p(y(I)fV'-'. u(t ), 0(1))
The ex peeled parameter "hanges can by a random walk
( I)
If a nnite. suffi cient statistic for t.his c.p .d.f. exists this
p(y(l)IP"', u(1),8(t))
a nd e(t) is a no rma1 whi te noise independ ellt of the data (D'-', ,,(I )) with known \'a riance e(t ) - N(O , u;') .
(6)
4203
C (t lt ) = C(I II - 1) C(t It - 1) %( t)zT(I)C( lit - I ) 1 + zT(t)C (t ll - I).(t)
(1 :\)
wh ere
(14 ) i~
t he output pred ict. ioIl {' rror. Th e t.ime-u pdate step (5 ) g l\'cs
9( 1+ Ill ) = 0(111 )
(1 5 )
C (t+ lit ) = C(l lf ) + Q (t)
(16)
The forgettin g d efIn ed by (lii)
IS
After the tree has hr.cll bllilt for given d epth d the po:::;t eri o r distributi on ( 18) c;tn be obtained as a marginal d istrihution
p(Ci " (t) IV"")
=:L P(c;, (t) , ... , c;, (t+d )IV"") (21)
{ i 1,,, . •I rl }
T his m a.rginal distrihu t io ll describes the pro ba bility distri b utio n over th e h + 1 h ran che~ from th e root node of the tree conditioned by t.h r. data up to t im e t+ d.
called lin ear forgetting. 3.2 E ualuation of Ihe tN" f or ARX model If th e parameter develop m ent mod els differ o nly in tlH' ('Qva ri a nce m atrices o f para meter changes, the c.p .dJ. (1 0 ) i, replaced by a set
3. A.LTERNATlVE PARAMET E R DEVE LOPMENT MODEl.S
p(O(/+ 1)I 'D', lI( t), Cl)
.3.1 Tn:e of po.')siblf: paramfler df1!l',:l opm cnt histories Suppo:-;e a set. o f It + 1 alternat.ive parameter development m orl el, 1'(11 (1+ I )IV', II(I. ). ,·; (I )) panune trized by thf' events Cl' i 0 ... . , h is givclI . Note that all the information a bo ut 1 he parameter cha nge a t t ime l i~ cont ain ed in the <' fu t ure" data . Th e rt ~ fnrc t.o co llect enough infor mrtt io ll fo r th e d ec ision abo ut th e tru e hypothesi s corretipond ing to t,he paramclcr chan ge at t ime t, it is necessary to upd a te the prior probability distribution over the set of possi ble event.s
=
p {c;(1) IV' )
(17 )
=
by the d ata D:r,' {u{l +l ). y(t+ I ), .. . , u( t+d) , y(t+d)} to the d-st ep Jap; posteri o r rl istr ibuti ll n ( 18) To d o t. his . th e t ree of all p u ssibl (~ ';parameter development h ls to ries" p(Ci,( t) . Ci , (I+I) . . . c;.( I+k)IV'-t« ) for k = 0 .. . . , d a nd i . E {O . .. It} mu st he propagated . The data-upd ate step
p(c;,in.c,, (I+ l ), . . . , C,~,(t+k .. I)IV'-t«) x p(!I(i +k) IV""'- ' , u(/+k),
= N(IJ(I J, Q(i) (I ))
ilnd the time-updat.e step (16) condit.ioned by the i-th bypoth esis
To evaluate t he posterior probability of a pat h (19) the predi ctive c.p.d.f. has t.o be evaluted for given dat a y(t+ k) . TillS c.p_d.f. is normal
p(y(t+k)(V'-tk- 1 , u( t +k) , c;,(/ ), . . .)
=
(22)
=N (zT(I + k) 8(t + kll+k~l , C;" . . . ) ; ",;' ( 1 + z T(t+ k )C (l+ k l l+k~I , Ci"
..
}zT(t+k )))
where lI(t+k It+k~l. c;" . .) and C(I+klt+k~I , co" ... ) arc th e st.atistic~ corresp onding to the final no rl p. o f the path cor responding to para met er change sequ ence cio (l },. C'I<_I (l + k-I ) fro m th e ro')t of the tree .
1 DECISION RA SE D AKD APPROXIMATION RASED TR.EE PRUNIN G
(IY)
cdn . .., C;~, (t+k ~ 1))
x p(ci , (L) . ci,(t+ l ), results fr om t.he 8 aycs formula . III 1.lle t ime- update step wc SIIPpOse th at ltit"' prio r distriblltio n over the pOf,sihle changes at l im p. 1+ k is itHlepeJl dt"I1t OIl t he previo us c hanges , i. e. for all k
Two a.pproaches how to keep the tree depth within a gi .... en limit can be used . Onc possibility is to select the hr'l nch of th e tree co rresponding to "op t im31" change c;o(t ) a nd c ut off o th er b ran ches or th e t. ree. Th en th e root o f the tree is s hifted I,ll •.he nex t nud e u n the se lected b ranch and the t ree is e:
p( ,,;,(I) ,<:i, (/+l ), . . . ci.it+k)ID'-t« ) =
= ph, (t + k) Iv '-t« }p(Ci" (I. ),
(20) .. Ci,_, (I + k ~ I) Iv'-t« )
4204
ill t he first Hoor o f t.h e t. ree by another c. p .d.f. from a give n class and use this approximat.ion as t he root of a new tree. This approach is ca lled approxima tion- based.
data {u(l), y(t)}
data {u(I+l) ,M(t +I)}
Oi,(t)
Ci , (1 +
I)
p(8(1+ 1) IV', "i, (1))
Fig . I . Part of th e tree of alternative hypotlwses fot' dept.h 2. Si rlLiiar approaches c a.n be fo und in the ext.p. nsiv(' li t.erat.ure o n non-lin ea r fi lteri llg from the 1960 's (sce the TeJcrcnccs in (Tittcring to tl. 198 ~))) . The basic differenr.e is th at o nly the fiit,ert!d values p(cdt) lvt) based on a single dat.a pair V; : :;: : {l/(l).y{l.)} were used. Ho wever , the (:ollclitioning by et ridlc r set. of t he data V:+d is crutial fo r t he illgo rit h m performance
1 . 1 Decision based appmQch Th e tr('e pruning cltn he fnrmuiflteu as a standard sequenti a l statistical decision prohl f'tn . The corresponding underlying event at time t is th e j -th possible parameter change cJ(i) E C = {II . . ,h } . The decision rI; E V corresponds to a("t :c pti H~ th e i-t ll dlange hypothesis and pruning the tree c orr p.s pnllllill~l y .
Introducing a loss funct io n J.(c}. d i ) the expected posteri or los::> (risk) for th e decision cl) ta.ken at. time t based on da t a ly-tJ can b(' eval uated as e(l'c(l ll+d),d] (tj) =
, = I: L(c, (t). d)(I)) p( cdtl IV '+')
(23)
..... here Pc{t lt + d) is anot her nOI.at.i o n for t he margina l (2 1). The Hu mber of e vents (changes) and number of decisi on:; are finite. T herefore the Bayes decision fu nction can be easily evaluat. ed and th e o ptimal de:cisio n (an Iw sel t'f:tcd n:-
loss functions sho uld s aLi~fy L(cord o) = '~ (cl , d d = 0 and /, (co , dd < rh , do I ., the decision d 1 ( "apply furgetting correspondi ng to Cl n) ror the "no cha.nge" event Co result s (unde r t.llC sufficient excitation conditiou s) only in the increa.<;cd sensitivity to noise while the decisio n do ("do not apply fo rgett.ing" ) for th e cha.n ge Cl ma.y disahle the parameter t. racking capabili ty of the algorithm. The set, D call al so contain m ore complex decisions , for exa mple to build a no ther Hoo r of the t ree. This enable1j liS to IJ se the tree with vari able depth . The optimal stoppi n ~ ru le should minimize th e expL'Cted total l o&~ whi ch consists of th e cost of t he tree update (d ependin g 0 11 the current. tree dcpth ) and the expected loss [ {e( p, (1 It+d + 1), dJ (I)) II+d} b o;;ed o n th e updated distribut loll .
4 .2 Approxim ation based approa ch
Instead o f t rel ~ pruning approach the dimenSio n of the tree can h~ reduced by approximation. The availabl e information in th ~ k-th flo,')r of the tree can be d e~ t: ribed by a rnixturt, of c. p .d.f.'s
I:
di~lriLutio!l
Lsing i'l sy mmct ri ~:al loss function L( Cj, dJ ) = 0 for i = j , {,( Ci , dj ) = 1 for i i j t h" MAP (max imum aposterior pro habiht.y ) decisio n-making algorit.hm is obtained _ Howev{'r, the ImilS fum:t ion ror pa rameter t racki ng is typically non symmctrir:al. Sup pose C(I describes "no change" event and Cl describ es i'L given dli.l.nge . Then reasonable
4205
{in
,i ~- l
p(9(t+k) !V'''c,, (t), .. .)
(25)
}
?'lot·e t ha t th e dist rihu tio n over the set of ch:mge sequence8 Cill (t ), . .. !ci ~ _,(t + k- l ) is conditionro by all the data '[) H-d whil e t.he filtered paramet ers estimatetl (condit ioned hy th E': data Dttk) are used. The mixturf' disl ribution (25) can be ap proxim a.ted by a c. p .d .L p.(9 (t+k) ) from a given c1ass_ Csin" this approximat io n 0 11 t.he fir~t floor of th e t rc.c, a new prior di stributio n p(9(t+l) IV') = pJ(/I(t+I)) can he used as a root of a newly built tree.
In the case o f the ARX model a ll the c. p .dJ.'s in the t ree ar(' lLorrnal
p(II (t+k)lv'+k . (',,(t ).
=.q
)=
,
(A(pUy(I)
1.5 .
0.5
(26)
0
9(I+k lt + k . C'" .. . ).
.,
~
0
20
10
"
and approximati oll o f t he mixt llrt'! (25) by a norfllal p.d.f. with matching tirst and f,('(":ond moments i:; the simple.st selection.
Fig . 2. Inpllt/out put data
L p(II(I+d)IV~ c;,(t) .... ) (27)
{'O,
, ',j _
10
80
"
00
6;(t+ I)
= &;(1) + ",b,(t),
lIi(t+ 1)
= "i(t)
L6b; (t ) = 0
I:
Both these ca....,es can be characterized by t.h e covari ance mat r io~s of t.h(' parameter increments of th e form
x p( c",(t), ... IV'-+
Qb =
nnc.~
;, EXA\IPLE The dec is io n- based approaeh will be illustrated by the tracking of the parameters of an A RX sys;t.cm with ab rupt changes o f the gain a.nd tim€' delay. Suppose a first-order system
COy
= 0.2 s. For o < Td < 'l7 ~ the s truClure of t he dicrete · time model will be (usin g ::-1 for a dd a:~' o pe rato r)
2
2
= O' L1 + (0'/
2
O' L)1I
-
.,.
= cov {t>a(t)) =0 variance {l"f in the direction " II'II = 1, and l1"i in all directions orthogonal to l.
(30)
vari-
Consider t.he hypot.h eses Co (no change), Cl (cha.nge in d elay) and C2 (change in gain). In t he case of no cha ngf' the corresponding covariarKe m atrix Q bO) O. In t he
=
case o f gain chall~c the c()vari ance matr ix Qil) wi ll be given by (30) wit.h the direction of possibl~ ch an get; given by the current parrlmet. er value (approxim ated by the bc,t ava ilable estimate b(tlt))
[&0 b,
1-
= 1 s. is sampled will) sa mpling period TIJ
{~b(I Jl
Qa
with
T
"
InpUu(l)
Some approxima ti on may also bl' necessary if the estimated paramet.ers are to be used for controller design . \Vhilc tbe dcci::;iofl abou t the clJanges (tree p ru ning) tCLkes plCLct' CLl LI([)(' I, the "real lime" when the latest items of dat a are meas ured is 1+ d . The informa.tion a bout the paranwfc: rs for co ntro l d esign is given as
p(IJ(t +d )lv'-kI) =
.
~,
b,iT
, 0"/
- J b6 +bl~ '
2
> 0,
fT.l
=0
In the case or del ay change, from the condition
L i Llb i =
o the covariance m atrix Qf) will he given by (30) wit.h th e d irection of "forbidden" cha.nges given by
where 11 (1) = [6 0 (1) 6,(t) b,(t) a,(t)JT are unknow n parameters. If the gain o r t.irrw-dday is changed, the expected cha nge of {h p pa r amc{j~ r :"- can be d e&:ribed In a relatively simple way as follows: - the cha nge or t.he gain l( ill (28j r~s u ll.~ in bi(l+ I) = bit/) a;(1+ li
I
= ~" .j3 .
0'/'
= 0,
i.e. on ly changes o rthogonal to l a re possible. Data correspoll uifl g to tht ~ fo ll owing paranle:f.cl"s - initial i'ictti ng J{ = 1, 1J = 0.1 1) Td 0 .25 - change Cl at time t 2r.'~ T{ - change C2 at tirm: t 1n, I{ = 2, Td = 0.25 - change Cl at t.inw t 60, I.... = 2, Td 0.1 \\'ith sufficient excitati o n hy a pscudora ndo m input werf'
= = =
+ "'bdl). 6b i (t) oc bolt)
= a,(t)
- the change of the delay Td in (28 ) res ul ts in
4206
=
= =
P IlfIllMlm" bO
--- ---,--,.- - --
.~-
- - ,.. _ - , - - -
~-
0.2
2.'
0.'
lJ
0
,.
0
.~~~~~~~ ~~··~,y_m,~' ~oo~~~~~~~~,~~~~
,
'"
, 30
,.
0."
o:r-
. ,.~
0.'
0.0.5
~.
"
60
"
V-'" 3
••
a nd t heir probabili t ies
used (see Fi g. 2) . Th e ba..<;ic \1 A P i\pp ruach to t ree pr u ning wa.s used with tree de pth d .::;: 1. and fixed pr ior probabilities of t he hypo t heses p( c, (I ) 'V') (0.9 0.05 0.0 5). The se lected hyp ot heses a.nd t heir probabi li tic.s arc d epictrxl in Fig :~ . Est im ated parameters a and b a re shown in Fig. 4. f'u tt· 1 hat in th iS case t he t ree dept h d = 2 i:-; s ufficient fo r c.o r ree l oN('d ion af a ll changes.
=
08
0.7
Anderoon, 13 D. O. and .I. B. Moore (1979). Op timal Filt t: T"ing. Prenti r.e Ha ll, En glf'wood C liffs. Fortesque, T . H.. I.. S. Kersclwnhaum a nd B. F . Ydst ie (1981) . Implementation ofst' lf-Lunin g reg ulato rs with vari able fo rgett in g facto rs. A Ill omatic(J : 17: 83 1 835. Hilv[ena , V. (19 93) . S im ultaneolls pa rarn ct.p. r track ing a nd IjtaLe es timation in a l i n l~ a r ~ys t e m. Automat.ica, 29 . 1041 1052 . Hav lena, V. and.l . Stecha (In9:)) . Fault Detection Based on Alt ern at ive .\ l odels of Pa rarnctr.r Developm ent. In: Pro ceedin gs of the Int ernat iona l vVor'kshop on .!lpp liuf / \ ut omatic Control IltAAC J93, pp. 44-47 , Pr ague.
4207
60
50
70
"
~
1",-\
li. CO N(,[XSI ON
7. REFE ltE\CES
"
P;II'M'OeIerb2
j
0.'
0: /\ novel approad l to Lime- varying parameter t racking based o n t he baycs iau vir:w point was prese nted . In many pract ical s it uatio ns a set o f a ltc l"llative mod els o f possible para llleter cha nges is avai lahl e. Rased o n t hi s set o f mod els a t rL'~; of possible "paranlet(!r hi storif'~'i" can be buil t. T ,vo basic: lllf!1.hods for pruuin g of this tree using decision-t heoreti ca l a.n d a. ppro x itll ati o n ~ based approach were suggest ed .
~
~-
0.2
0
"
IW
o.
SclCd ~d h ypothes(~
,.
I
0.'
'2'·~_~--e'. o 10 20
60
P.ametef bl
0.2 .
0.6
Fig. 3.
t
PM:wncIer 31
l
;J
V
__ . .l-... ___ _ -"
10
W
•
~
~
ro
ro
j
ro
F ig . 4. True a nd o:.st imated par ameters
Ku lhavy, R. and M . Zarrop (1993). On agclleral concept of fo rgetting. 1nl. lo ,moal of Conlrol, 5 8 , 4, 905- 924. Kraus F. J. (1 986). Das V"p.rgcsscn in n:kursiven Paramd f;T"sdHllzvcr/afiren . PhD. thesis, ETU Zu rich. Lew is . F. L. (1986) . Oplima l Estimation , J ohn W iley, New York . Papoulis, A . ( 1965). Probubiiily, Random \iariables, lIlId Sl ochasl ic ProceSM?5. J\.kG raw Hill , f'ew Yo rk. Peterka, V. (198 1). Raycsian Approach to Syst< m Id entification_ In : Tre nd." nnd Progress in Sys t em Idenf-ljic(ltion ( P. Eykhoff, 8d. ), Perg amon P ress, Oxford . Stecha, J., V HavleM and O. Vasicek (1 99:1) . Smoothing wit. h Alte rnat ive Mod els of Param et er Developme nt. In: Procer;dirlgs of the J2th lASTED 1nl. Conlerello ' " Modelling, Identificat ion and Control", l nn s hruck. Austria, Vol. I, pp. 179- 182. Tit,t.cr ill gto n , D. M .. A . F. M . Sm ith and 11 . E. \ \ akov ( 1985). Stat istical Analy.,>i.r; of Finit e lH il:lure Distf'ibtJ.iiofls . \V iley, New York. Viterbi, A. ,I. (1 967) . Ort hogonal Tree Codes for Commu nication in t he Prcsl:nce of \Vhite Gaus..c:;ian '\~o i s e . I E I ,' /<, [mn". Comm. Tech., COM 15 , pp. 238- 242 .