Time-Varying Parameter Estimation Combining Directional and Uniform Forgetting

Copyrig lll © I FAt: Illh Triennial \\'orld Congress. Tallinn. Estonia. L'SSR. 1'1'10

TIME-VARYING PARAMETER ESTIMATION COMBINING DIRECTIONAL AND UNIFORM FORGETTING P. LOhnberg*, A. Stienstra* and H .

J.

A. F. Tulleken**

*Department of Electrical Engineering, University of Twente, Enschede, The Netherlands **Shell Research B. V. , Amsterdam , The N etherlands

Abstract. The paper starts with a unified description of uniform and directional forgetting of old information only and of incoming information also, clarifying the mutual differences. Based on this unification, a new method is introduced. Compared to uniform forgetting, the novel method (like directional forgetting) forgets the same amount of information in the direction of incoming information, but (unlike directional forgetting) it forgets also a fraction of the information in other directions. This fraction is determined from UD-decomposition of the covariance matrix, such that directional forgetting prevails when the probability of covarianc e blow-up i s high. The proposed method is able to follow fast parameter variations with low probability of covariance blow-up. This method as well as known methods have been applied to simulations of a two-parameter process with systematic combinations of constant, gradually changing and suddenly changing parameters, for several signal-to noise ratios. The mean squared parameter estimate error without covariance blow-up appeared to be the lowest for the new method . Time-varying systems; stochastic systems; process parameter Keywords. estimation; recursive least squares; stability of numerical methods.

I NTRODUCTI ON

performance of all methods on simulations of a simple process wi th typical parameter variations and noise .

In many practical situations it is required to track fast as well as slow parameter variations simultaneously. E.g. in robotics, frictions may change slowly , whereas pay loads may change suddenly. Moreover, there may be little excitation (constant reference signal, small disturbances in closed loop). In the field of petrochemical process engineering one can think of the relatively fast variations in the feed composition or throughput to be processed (e . g . crude oil) versus the relatively slow changes due to fouling or aging of equipment.

TRADITIONAL FORGETTING STRATEGIES Process, Model. and Weighted Least-Squares Understanding of the new method is facilitated by a brief description of traditional forgetting strategies. In this section those strategies are considered as special cases of the weighted least squares procedure .

The tracking of fast parameter changes can be improved at the cost of parameter accuracy for slowly varying parameters by forgetting only old informa t ion in the est ima tor and not new incoming information. Even more important in this respect is the direction in which information is forgotten. For fastest parameter tracking, information should be forgotten uniformly over the parameter space (e.g. Fortescue, Kershenbaum and Ydstie, 1981).

T

Given process Yi = 'l'iOi + "'i with time-dependent, true parameter n-ve c tor 0 i' 'l'i'

For the described poor excitation, little information will enter in certain directions of the parameter space, and hence uniform forgetting will eventually cause cova r iance blow- up, leading to unstable controller adaptation . This blow-up can be circumvented using ad-hoc techniques. A systematic strategy is to use directional forgetting (e.g. Kulhavy, 1987), which may be slower in parameter tracking than uniform forgetting.

observa tion n-vector

and white process noise "'i'

N observations of

Yi and 'l'i (i = k-N+1,

k) yield output stack

Y , observation matrix ~k k related by Yk = ~kOk + Wk '

and

noise

Consider for estimate Ok of Ok at

stack

W, k

instant k and

diagonal weight matrix Q the weighted quadratic k a-posteriori prediction error cost function (1) This criterion is minimised at

Therefore Bertin, Bittany and Bolzern (1985) in a heuristic way combined directional forgetting with a variable forgetting factor from uniform forgetting. The present paper introduces a new method to combine the advantages of both uniform and directional forgetting at the cost of slightly higher computational complexity. This method is motivated by and derived f r om a brief survey of traditional uniform and directional methods. A next section describes the behavior and compares the

(2)

where the inverse information matrix P

k

satisfies (3)

387

Forgetting Total or Old Information

the most factor.

Forgetting old and new (total) information . To track time-varying parameters, e . g. Fortescue, Kershenbaum and Ydstie (1981) have introduced a variable total forgetting strategy related to

elegant

one

is

a

variable

forgetting

Uniform forgetting. Fortescue, Kershenbaum and Ydstie (1981) brought up the heuristic idea to keep the a-posteriori cost (1) constant at target J : O (13)

(4)

Using Uniform forgetting of old and new (Total) information (UT- method), these authors obtained the approximation

(5)

F denoting Fortescue, should

obey

0 < ;l.k F

where forgetting factor ;l.k

'"

Thus,

1.

of

the

(14)

total

T

+ consisting of old Sk_1 'l'k'l'k' T F information Sk_1 and new information 'l'k'l'k' only a part ;l.k is retained .

information

As (14) may become negative for

Forgetting old information onlv . To improve the speed of convergence, Ydstie (1985) and for the multivariable case Tulleken (1987) presented the exact solution to the case QO k

[:k~~:l_:

Q-J. : 1

(6)

0 T SO = ;l.kSk_1 + 'l'k'l'k' k

(7)

QT

o denoting Old. Thus of the old information

this

k

Using Uniform forgetting of Old information only (UO-method) (Tulleken, 1987), the strategy becomes d

;1.0 = k

o Sk-1

k

+

Ict k2

+ 4c

k

2

Minimal forgetting factor. The estimate is based on an effective memory length X = trace Qk. When k forgetting total information, it follows from (4) that

T

'l'k'l'k is retained completely . can

icki,

;l.T

only a part ;l.k is retained, whereas new information

These formulas Combined description . described by the general weighting matrix

large

requires a test and possible correction . Application of (9), (10) , (12) and (13) yields the exact solution which always 1 ies in the interval <0,1] (Kulhavy, 1987)

be

(15) (8)

and when forget ting follows from (6) that

= el k/"1 k , the different versions follow from table 1.

with relative retaining factor ' \

where

0

No forget ting

1

Forgetting of total information

;l.k ;l.k 1 ;I. 1 ;I.

Forgetting of old information

k

1

r

c

k

P

k

T ~[I - Kk'l'k] Pk-1' elk

J

k

;l.k J k _1

+

T ;l.k c k[l

c ), k

;1.0

k

Xk - 1

( 16)

-0--

O

Xo

related to

(Tulleken, 1987) follows from (1) as (17)

Directional forgetting . Uniform forget ting introduces blow - up in directions of the Tparameter space in which not enough information 'l'k'l'k enters. In those directions no information is forgotten by directional forgetting (Kulhavy, 1987). A short-cut derivation of Kulhavy's algorithm is given in the appendix .

(9)

(10) (11 )

T cl>kKk]C k ·

.

The nominal asymptotic memory length

with a-priori prediction error e k = Yk - 'l'k9k_1 and Kalman gain, inverse information matrix and cost

+

1

it

and it is taken ;l.k = max{;l.min,A ) · k

k

J

"1 k Pk'l'k = Pk - 1'1'k / (ok T 'l'k Pk- 1'1'k'

+

only,

Strong noise may cause ;l.k and hence X to become so k low that the parameter estimates change abrupty . To prevent that, a minimum X of X is chosen, the min k minimum ;l.min of ;l.k is calculated from (15) or (16),

For (8), the recursive form of (2) becomes

Kk

0

information 0

Xk - 1

TABLE 1 Constants in (8) for ForQotten Information elk "1k '\ 1

o

Xk = ;l.k Xk_1

old

(12)

The appendix shows that no information is forgotten perpendicular to 'l'k' when

Forgetting Uniformly or Directionally Constant forgetting factor . For constant ;I. < 1, accord i ng to (5) or (7) Sk decreases , he nce P k increases and the distribution "blows up" (Anderson, 1985). Of the many ways to prevent this,

(18)

i3 k is chosen such that in the updating direction

388

Pk~k

of

the

estimates

the

same

information

is

forgotten as uniform forgetting strategies do. That yields (see appendix) for Directional forgetting of Total (old and new) information (DT-method)

~i

v

(19)

Ok'

A simple

i {~ otherwise

k =

one

for ith element d < ~i 'I i

E

i

[ I,

is

to

select

a

of 0 such that

0 "

,

nJ

may be chosen twice the value of d after i convergence, or one ~ may be choosen as the maximum over ~i' Details are in Stienstra (1989). ~i

and for Directional forgetting of Old information only (OO-method)

k

of

threshold

if d

T 1 - ;\.k ~k = ;\.k - ~'

~O

function

I - ;\.k

(20)

1 -

SIMULATION N~

Tracking Experiments

FORGETTING STRATEGY

A simple mul tiparameter process is a first-order 5150 process Yk = akYk-1 + bku k _ 1 + ~k' where the two parameters pole a and gain b are varied such k k that typical parameter change combinations occur systematically. This is achieved by the variations shown by the straight lines in figures Ib and c. The combinations of those parameter changes are as indicated in table 2. The maximal value of a has k been chosen near the unit circle.

The difference between uniform and directional forgetting concerns the amount of information forgot ten in the direct ions perpendicular to the directions of incoming information. This difference will be calculated below for forgetting old information only, and used in a new algorithm. Only forgetting old information considered. This can be described by

J

will

be

(21)

TABLE 2 Process Parameter Change Combinations Samples a b k k

For uniform forgetting it follows from (7) that the (uniformly) forgotten amount of information is

o-

lOO constant constant 100-300} slow constant 300-400 300 slow fast 400 fast fast 400-500} constant slow 500-600 500 fast slow

(22) For directional forgetting it follows from (18) and (20) that the (directionally) forgotten information is (23)

The test signal was taken a Generalized Binary Noise (GBN, see Tulleken 1988) with unit amplitude. For constant a this should have the asymptotically k optimal non-switching probabil ity p = (I + a )/2. k For average a '" 0.95, p would become 0.975. For k fast following of time-varying parameters, p should be selected somewhat lower and hence was chosen 0.9.

The information forgotten in the direction of the incoming information vector Pk~k can be shown to be equal for both strategies and hence must equal JD, as the directional forgetting strategy only forgets tha t informa t ion. Therefore the informa t ion forgotten by the uniform forgetting strategy in the direction perpendicular to ~k must be

The equation noise

(24) For an intermediate strategy between uniform and directional forgetting (DU-method, here the DUO version instead of the similarly derivable OUT version), the difference JOu can be multiplied by a weighting factor v between 0 (directional forgetting) and I (uniform forgetting). This yields the combined forgotten information

0.3,

'" k < 300,

and CT~ = 0.2,

In order to

300 '" k '" 600.

Hence the signal-to-noise ratio RSN 2

var[Yk -YD, k J

=

akYD,k-1

bku k _ 1

+

bk /

var[~J

is

the

'" 2.4,

var[YD,kJ / where YD, k

deterministic

=

output

(var[!/IkJ = 0).

(24), The minimal memory length X (IS) (16) was min choosen 10, which is a reasonable value for this R ' Also, the constant J in (13) was chosen 1.5 SN O in order to obtain a nominal asymptotic memory length (17) yielding good performance

This weighting factor v can be taken small if there is a negligible probability of covariance blow-up. High covariance implies large elements of the diagonal matrix 0 resulting from UD-factorization T (Bierman, 1977) of P = UDU , where U is an upper triangular matrix, needed anyhow to update P in a numerically stable way. Several methods are conceivable to choose v

wa s an approximation of zero

study the effect of different signal-to-noise ratios, the standard deviation CT~ was taken CT~ =

(25) SUbstitution into (21) of subsequently (25), (23) and (22) yields

~k

mean Gaussian noise with variance CT~.

k

Xo '"

l"'k<300 16. 7, { 37 . 5,300"'k"'600

(26)

For the DUO strategy because the elements of the diagonal matrix converged to about 0.05 for 1 '" k '" 200, a decision level ~ = O. 1 was chosen. The OUT strategy was not tested extensively, as preliminary simulations showed about similar behaviour.

as a

389

Each experiment was repeated for 100 realizations of input u and noise ~k. k

independent

Tra cking Results The simulations were carried out for a ll six methods. As an exa mple, only the results of the DUO strategy are shown, see Fig. 1.

Fig. la

~ 1.2o.

g

1. DO. ... 0.8o. 0.6o. 0.4o. 0.2o.

d. Pole a

k

and the average over 100 estimates

-o.on~~~~~~H-~~~MHKH~~~

-0.2o. -0.4o. -0.6o. -0.80 -1. DO.

O.

-

-1.2nU+--~----'---.----r--~--~

0.00 100

200

300 400 500 600 SaMple nUMber ---)

a. Rea lizati on of test signal used for Fig . lb, c

, (fU

e. Gain b

.. ~

100 b . Pole a

k

200

300 400 500 600 SaMPle nUMber ---)

and sing le realization of estimate

k

and the average over 100 estimates

40 .

\SI

.i

Fig. If

30.

, 20. ~

o

E 10.

r

O.DU~--~--~----~--~--~----~

0.00 1

f. Memory length X k Fig. 1. Results of DUO strategy

c. Gain b

k

and single reali zation of estimate

It can be seen (cf . Figs. 1b /c) that the estimates follow the par amete rs reasonably, with specially b k (Fig. lc) depending on exci tation (Fig. la). Figs. 1 di e show th a t the algorithm co nverged after about 50 samples. From sample 50 - 300 the parameters were estimated well, and the memory length (Fig. l e) converged t o the expected values according to (26). From sample 300 - 600 , convergence was not as good because of fast parameter changes. The prediction error converged to the noise (not shown) . The results of the UO and UT strategy were similar. The DUO algorithm appeared to be much faster than the DO and DT strategies.

Frequency of Uniform Forgetting In order to c heck the frequency of uniform forgetting. it was checked in how many out of 100 simulations forgetting was unif orm at a certa in sample k . The result in Fig. 2 shows that the percentage uniform forgetting almost never re ached 100%, and thus directional forgetting did occur. Blow-up Check In order to check parameters were taken

covariance constant a

blow-up, the 0.95, b ~ k k 0.04. and the test signal was selected a GBN with asymptotically optimal non-switching probability p ~ 0 . 94 up to sample 200, and subsequently no input switching through sample 2000. Thus no information about b entered any more. ~~ was 0 . 2, k ~

1.

E

~

~

o

~

90.


C80. :::I

d

70.

U 60.

0.60 0.00

; 50. ~ 40.

0

30.

~ 0'05~,

20. 10.

~

O.UU~--~---T--~r---~--~--~

•

Fig. 2 . Per c entage of uniform f o rgetting at each k

correspo nding wi th RSN = were sele c ted as in Fig. in Fig. 3.

.. 15. :::I

1. 1.

All

other

Fig.

soT" Sat'lp le

The resul ts are shown

3 Nonper s istent ex c itat ion f or DUO strategy

u·

b: d

until d

1 (uniform

until

k

a nd a ; k

is below 0.1 again .

Then v

j

c: d ; 2

;

it

reaches

of this procedure,

5.00

a

e: b

and b

k

k

and

8

d

Z 0 .1.

return s k continues to The n

to 1 rise

dire c tional

2 over permanently, as no be co mes avail a b l e. As a result

k both a

k

(Fi g .

3d) and b

k

(Fig.

4e) were estimated well.

-5.00

Expe rimental extensi on up to 32000 samples showed robustness regarding covarian c e blow-up for the DUO (and obvious l y for the DT and DO) method, in contrast to fast blow-up of the UT and UO methods.

-10.0

Performan c e Indi c e s

O.OO~~------------------------

As the qualities o f the strategies are hard to evaluate from these qu a litative re s ults, the simulations were u s ed al so to cal c ul ate for quantitative comparison th e me an s quared erro r

-15.01+-----~------r_----~----~

0.00

J

... 0.60

Fig . 3b

Q

of

0.5

a,

a where

realiz a tion t,

b

d:

forgetting),

forgetting takes information about b

~ 10.0

a

nul'lber -- -)

condi tions

Fig.3a

~

•

Fig , 3e

0.08.00

a:

j

lo..

-l~O~OO ~~ I

indic a te s the estimate kt and similarly J f o r b . b

of

a

k

in

The resulting performances and bl ow-u p robustness results i n table 3 show the rel a tiv e s uperiority of the DUO strategy.

0.3

0.2

TABLE 3 Pe rf o r ma n c e and Robu s t ness of Methods Cos t l UT I UO I DT I DO I DUO

L-

O.ool~~~_T------r-----_r----_,

0.00

S 0.

a

1~

10

114

161

155

113

822

956

8 77

807

I - I

+

Ro~1

Fig. Jc

0.00 c 0.00

115

878

I

+

I

+

CONCLUSION

0

:; 0

1600 saMple

1500

n~ber

2600 ---)

This paper presents a different forgetting derivation and revisi o n strategy by Fortescue, (1981) and a transparent forgetting introduced by

Figure 3b shows no limit on d

even during constant 1 u (Fig. 3a), because information in the direction of a keeps coming in. As soon as d exceeds 8 = 1 1 k 0.1, v becomes 0 and forgetting becomes k directional. Hence d (Fig. 3c) stays constant 2

uniform descripti o n of strategies including a of the uniform forgetting Kershenbaum and Ydstie derivation of directional Kulhavy (1987).

Moreover, a new method (21) - (26) is introduced, combining the two approaches into one forgetting

:-\~)

I

strategy. This strategy (like uniform forgetting) is able to track rapidly changing parameters , but (like directional forgetting) is able to circumvent covariance blow-up.

In general, the information is proportional to

The mean squared parameter error over a large number of realizations in a fairly realistic Monte Carlo simulation with severely time-varying parameters appeared to be the lowest for the new method. Furthermore, the new method did not suffer from any covariance blow-up during extensive simulation trials.

(31 ) For directional forgetting , the result of applying the matrix inversion lemma to (31) is

[I -

.D


and r

For a simple derivation of directional forgetting (Kulhavy, 1987), assume a general updating equation for the signal matrix

Anderson, B.D . O. (1985). Adaptive systems, lack of persistency of excitation and bursting phenomena . Automatica, 21, 247-253. Bertin D., S. Bittanti, and P. Bolzern (198 5) . A prediction-error dire ctional forgetting technique for recursive estimation. Systems Science. 11. 7-13. Bierman, G.J. (1977). Factorization methods for dis crete sequential estimation . Academic Press, New York. Fortescue, T.R . , L.S. Kershenbaum, and B.E. Ydstie (1981). Implementati on of self-tuning regulators with variable forgetting factors. Automatica, 11, 831-835. Kulhavy, R. (1987). Restricted exponential forgetting in real-time identification . Automatica, 23, 589-600. Lohnberg, P . , and M.C. Vlot (1988). Recursive triple least squares identification for Box-Jenkins models. 8th IFAC Symposium on Identification and System Parameter Estimation, Pergamon, Oxford, preprints 684-698. Stienstra, A. (1989). Analysis of the DAFOD algorithm for estimation of time-varying parameters. Master's thesis 89R028, Control Laboratory El. Eng. dept. University of Twente . Tulleken, H. J. A. F. (1987). An adaptive forgetting strategy for on-line identification of mul tivariable pro cesses. Journal A, 28, 195-208. Tulleken, H.J . A.F. (1988). A generalized binary noise test-signal concept for improved identification - experiment design. 8th IFAC Symposium on Identification and System Parameter Estimation. Pergamon, Oxford, preprints 846-853. Ydstie, B.E . (1985) Adaptive control and estimation with forgetting factors. 7th IFAC symposium on Identification and System Parameter Estimation. Pergamon, Oxford, preprints 1761-1766 .

must be determined . 9 -T

k

-

9

is

k

-

N(O,Sk) distributed , i.e. p(ak) - e-9kSk9k/2 is choosen such that no information is forgotten

of

the

components

perpendicular to

~k

-P 9

of -9 k k defined by

in

the

directions

(28)

O.

This

requires

that

in

those

directions

the

equi-den:ity curves p(a:) remain the same, that is (29) Substitution of (27) (29) yields with (28)

into

the

left

hand side of

(30) The equality of

(30)

to

(29)

requires

~

= 1 for the UO-method

REFERENCES

(27)

k

k

Hence for the DT-method, the equality of (34) to (33) requires (19), and for the DO-method, the equality of (35) to (33) requires (20).

Simple Derivation of Directional Forgetting

~k

yields with

(35)

APPENDIX

and

(33)

(34)

The authors like to thank H. ~ilmink for his contribution in an earlier trial to combine uniform and directional forgetting, Prof. A. Bagchi for the review of ~ilmink's thesis, while Prof. J. Van Amerongen also reviewed the underlying one by Stienstra (1989).

~k

-1-1

f3 k 1(f3 k +c k )·

Substitution into (31) of (9) - (11) table 1: r = Ak for the UT-method k

Assume parameter estimate error a

(32)

Substitution into (31) of (32) yields

ACKNO~LEDGEMENTS

~k

the direction of

Hence the relative change of that information is

The novel strategy still requires a rather arbitrary choice of a decision level, from which it is decided to switch from purely uniform to directional forgetting or vice versa. The method can be enhanced using smooth transition between those modes . Also the method should be tested on higher-order processes and Box-Jenkins processes with additional convergence analysis (Lohnberg and Vlot, 1988) .

in which the scalars

in

Pk~k

1,

so

(27) becomes (16).

f3 is chosen such that in the estimate updating k direction Pk~k (9) the same amount of information is forgotten as uniform forgetting strategies do.

3Y2