Filtered and Predicted States for Discrete-Time Adaptive Control

Copyright © IFAC Adaptive Systems in Control and Signal Processing, Glasgow, U K, 1989

FILTERED AND PREDICTED STATES FOR DISCRETE-TIME ADAPTIVE CONTROL K. Warwick* and K.

J. Burnham**

*Department of Cybernetics, University of Reading, Whiteknights, Reading, UK **Department of Electrical, Electronic and Systems Engineering, Coventry Polytechnic, Priory Street, Coventry, UK

Abstract : The paper considers the adaptive control of discrete-time, noise corrupted, systems whose characteristics can be represented by means of an ARi~A model, this being posed within a canonical state-space framework. The effects of using different state vector forms, on self-tuning controller performance, are investigated with regard to filtered and predic t ed state representations when applied to various system types. It is shown how only the optimal observer for the state-space filter model is truly optimal in the sense of being a cons t ituent part of an optimal controller. The important aspect of t hese results is that the state-space prediction model, also known as the innovations model, is that which is widely used in practice, despite the fact that overall optimality conditions cannot be obtained with a state-feedback controller based on this state-space model. The suggestion is made that the state space filter model, also known as the noise-free measurement model, is a much more viable alternative. Keywords.

Adaptive control;

state-space methods; discrete time systems; stochastic

control; observers.

space framework. It is also worth pointing out that although an observer may well be optimal with respect to a particular state-space model, it will almost surely be non-optimal with respect to any other state-space model of the same system.

INTRODUCTION When a system is modelled by means of an ARMA process, in a univariate discrete-time framework, a controller design can subsequently be derived either in terms of a polynomial based method (Astrtlm and Wittenmark, 1973), or in terms of a state-space based method (Warwick, 1981). In each case, the system characteristics may vary slowly with respect to time, and when this is so the ARMA model is required to track these variations by

Of all the state-space descriptions of the ARMA model which can be employed, the innovations form (Goodwin and Sin, 1984), is perhaps that which is most commonly encountered, and this form is described in this paper as the state-space prediction model. It has recently be shown however, (Warwick, 1987, 1988), that the discrete-time ARMA model can also be reformulated as a noise-free measurement description, and this form is described in this paper as the state-space filter model.

means oft for example, a recursive parameter esti-

mation technique such as RLS-Recursive Least Squares (Shah and Cluett, 1988 ) or SLS-Simplified Recursive Least Squares (Farsi and Co-Workers, 1984), in order to operate an adaptive controller scheme .

It is shown here how both state-space and polynomial routes can be taken in order to obtain pole placement controllers, and it is shown that the routes can be equivalent. The central issue raised is the employment of the scate-space filter model, than the much more widely encountered state-space prediction (innovations) model in order to obtain state-space equivalence with pole placement forms (Wellstead and Co-Workers 1979). Hence, anomolies raised by the use of the prediction model are resolved and the automatic employment of such a model is seen to be 'bad practice', particularly insofar as observer optimality claims are concerned Further, the filter model is put forward as a much more viable alternative and, it is suggested, is that which is the most applicable selection in the majority of cases.

In the state-space approach, although the system input and output signals are available as measured variables, the state vector is assumed to be unknown and must therefore be reconstruc t ed in terms of system input / output data combined with the estimated system ARI~A model parameters. It is a general requirement that the observer employed to reconstruct the state vector sho u ld be optimal, in the sense of minimizing the mean square error between the reconstructed state vector and its actual, but unknown, value (Astrtlm, 1970 ) . In fact the requirement would normally be for an asymptotic observer whose reconstructed state vector tends to the true vector as time tends t o infinity. There are, though, a large number of ways in which the ARMA model can be represe n ted in s t ate-space form, each form being dependent on how the state vector is defined. So, although the observer to be employed may well be optimal in the sense of a particular state-space description, because many descriptions are possible, this means that there are many different optimal state estimates which can be used. Clearly the definition of optimality, with respect to a state vector is fairly meaningless, in a global optimality sense and only has a meaning, at present, within each individual state

In this paper the effects of using the different state vector forms, on self-tuning controller performance, are investigated with regard to various system types . It is also shown how the pole placement control of the state-space filter model is equivalent to that obtained by polynomial means. SYSTEM DEFINITION It is assumed that the system to be controlled can

129

130

K. Warwick and K.

be considered in terms of the univariate discrete time ARMA description:

J.

Burnham P, Q, S and Hare

~atrices

0 ..... . . .........•.. . ... .. 0

0" '"

Ay(t) = z-kBu(t) + Ce(t)

1

where the polynomials A, Band C are defined as A B

,

a z- '

+

+ b1z- 1 +

bo

C = 1

a, z

+

-,

b, z-,

+ C 1 Z - 1 + c 2 z-1

......

+

+

+

a z n

+

. . . .. . b z -n n

",

-a

,, ~ ,

0

1

n

bo

n

0

,Q=

:

" <;~

o .... . .. ... . . .. . .

(2)

b

0

0

',' ","

P=

-n

'

-a

,

0 (6)

....... c Z-n n

0

0

in which Z- l is the unit backward shift operator,

0 c

, HT

such that z- iy(t) = y(t - i). (u(t) : ttT) and (y(t) : tOT) are the system input and output sequences respectively and (e(t) : tOT) is a white noise sequence with zero mean and finite variance .

S

It is assumed that the highest common factor of the pair (A,B) = 1, and of the pair (C,A) = 1 . Also the integer part of the transport delay is denoted by k ~ 1, which necessarily means that for the definitions given in eq. (2) to hold, b o must be nonzero. Further, the order of each of the polynomials is given as n - realistically n is the highest of the orders of A, B, C such that where a particular polynomial is in practice of a lower order than n, at least one of its highest order terms would be zero. It must be remembered though, that at least one of the polynomials A, B, C is of order n.

The dimension of P is (n + k) x (n + k), whereas that of Q is (n + k) x 1 in which its (k - 1) lower terms are zero. Also the dimension of S i s (n + k) x 1 , in which its (k - 1) upper terms are

To conclude this section, the definition of the independent stationary noise sequence is such that E[e(i), e(j»)

0 for i " j =

(3) {

n for i

= j

= 0

o

C1 1

zero, and HT is of dimension (n denotes the transpose of H.

(4)

It is a fairly straightforward procedure to check that the state- space description, eq. (6), repre sents the original model, eq. (1), by eliminating the state vector ~(t) from the state- space equations. The state vector x(t) is itself unmeasurable and in order to app l y-state feedback control it must be reconstructed. The steady- state filter problem (Kal man , 1960) is for this case synonymous with the steady- state optimal observer problem (Warwick , 1987), in which it is required to find

where g(t/t) denotes the state estimate ob tained at instant t, given information up to and including that at time instant t .

[ I - KHJ{Pg(t) STATE-SPACE FILTER MODEL The state- space prediction model has, in the past, been generally regarded as the way in which an ARMA model, as shown in eq. ( 1), should be represented in state-space form, and this has provided a basis f or previous work ( Lam, 1980; Caines, 1972 ) . It is shown in this paper however, how such a formulation can lead to 'erroneous' results and can leave certain questions unanswered when attempting to link polynomial and state- space contr o ller solutions. In this section a s t ate - space formulation is introduced which also represents an ARMA model, but which contains a noise - free output measurement signal, it is termed the state-space filter model. The state vector obtained through this formulation is therefore different to that obtained with the state- space prediction model, although close connections exist (Warwick, 1987) . The state-space filter model of the ARMA process in eq. ( 1 ) is given by 1) =

P~ ( t )

+

Qu ( t )

+

Se Ct

+

1) (5)

y (t ) =

k) x 1 where HT

A full order observer for the state vector be written as

although this restriction can, if necessary, be readily relaxed by problem generalization.

~( t +

+

(7)

where E[.) denotes the expected value and n is a finite positive definite scalar value. Also, as the noise is assumed to be zero- mean, E[e(i) J

n

H~ ( t )

in which the vector of state variables is defined as ~(t) at the sampling instant t and where the

in which that the variance error is states 6(t ) =

+

Qu(t»

+

Ky(t

~(t)

can

1)

(8)

+

K is the Kalman gain vector chosen such mean square reconstruction error cois minimized, where the reconstruction the error between the true and estimated

~(t) -

(9 )

g(t)

and the error covariance is V( t ) := El (6 ( t ) - E[ 6 ( t ) j}{6 ( t ) - El 6 ( t)]}T )

( 10 )

The error covariance V(t) is then the minimum value of V( t), found when K = S (Warwick, 1987), such that the observer of eq. (8) becomes g(t

+

1) = [ I - Z_ lp) - l{ l I - SH)Qu(t)

+

Sy(t

+

1)} ( 11)

in which

P

=

P - SHP

(12)

With the Kalman gain chosen in this way, the error covariance series (V(t » converges to the zero matrix from any initial condition V(O) , i.e. the initial choice of state vector estimate x(t) is arbitrary, and stability of the observer-i s governed by the characteristic polynomial C.

Filtered and Predicted States for Discrete-time Adaptive Control 7he state feeback controller is formed from (l3)

where the state vector estimate x(t)is in reality used in place of its true, but unknown, counterpart. Also, it is assumed, for explanation purposes, that the controller requirements are merely for regulation, such that a reference input signal is not included in eq .(3 ). The inclusion or not of a reference input signal does not affect the purpose of this paper and in fact the addition of such an input does not affect the concluded results. By substitution of the control input u(t) into the state-space equations (5), we obtain the closed-loop expression (14)

P'

~

P

+

QF

(15)

The closed-loop denominator is therefore given by the determinant of I - z-Ip'. When the control objective is to place the closed-loop poles in pre-specified locations, if it is assumed that the required closed-loop denominator is given by T, the state feedback parameters F are found in order to satisfy the equation ~

det.(I - z-Ip') where T

= 1

+ t

T

1 Z_1

(16) + t

z z-2

+ •••••

+ t

n

z-n

( 17)

and ti are pre-specified. In the adaptive controller implementation results shown in a later section, the controller is based on a pole placement objective with F found from eq.(16), and the optimal state vector estimate g(t) is obtained from eq. (11).

STATE-SPACE PREDICTION MODEL Rather than employing the state-space filter model described in the previous section, the more standard innovations model referred to here as the state-space prediction model, can be employed in order to describe the ARMA system of eq. (1). It is obvious from the following equations that the state-space prediction model contains a noisy output signal and a state equation that depends only on past noise signals. The state vector obtained through this formulation is distinctly different to that obtained with the state-space filter model, although a close link exists. In order to gain an understanding of this statespace model it is perhaps best to consider in the first instance that the ARMA model parameters i

{ai' bit cif

= 1, ..... .

t

n} are known

values. The problem of time varying parameters and the need for an adaptive controller scheme need not therefore be taken into account until the next section which considers implementation aspects. The state-space prediction model can be written as: ~(t +

1)

~ P~(t )

+

Qu(t) + Re (t) (18)

y(t)

distinctly different from the state-space filter model state vector defined in eq. (5). Also P, Q and H are identical to those shown in eq. (6), whereas R is given by 0

c

n - an

R

(19)

Cl

- aI

It can easily be shown that the state-space description of eq. (18) is also a representation of the original model, by eliminating the state vector ~(t) from the state-space equations. It is also a straightforward procedure (Warwick, 1987) to obtain the minimum mean square estimator of the state vector ~ ( t) as ~(t + 1) ~ P~(t) + Qu(t) + Ry(t)

in which

~ H~(t )

+

e(t )

in which the vector of state variables is defined as ~(t) at the sampling instant t, where ~(t ) is

131

(20)

which can be seen in very simple terms by eliminating e(t) from the state-space equations (18), and P ~ P - RH. Not only is the state vector ~(t) distinctly different from x(t), but also the optimal observer equation (20) is different from the optimal observer equation (11). A major point of difference is the inclusion of the most recent output value in the state vector representation, eq. (11), whereas in eq. (20) the state vector reconstruction depends on only past output values. Both ~(t) and g(t)are though, optimal state vector reconstructions for state-space models which represent the original ARMA model. In both cases the optimal observer characteristic polynomial is given by the disturbance polynomial C, hence in the case of a system affected by white noise only, when C ~ 1, so the poles of the optimal observer all lie at the origin of the z-plane. In a similar fashion to that with the state-space filter model, the state feedback controller from the prediction model is formed from (21 )

where the state vector estimate w(t ) is, in reality, used in place of its true, but unknown, counterpart. By substitution of the control input u(t) into the state-space equations (18), we obtain the closedloop expression y(t)

~

(1 +

H(I - z-Ip'

)_1

R}e(t)

(22)

in which P' is that defined in eq. ( 15 ). The closed-loop denominator is therefore given by the determinant of I - z-Ip', as was the case with the state-space filter model, and thus the design of such as a pole placement control scheme is identical to that described in the previous section with particular reference being made to eq. (16). So no matter which of the state-space models is employed, filter or prediction, the state feedback parameters F are calculated in the same way, in order to satisfy a specific control objective in this paper the pole placement objective is described. The overall feedback control is however distinctly different in the two cases because of the different state estimates employed. The state estimates are both optimal, in the same optimality sense, however because the state-space models are different, so the state vector estimates themselves are di fferent.

K. Warwick and K. J. Burnham

132 IMPLEMENTATION RESULTS

C

It has been assumed, up to this point, that the ARMA system parameters are known fixed values, and hence the required controller parameters are also fixed . However where the system characteristics vary with respect to time, so it is required that the controller parameters are adapted such that they ~rack the system variations in sympathy. To this end a recursive parameter estimator (Shah and Cluett, 1988 ) is employed to continuously update an ARMA model of the system, such that at each sampling period, the following sequence of events occurs. 1.

Sample plant output signal.

2.

Update system parameter estimates, using such as a RLS procedure.

3.

Employ updated system parameters and regressive data to calculate state vector estimate g(t) or !(t).

4.

Employ updated system parameter estimates to calculate state feedback parameters F.

5.

Find new control input; u(t) u(t) = F!(t).

=1

- 0 .6z-'

O.lz-'

+

rather than C

1 as was the case .

Figures 7 and 8 illustrate the system output and control input for the filter and prediction models respectively. It would appear that in this case the prediction model provided both a smoother control input and a less variable output Variance u(t)

Variance y(t)

Filter model

0.091

7.576

Prediction model

0.066

6.379

Convergence behaviour of the RLS parameter estimates is shown in Figs. 9 and 10. It is interesting to note the biassed steady-state values which result, due to the noise colouring polynomial. System 3 : The third simulation was of a linear, open-loop unstable system, with the closed-l oop being placed at s = - 2 ± j3 in the s-plane. The system was given by y(t) = 2.1y(t - 1) - 0.7y(t - 2) + u(t - 1) + 0.5u(t - 2) + e(t)

6.

Apply new control input.

7.

Regress data values .

8.

Wait for next sample pulse, then GOTO 1.

with a sample period T input signal applied.

It is worth noting that in this sequence of events, it is not necessary to calculate the matrices present in the state-space description, all that is necessary is to find the state vector estimate and the state feedback parameters.

0.1 secs and a 0.5 step

Figs. 11 and 12 correspond to the Filter and Prediction state-space models respectively. It is apparent in this case that in terms of both lower control effort variance and lower system output variance , the filter state form is superior. Variance u(t)

To investigate controller performance under the different types of state-space control the following simulations were carried out. System 1: The first simulation was of a linear system affected by a unit variance white noise disturbance only, the closed-loop poles being placed as s = 3 and s = - 10 in the s-plane. The system transfer function was programmed as: y(t) = 1.5y(t - 1) - 0.7y(t - 2) + u(t - 1) + 0 . 5u(t - 2) + e(t) in which a sample period of T = 0 .1 secs was taken and a unit step input signal was applied. It is evident from Figs. 1 and 2, and from the values given that although the prediction model produces a smoother, less active control Signal, the output regulation of the filter model is better.

Variance y(t)

Fil ter model

0.859

8.613

Prediction model

1.092

10.481

It is worth making two observations. Firstly the results obtained from the state-space filter model are identical to those obtained by means of the polynomial based controller (Wellstead and CoWorkers, 1979). This is not surprising as the control actions are equivalent. Secondly, the trade-off between lower input variance (less active input), but higher output variance, which was most apparent in System 1, has been found in the past (Warwick, 1981) to be the selection criterion between the state-space approaches. However, it was shown by means of Systems 2 and 3 that different system characteristics produce different results and no one state-space method is universally 'better'.

CONCLUSIONS Variance u ( t)

Variance y ( t )

Fil ter model

0.181

10.403

Prediction model

0.169

11.221

Figures 3 and 4 merely illustrate the control effort once more, multiplied by a factor of 4 for greater clarity. Convergence behaviour of the RLS parameter estimates for the filter and prediction models is shown in Figs. 5 and 6 respectively, and it is very difficult to find much difference at all between these plots. System 2: The second simulation was of the same linear system, but this time a coloured noise sequence was considered, with the closed-loop poles being placed at s = - 2 ! j3 in the s-plane, the sampling period T = 0.1 secs and a 0.5 unit step input applied. The only difference between this and the System 1 model described is that

The relationship between state-space based, pole placement self-tuning controllers has been considered, and it has been shown that differences which occur are due directly to the particular state-space formulation of the original ARMA model. The two techniques looked at both involve optimal observers in the feedback path, however only one of these, that based on the state-space filter model, produces a control which is equivalent to a polynomial pole placement controller, as was shown in the previous section. Indeed, a major result highlighted in this paper is the use of the noise-free output measurement, state-space filter model in order to achieve c ontroller equivalence with the polynomial form. The controller equivalence results, obtained by employing the state-space filter model, are in a different vein to the majority of state-space based (ARMA) model reconstructions which employ the state-space prediction (innovations) model.

Filtered and Predicted States for Discrete-time Adaptive Control

133

However, the prediction form, even when an optimal state estimate is employed, does not achieve an equivalent controller to that obtained by the polynomial approach. In fact a non-optimal, state estimate, in the state-space prediction model sense, must be employed in order to obtain controller equivalence. REFERENCES Astr~m,

K. J. (1970) . Introduction to stochastic control . Academic Press, New York. Astr~~ and B. Wittenmark. (1973). On selftuning regulators. Automatica, 9, 185-199. Caines, P.E. (1972). Relationship bet;een BoxJenkins-Astr~m control and Kalman linear regulator. Proc. lEE, 119, 615-620. Farsi, M, K.Z. Karam and K. -warwick. (1984). Simplified recursive identifier for ARMA processes. Electronic Letters, 20, 913-915. Kalman, R.E. (1960). A new approach to linear filtering and prediction problems. Trans. ASME, Ser . D, 82, 35-45. --Lam, K.P. (1980). Implicit and explicit se1ftuning controllers. DPhil thesis, OUEL Report 1334/80, Oxford University. Shah, S.L. and W.R. Cluett. (1988). RLS based estimation schemes for self-tuning control. In K. Warwick (Ed.), Implementation of SelfTuning Controller. Peter Peregrinus Ltd. , 23-40. Warwick, K. (1981). Se lf-tuning regulators - a state-space approach. Int. J. Control, 33, 839-858. Warwick, K.(1987). Optimal observers for ARMA models. Int. J. Control, 46, 1493-1503. Warwick, K. (1988) . On the relationship between

-5 .9

Fig.

Filter model, system 1 output !input

l\ t)1

'\ ,/ I,.) 1, I '/

1.5.9

)

"

I,'

I

\ "I'.

'-,\

It'

\~

",

~I~e.~:::::::::::::::::====:::::==:::::====::::~'ltJ

9 • 9

-5.9

Fig. 2.

Prediction model, system 1 output/input

mInImum variance and minimum quadratic con-

trollers. Proc. Int. Conference Control 88, Oxford, 587-591. Wellstead, P.E . , D. Prager and P. Zanker. (1979). Pole assignment self-tuning regulator. Proc. lEE, 126, 781 - 787

2.5

9. 9

h-IlAfI~----------------

-2.5

Fig. 3 .

Filter model, system 1 input

2.5

,It!

9,e~~

____________________________

-2 .5

Fig. 4. Prediction model, system 1 input

134

K. Warwick and K.

J.

Burnham

Ili~r\.,~-------- ;, Fig. 5.

-2.0

Filter model, parameter estimates (system 1).

Fig. 9.

2.0

Filter model, parameter estimates (system 2)

J"il.-_._-----~-----.----------- i.

11'\\-··,., ,_.-

11{1~'v----'-~_____ 0.0~1_ _ _ _ _ _ _ _ _ _ _ _ _ _ __

.;.,

2 .0

rilt''I,r-----, - - - -. - - - - - - - •.

l

(',0 11 \

I~",. . ~-----------_.

o

_

'I

J---

~_ _ _ _ _ _ _ _ _ _ __

l,r',:(" _ _ _ _ _ _ _ _ _ _ _ _ _ __ i.

o~~~

", \.'Y-""'-...---..----.__-___----- i,

"

-2.0

Fig_ 6.

Prediction model, parameter estimates (system 1)

ll\-------2.0

Fig. 10.

Prediction model, parameter estimates (system 2)

Fig. 11.

Filter model, system 3 output/input

1.:>.11 1.5 . 0

-5.0

Fig. 7.

Filter model, system 2 output/input.

1.5.0

1.:5.0

(i

~,

l ... / \

/'' \1.,/ '-' / ' /

1

,1

/.'

\,

'\

../ .

.(\

.

0'

l

\, l '\

/

\.

\......

-.. \ ",,/ \,1

",,'

\

\

\

,tll

\/

i\ I

0.0~!~'o~1---------------------:5.0

Fig. 8.

Prediction model, system 2 output/input .

-:5 . 0

Fig. 12.

Prediction model, system 3 output/input