- Email: [email protected]
Adaptive Control Based on Instrumental Variable Identification
Cop\rig-ill
©
IF:\( : Idl'llli f'i(atio ll allt! S\~I{'JII ParalIH'I(T
ESlilllalioll I ~I~F), York, l l K, I~H'G
ADAPTIVE CONTROL BASED ON INSTRUMENTAL VARIABLE IDENTIFICATION P. De Larminat IR2 1. 11 rl'. 1 1111' rill !Hllrhhll{ .10/11"' . -1 ·10-11 N'IIIII'S Ch/I'X. Fmllt!'. I'! (;W'T () ({JI)) "Svllr'l1I1'I IIrll//lll/lili". C'.N.R.S.
Abstract. The Instrumental Variable Identification method is revisited, in order to deal with global stability requirement, when inc luded in adaptive control loops. It is shown that an exact symmetrized version can be introduced, permitting to work with over-dimensioned instruments. This over-de termination is entailed by stability r equi rement s , when identifyi ng a closed loop system, us ing delayed input-output signals as instrument. Then, it is shown that the instrumental variable algorithm satisfies some us ual conditions, permitting to garantee the global stability in the indirect adapt i ve control schemes. I - INTRODUCTION The Instrumental Variable (IV) Estimator has been ex tensively used since the bas i c papers of Wong and Polak (1967) and widely studied , particularly by Young (1967, 1979, 1980), Stoica and Soderstrom (1983) . The IV estimator i s some times mentioned, among the other recursive identification methods, when dealing with indir ec t adaptive control. But, to our knowledge, there exists practica lly no work concerning the fundamental problems of convergence, and of local or global stability of adaptive control including I V method . In fact, there exist many studies about the properties of RLS (Recursive Least Square), ELS (Extended Least Squa re), RML (Recursive Maximum Likelyhood) or EKF (Extended Kalman Filter). The r easons are multiple: when dealin g with deterministic case, or with autoregressive, exogeneOuS input model s (ARX) , RLS is obviously sufficient. When an ARMAX model is introduced, the other above methods may be asymptotically optimal . On the other hand, a major interest is the presence in these algorithms of a positive covariance matrix, which can be used when looking for Lyapunov functions for the stability analysis .
and Theil (1958). There are two interests in this int erpretat ion : first, it l eads to symmetric matrices in the algorithms , secondly, 2SLS generalises the IV, permittin g to deal with over-d imensioned instruments. Then, an exact recursive form is given in section Ill. A sufficient stability condition for IV is then introduce d in section IV. Under determini s tic hypoth esis , a complete study is provided, which is based on the proposed symmetri c form . In order to verify this condition, it is shown in section V that, when identifying a n-order ARMAX model, it is necessary to use a 3n vector as instrument. Then, th e above results are applied to the problem of indirect adapt i ve control based on IV estimator. It is shown that if the unbiasedness and stability condit i ons are to be strictly respected, an-order time lag must be introduced in th e control system. In conclusion, th e properties of the propos ed IV version are compared with those of the classical me thod, when used for adaptive control purpose . 11 - NOTATIONS AND DEFINITIONS
Suppose N scala r observation equations, globally written in the form Y
Conversely, the IV method can be criticized. First, the so ca lled "In s trumental Variable" is not a well defined, standard method, but a general principle, from which many different applicationl exist, mainl y in the choice of the instrument : this choic e depends on the context (open or closed loop system), on the stabilit y of the process, on the type of model under cons idera tion (ARMAX or other) . Th ese options depend also on the desire of the designer to approach the statistical optimality, which can lead to introduce an updat ed prefiltering of the data (such as in RML ... ).
y
8
W
8
+
W
( 1)
Nx vector of observations , N x n matr ix, n x vector of unknown parameters, vec tor of noise. Nx
An N x n-matrix Z, ca lled instrumental matrix, is give n. Z is assumed to satisfy : lim.!.. [ZT
}
full rank, lim ~ [ZTwJ
N-+oo N
N-+oo
o
(2)
The IV es timator is defined as The last c ri sticism concerns the non-symmetric form of the recursive matrix equation in the classical IV method, which does not present the interesting property of positivity. Ther e exist some at temps in order to symmetrize the IV, but up to Friedland (1983), the proposed solutions were only more or less goo d approximations (Pandya (1973), Young (1980), Solo (1981» . In thi s paper, an se d. For that, in a particular case (2SLS) es timat or,
exact symmetric version i s proposect i on 11, the IV is viewed as of th e Two Sta ges Le ast Squa r e first introdu ced by 8asmann (1957)
(3)
8
Suppose a new scalar observation
~T 8
y
+ w,
and a new instrumental variabl e z.
Then, the augmented vectors and matrices are : y
+
=
['yi] '
+
[!TJ Z
=
and the new estimator i s
+
(4)
1'. Dc Larminat
600 8+ =
[Z:
J-l ZT Y + +
(5)
The classical IV recursion is 8
+
S+
(.pT
P - p.p(q,Tp
p
8 + S z (q? S z + 1)-1 (y
S - S z
Then, dire ct but tedious expansions show that (17) and (18) are equivalent to the equations
- .pT El)
S z + 1) -1 zT S
(6)
e
(7)
p
in which S-l ~ ZT <1>. S must be noticed to present no nice property, such as symmetry or positive definitness. Moreover, the global form (3), as well as the recursion (6, 7), make sense only for instrumental.vectors z of the same dimension n as .p and 6, but ln some cases, it may be interesting to introduce an instrumental (m x 1) vector, such that m ~ n.
8
O + P(\>(
+ +
Moreover, (8) can be written in the form:
(~T~)-l ~Ty
( 10)
R being the (m x n) regression matrix: R ~ (ZT Z)-l ZT
p - ~(~T~
(22-a)
e+
1) -1 (~_~Te)
(22-b)
p~(~Tp~
Equations (14, 15, 16, 19 to 22) define finally an exact recursive form for IV - 2SLS (see LeyvaMontiel, 1984). The ordinary least square recursion (RLS) can be recognised in (21) and the IV effect results from (22) .
When ~, W, r are stoc~astic, time-invariant proces-
ses, satisfying (2), 8 necessarily converges towards 8 and the stability problem does not arise. But, if IV is used in some context where (2) does not necessarily hold, the convergence is not ensured. gorithm may perties are riori error ..... :t.. and if 8 +-8
be still acceptable if some other proobtained, for instance if the a poste~ + ~ y _.pT§ + is bounded (Lco stability), -.. O.
It is known that RLS satisfies these properties, under
(11)
This later form (9, 10, 11) defines the Two-StageLeast-Square (2SLS) estimator which can be introduced independantly from IV (see Basmann-1957, Theil1958). (12)
the sole hypothesis that w is bounded in (1). In order, to ensure IV stability, introduce the following additional hypothesis : z satisfies y = t
T
z + v , and
.p = TT z + v
(23)
where and T are time-invariant quantities, and where V and v are bounded sequences. From this hypothesis, (weaker than (2», a stability analysis could be performed.
(8) can also be written in the following form, which will be used later : ( 13) it can be The 2SLS presents an obvious advantage implemented using well conditioned numerical methods, such as Householder algorithm.
This paper reduces into a result of deterministic stability : only the ideal case is considered, where w = 0 in (1), v = 0 and v = 0 in (23). Then, it follows easily (see Leyva-Montiel, 1984) 8 is bounded, 8
In the past, some attempts to derive symmetric IV 2SLS recursive forms have been proposed, but up to
y
now, those were only approximations. So, an exact
recursive formulation is proposed in the next section. III - AN EXACT IV - 2SLS RECURSIVE FORM Let's introduce the recursive equations 1 l: - l:z(zTl: z + 1) zTl: l:+ R+ R + l:z(zTl: z + 1)-1 (
(21-b)
If the parameter convergence does not occur, the al-
(9)
where t? Z R
(y- 8 )
IV - STABILITY PROPERTIES (8)
This form holds obviously when (m=n). Then, (ZT
8
(21-a) T-
1) -l ~Tp
Then, (3) can be generalized into :
8 = (TZ(ZTZ)-lZTTZ(ZTZ)-lZTy
-1
( 14) ( 15) (16)
If l: is large, Rand r bounded, R will be close to definition (ll).oThen,odefine P
( (7)
8
( 18)
- .pTe
- 6
where y
+
-.. 0, y
- 8 -..
+
-T-
0
- .p 8+ -..
, 0
-T 6 zT T z r R +' = +
The above properties are the fundamental conclusions of our study. This form is particularly convenient when dealing with global stability of adaptive control systems. (e.g. De Larminat-1982). Recall that they are obtained from noise-free hypothesis, but without boundedness, neither full rank assumption. On the other hand, an important condition was introduced : existence of the matrices T and t in equation (23) . Obviously, this condition will play a major role when defining the instrumental vector z, and it could impose a dimension of z greater than of <1>. Then, the interest of IV - 2SLS form bpcomes fundamental when dealing with vectors z and
y*
6
T
=r z
(19) (20)
Suppose a process, inpu~ u k ' output Yk' disturbed by a sequence e , descrlbea by the followlng equation :
k
601
Adaptive Control +e +c e _ +·· .+cne _ k 1 k 1 k n
(24)
(24) can also be written as A(q
-1
(25)
) Yk
where A, B, C are the classical polynomials in the shift operator q-l. The transfer function B/A is not necessarily stable, but is is assumed that I/C is asymptotically stable.
e
The parameter vector
is to be identified (26)
chastic case, which yields E(y -z~tk)2 minimum. If a priori knowledges about C ar~ ava~lable, they are to be used. If not, our point of view is that possible refinements permitting to identify Care not to be recommended, because they are not necessary in order to preserve the unbiasedness,and because they risk to introduce some parametric feedback, which are unstability causes. Then, we consider here K-I(q-I) as an arbitrary design factor, not necessarily equal to C-I(q-l). Z being defined by (33), then we study the IV 2§LS algorithm in the context of an adaptive control of the process.
VI - IMPLEMENTATION OF AN ADAPTIVE CONTROL USING THE IV ESTIMATES (27)
w ~ e + c e _ + ... + cne _ k k 1 k 1 k n
(28)
Then (24) becomes
"'~ e +
Yk :
(i) Instrument based on an auxiliary model Define k : uk ' and
z~ ~
i
[]k-l'
"Yk - n
: uk-1" .uk_J
(30)
J
u k - n - 1 •• .uk - 2
-I -I )u k ' ~k = K (q )Yk
uk=-Pluk-l ... -Pnuk-n+qoYk+qlYk-l+' .. +
(32)
Then, define :
qn-1 Yk-n-l Or, with vectorial notations (38)
In order to restore this independence, a first sug gestion is to replace (38) by
Where +* and y* could be provided by the identification algorithm itself (see 19). We reject this solution because it introduces in fact the parameters Rand r in the control law. Moreover, these parameters depends on the coefficients fk of the control law, thus this solution introduces a feedback loop on the parameters, which is not to be recommended. A second solution consists in defining the predictors of '" and y from the given pOlynomial ~ and _ from the unbiased identified coefficients A and B (see Appendix). Now, the block diagram of the whole control law can be represented (Fig.l) : r"------, u PROCESS
~k-n-l"'Z;k-2n uk_I .•. uk_nl
(33)
Now, the dimension of zk is 3n, but it can be easily shown, from appendix, that in the deterministic case, there exist a vector t and a matrix T satisfying the equa1ities T Yk = zK t ,
T
tk
=
T zk T
(37)
(39)
From the two above fundamental options, many variantes and extensions exist. Some of those permit to identify the polynomial C(q-I), or even a more general model (such as y = (B/A)u + (C/D)e). Others are designed in order to optimize the identification procedure. Having in mind these previous solutions, we propose here an instrument adapted to the closed loop identification problem, and capable to satisfy the above stability condition «32). From an arbitrarily given stable transfer function K-l (q-l), the following signals are defined -I
More precisely
(31-b)
(31-b) is convenient for closed loop identification, provided that the controller order, and also the order of C(q-l) be lower than n.
-1
D beeing any given Hurwitz polynomial. Then, the following control law is applied
It is clear that the independance of zk and w is k not longer verified when using (38)
(31-a)
uk - 1 ·· .uk-J or :
(35)
(36)
(ii) Instruments based on delayed observations, such that :
= K (q
--I
S/A
The independance of zk and w is met, in open loop, if u and w are independant~ without any hypothek k sis o~eE the order of C(q-l). If B/A is unstable, then B/A cannot approximate B/A, and in closep loop, zk and w are not independent. Then, the above zk (eq.30) k~s not convenient here.
~k
-I
(29)
wk
There exists classical approaches in order to derive some instrumental vectors zk'
Y
-
Suppose that a model ~_(q ), B(q ) is recursively identified. Then, t~e classical approach of the indirect adaptive control, via pole placement, consist into solving at each time k the diophantine equation :
(34)
which was not possible with instruments (30 or 31). See also Young (1979), which does similar analysis. On the other hand, the choice of K(q-l) permits to optimize the method, by setting K = C in the sto-
(see appendix) n step ahead I •. Id
PREDICTORS (H, F, G) ahead I. ·11
P. De Larminat
50';!
It is clear that the whole feedback uly is equivalent to a transfer function such as M(q
-1
)u
-n-l
k
= q N(q
(40)
)Yk
Where M and N are of order 2n-l and n-l respective ly. The factor q-n results from the presence of q-n in : (q-n/K)Yk
f;k-n
On the other hand, the characteristic polynomial of the whole closed loop system is given by the product of D(q-l)(desired) by K(q-l) (introduced by the predictors). Then, the third and finally retained solution consists in determining directly the polynomials M a~d N, defining the global feedback (40), by solvlng the equation :
AM
+
- -n
Bq
N = DK
(41)
Under this last form (40), it is clear that using IV identification yields necessarily a loss of performances, since a time lag must be introduced in the feedback control. That is the price which is to he paid if one want benefit of the advantages of the IV method presented here : - unbiased estimates in the stochastic case stability properties proved in the deterministic case,(conjectured in the stochastic case) due to an over-dimensioned instrument.
'
An other point concerns the dimension (3n) of the instrument : a natural suggestion could be derived from the classical choice (31-b), which preserve the independence condition
In the case of time invariant control, T
u k = f ~k + qoYk
(43)
The existence of t and T would be restored. Thinking now in terms of adaptive control, it appears that if convergence occurs, then (43) asymptotically holds, from (38). However, this solution introduces again a parametric feedback and then a risk of unstability. Thus, it seems finally that an over dimensioned vector is really a necessity when using IV for adaptive control. (In relation it is to be mentioned that all the other classic;l methods (ELS, RML, EKF) are also, at least, 3n dimensioned). Then, it was necessary to introduce an IV recursive version, able to work with overdimensioned vector. And, when dealing with a sensitive problem, such as adaptive control, an exact solution is highly preferable to any approximate methods, even if computationally simpler. VII - CONCLUSION The goal of this paper was to define how to implement the IV algorithm in the case of adaptive control. A preliminary question is : why IV would be prefered to other classical methods ? We will not risk a definitive response, only some arguments : in the general case, RLS is to be rejected because of its bias (~xcept in some specific cases, such as when used
1n
connection with minimal variance
control). Concerning ELS, it is known that a positivity condition over C(q-l) must be satisfied. On the other hand, RML and EKF are based on a simultaneous identification of a . , b . , and c .• Then, the data u and Yk are filter~d b9 l/C, ekplicitely in k RML, implicitely in ELS and EKF. The convergence problems of ELS results from this parametric loop. An other inconvenient is that filtering by C- 1 makes more difficult the stability analysis of the adaptive systems based on this method, and a moni-
toring of C is sometimes necessary in order to ensure the stability. An other major objection to the methods based on the simultaneous estimation of C lies in the principle itself of modelling the processes by the ARMAX model. This modelling is commonly accepted, but is often rather inadequate : ~he non-determinism of the noise does not implies lpSO facto that a rational spectrum model be adequate. On the other hand, it is commonly recognized that the identification of C is generally difficult : unfo~tunately, noises are generally large enough to blase a RLS identification, but too small for a correct identification of C. Then, it is obviously not recommended that the identification of C be strongly integrated in the method as in ELS RML or EKF. " For this reason, we do not follow the authors which are looking for IV refinements, in order to identify C : in fact, it is just the main interest of IV, to be able to provide unbiased estimates of A and B, without estimating C ! ~onversely, it is interesting to preserve lmp~ovements by using a priori knowledge,
possible if given. I~ lS the reason why we introduced the filters K l(q-l) in the method. REFERENCES Basmann R.L. (1957) "A generalized classical method of linear estimation of coefficients in a structural equation" Econometrica, 25, 77 Bierman (1977) "Factorization methods for discrete sequential estimation", Academic Press, N.Y. De Larminat P. (1982) "On the stabilizability condition in indirect adaptive control", 6th IFAC symposium on identification and system parameter estlmatlon. Washlngton, June 1982 Dhrymes P.J. (1970) Econometrics-statistical foundations and applications, New-York, Harper and Rowe Hs ia T. C. (1981) "A two stage least squares procedure for system identification", IEEE trans .A.C., vol. AC-26 Isermann R. and Baur U. (1974) Comparison of six on-line identification and parameter estimation methods, Automatica, vol.l0, pp. 69-79, Jan. Leyva-Montiel J.L. and De Larminat Ph. (1984) "Une nouvelle formulation recursive de l'identification par variable instrumentale" rapport technique LR2I, Fevrier 1984 ' Landau 1.0. (1976) "Unbiased recursive identification using model reference adaptive techniques", IEEE trans.A.C. AC-21, 194 Mayne D.Q. (1967) "A method for estimating discrete time transfer functions", IEEE, conference on computers in control, University of Bristol, paper n"C-2 Mendel J.M. (1975) Multi-stage least squares parameter estimators IEEE, trans.Aut.Cont . AC-20 n06 Pandya R. N. and Pagurek ~. (1973) IIIdentification and system parameter estimation"
edited by P. Eykhoff (New-York: American Elsevier; Amsterdam: North Holland), pp. 701 710 Solo v. (1981) "The convergence of an instrumental-variablelike recursion", Automatica, vol.17, n03, pp.
545-547, 1981 Theil H. (1958) "Economic forecaste and policy", (Amsterdam
North-Holland publishing CO.)
Adaptive Control Wong K.Y. and Polak E. (1967) "Identification of linear discrete time systems
using the instrumental variable method", IEEE Trans., AC-12, 707 Young P.C. (1967) "An instrumental variable method for real time identification of noisy processes", Fourth IFAC Congress, Warsaw, paper n026.6 . Young P.C. (1979) "Refined instrumental variable methods of recursive time-ser1cs analysis, part I, single input,
single output systems", Int.J.Control, vol.29, N°1, 1-30. Young P.C. (1979) "Self Adaptive Kalman Filter", Electronic Letters, 15, 358-360 Young P. and Jakeman A. (1980) "Refined instrumental variable methods of recursive time-series analysis, part Ill, Extensions" Int.J.Control, vol.31, N°4, 741-764 APPENDIX - d-step ahead predictors Write the process equation (24) as : Aon Y = B1n u + Cone Where the first index is for the first non nul coefficient of the polynomials, and the second index is for the order. Recall the filtered data (32)
r;=if- F,=-ion
on
We look for a d-step ahead predictor, based on the input data, until present time k, and on the output data, only until k-d-1 : Yk = h 1u k _ 1+·· .+hduk_d+f1r;k_d_1+···+fn1;k_d_n +glF,k-d-1+" ·+gnF,k-d-n Using the above notation, it follows -
y = H
1d u + q
-d F 1n
~ u
+ q
-d G1n K Y
on
on
optimality will be reached if the prediction error Yk is only function of e - ... e - (unprek 1 k d k dictable from the output known until k-d-1) :
Y
Thus the following equality must hold -d F 1n -d G1n y = H1d u + q ~ u + q ~ y + Lode on
on
which leads to (K
It
1S
on
- G q 1n
-d
)y
clear that this equality hold if :
K L
on on
For that, set first : K = C on on
Then, solve with respect to Land G the diophantine equation
A L
on od
+
q-d G = K 1n on
Then, solve with respect to Hand F K H on 1n
+
q-d F
1n
= B L
1n od
603
For the given polynomials order, the solution always exists. If C is not given, take any stable K . The predicto~nis not longer optimal : but b8nnded e leads to bounded prediction error, and in the deterministic case, the prediction error is
asymptotically zero. From the existence of the above predictors, existence of t and T for the proposed instrument (32), (33) is straightforward.
©
IF:\( : Idl'llli f'i(atio ll allt! S\~I{'JII ParalIH'I(T
ESlilllalioll I ~I~F), York, l l K, I~H'G
ADAPTIVE CONTROL BASED ON INSTRUMENTAL VARIABLE IDENTIFICATION P. De Larminat IR2 1. 11 rl'. 1 1111' rill !Hllrhhll{ .10/11"' . -1 ·10-11 N'IIIII'S Ch/I'X. Fmllt!'. I'! (;W'T () ({JI)) "Svllr'l1I1'I IIrll//lll/lili". C'.N.R.S.
Abstract. The Instrumental Variable Identification method is revisited, in order to deal with global stability requirement, when inc luded in adaptive control loops. It is shown that an exact symmetrized version can be introduced, permitting to work with over-dimensioned instruments. This over-de termination is entailed by stability r equi rement s , when identifyi ng a closed loop system, us ing delayed input-output signals as instrument. Then, it is shown that the instrumental variable algorithm satisfies some us ual conditions, permitting to garantee the global stability in the indirect adapt i ve control schemes. I - INTRODUCTION The Instrumental Variable (IV) Estimator has been ex tensively used since the bas i c papers of Wong and Polak (1967) and widely studied , particularly by Young (1967, 1979, 1980), Stoica and Soderstrom (1983) . The IV estimator i s some times mentioned, among the other recursive identification methods, when dealing with indir ec t adaptive control. But, to our knowledge, there exists practica lly no work concerning the fundamental problems of convergence, and of local or global stability of adaptive control including I V method . In fact, there exist many studies about the properties of RLS (Recursive Least Square), ELS (Extended Least Squa re), RML (Recursive Maximum Likelyhood) or EKF (Extended Kalman Filter). The r easons are multiple: when dealin g with deterministic case, or with autoregressive, exogeneOuS input model s (ARX) , RLS is obviously sufficient. When an ARMAX model is introduced, the other above methods may be asymptotically optimal . On the other hand, a major interest is the presence in these algorithms of a positive covariance matrix, which can be used when looking for Lyapunov functions for the stability analysis .
and Theil (1958). There are two interests in this int erpretat ion : first, it l eads to symmetric matrices in the algorithms , secondly, 2SLS generalises the IV, permittin g to deal with over-d imensioned instruments. Then, an exact recursive form is given in section Ill. A sufficient stability condition for IV is then introduce d in section IV. Under determini s tic hypoth esis , a complete study is provided, which is based on the proposed symmetri c form . In order to verify this condition, it is shown in section V that, when identifying a n-order ARMAX model, it is necessary to use a 3n vector as instrument. Then, th e above results are applied to the problem of indirect adapt i ve control based on IV estimator. It is shown that if the unbiasedness and stability condit i ons are to be strictly respected, an-order time lag must be introduced in th e control system. In conclusion, th e properties of the propos ed IV version are compared with those of the classical me thod, when used for adaptive control purpose . 11 - NOTATIONS AND DEFINITIONS
Suppose N scala r observation equations, globally written in the form Y
Conversely, the IV method can be criticized. First, the so ca lled "In s trumental Variable" is not a well defined, standard method, but a general principle, from which many different applicationl exist, mainl y in the choice of the instrument : this choic e depends on the context (open or closed loop system), on the stabilit y of the process, on the type of model under cons idera tion (ARMAX or other) . Th ese options depend also on the desire of the designer to approach the statistical optimality, which can lead to introduce an updat ed prefiltering of the data (such as in RML ... ).
y
8
W
8
+
W
( 1)
Nx vector of observations , N x n matr ix, n x vector of unknown parameters, vec tor of noise. Nx
An N x n-matrix Z, ca lled instrumental matrix, is give n. Z is assumed to satisfy : lim.!.. [ZT
}
full rank, lim ~ [ZTwJ
N-+oo N
N-+oo
o
(2)
The IV es timator is defined as The last c ri sticism concerns the non-symmetric form of the recursive matrix equation in the classical IV method, which does not present the interesting property of positivity. Ther e exist some at temps in order to symmetrize the IV, but up to Friedland (1983), the proposed solutions were only more or less goo d approximations (Pandya (1973), Young (1980), Solo (1981» . In thi s paper, an se d. For that, in a particular case (2SLS) es timat or,
exact symmetric version i s proposect i on 11, the IV is viewed as of th e Two Sta ges Le ast Squa r e first introdu ced by 8asmann (1957)
(3)
8
Suppose a new scalar observation
~T 8
y
+ w,
and a new instrumental variabl e z.
Then, the augmented vectors and matrices are : y
+
=
['yi] '
+
[!TJ Z
=
and the new estimator i s
+
(4)
1'. Dc Larminat
600 8+ =
[Z:
(5)
The classical IV recursion is 8
+
S+
(.pT
P - p.p(q,Tp
p
8 + S z (q? S z + 1)-1 (y
S - S z
Then, dire ct but tedious expansions show that (17) and (18) are equivalent to the equations
- .pT El)
S z + 1) -1 zT S
(6)
e
(7)
p
in which S-l ~ ZT <1>. S must be noticed to present no nice property, such as symmetry or positive definitness. Moreover, the global form (3), as well as the recursion (6, 7), make sense only for instrumental.vectors z of the same dimension n as .p and 6, but ln some cases, it may be interesting to introduce an instrumental (m x 1) vector, such that m ~ n.
8
O + P(\>(
+ +
Moreover, (8) can be written in the form:
(~T~)-l ~Ty
( 10)
R being the (m x n) regression matrix: R ~ (ZT Z)-l ZT
p - ~(~T~
(22-a)
e+
1) -1 (~_~Te)
(22-b)
p~(~Tp~
Equations (14, 15, 16, 19 to 22) define finally an exact recursive form for IV - 2SLS (see LeyvaMontiel, 1984). The ordinary least square recursion (RLS) can be recognised in (21) and the IV effect results from (22) .
When ~, W, r are stoc~astic, time-invariant proces-
ses, satisfying (2), 8 necessarily converges towards 8 and the stability problem does not arise. But, if IV is used in some context where (2) does not necessarily hold, the convergence is not ensured. gorithm may perties are riori error ..... :t.. and if 8 +-8
be still acceptable if some other proobtained, for instance if the a poste~ + ~ y _.pT§ + is bounded (Lco stability), -.. O.
It is known that RLS satisfies these properties, under
(11)
This later form (9, 10, 11) defines the Two-StageLeast-Square (2SLS) estimator which can be introduced independantly from IV (see Basmann-1957, Theil1958). (12)
the sole hypothesis that w is bounded in (1). In order, to ensure IV stability, introduce the following additional hypothesis : z satisfies y = t
T
z + v , and
.p = TT z + v
(23)
where and T are time-invariant quantities, and where V and v are bounded sequences. From this hypothesis, (weaker than (2», a stability analysis could be performed.
(8) can also be written in the following form, which will be used later : ( 13) it can be The 2SLS presents an obvious advantage implemented using well conditioned numerical methods, such as Householder algorithm.
This paper reduces into a result of deterministic stability : only the ideal case is considered, where w = 0 in (1), v = 0 and v = 0 in (23). Then, it follows easily (see Leyva-Montiel, 1984) 8 is bounded, 8
In the past, some attempts to derive symmetric IV 2SLS recursive forms have been proposed, but up to
y
now, those were only approximations. So, an exact
recursive formulation is proposed in the next section. III - AN EXACT IV - 2SLS RECURSIVE FORM Let's introduce the recursive equations 1 l: - l:z(zTl: z + 1) zTl: l:+ R+ R + l:z(zTl: z + 1)-1 (
(21-b)
If the parameter convergence does not occur, the al-
(9)
where t? Z R
(y-
IV - STABILITY PROPERTIES (8)
This form holds obviously when (m=n). Then, (ZT
8
(21-a) T-
1) -l ~Tp
Then, (3) can be generalized into :
8 = (TZ(ZTZ)-lZT
-1
( 14) ( 15) (16)
If l: is large, Rand r bounded, R will be close to definition (ll).oThen,odefine P
( (7)
8
( 18)
- .pTe
- 6
where y
+
-.. 0, y
- 8 -..
+
-T-
0
- .p 8+ -..
, 0
-T 6 zT T z r R +' = +
The above properties are the fundamental conclusions of our study. This form is particularly convenient when dealing with global stability of adaptive control systems. (e.g. De Larminat-1982). Recall that they are obtained from noise-free hypothesis, but without boundedness, neither full rank assumption. On the other hand, an important condition was introduced : existence of the matrices T and t in equation (23) . Obviously, this condition will play a major role when defining the instrumental vector z, and it could impose a dimension of z greater than of <1>. Then, the interest of IV - 2SLS form bpcomes fundamental when dealing with vectors z and
y*
6
T
=r z
(19) (20)
Suppose a process, inpu~ u k ' output Yk' disturbed by a sequence e , descrlbea by the followlng equation :
k
601
Adaptive Control +e +c e _ +·· .+cne _ k 1 k 1 k n
(24)
(24) can also be written as A(q
-1
(25)
) Yk
where A, B, C are the classical polynomials in the shift operator q-l. The transfer function B/A is not necessarily stable, but is is assumed that I/C is asymptotically stable.
e
The parameter vector
is to be identified (26)
chastic case, which yields E(y -z~tk)2 minimum. If a priori knowledges about C ar~ ava~lable, they are to be used. If not, our point of view is that possible refinements permitting to identify Care not to be recommended, because they are not necessary in order to preserve the unbiasedness,and because they risk to introduce some parametric feedback, which are unstability causes. Then, we consider here K-I(q-I) as an arbitrary design factor, not necessarily equal to C-I(q-l). Z being defined by (33), then we study the IV 2§LS algorithm in the context of an adaptive control of the process.
VI - IMPLEMENTATION OF AN ADAPTIVE CONTROL USING THE IV ESTIMATES (27)
w ~ e + c e _ + ... + cne _ k k 1 k 1 k n
(28)
Then (24) becomes
"'~ e +
Yk :
(i) Instrument based on an auxiliary model Define k : uk ' and
z~ ~
i
[]k-l'
"Yk - n
: uk-1" .uk_J
(30)
J
u k - n - 1 •• .uk - 2
-I -I )u k ' ~k = K (q )Yk
uk=-Pluk-l ... -Pnuk-n+qoYk+qlYk-l+' .. +
(32)
Then, define :
qn-1 Yk-n-l Or, with vectorial notations (38)
In order to restore this independence, a first sug gestion is to replace (38) by
Where +* and y* could be provided by the identification algorithm itself (see 19). We reject this solution because it introduces in fact the parameters Rand r in the control law. Moreover, these parameters depends on the coefficients fk of the control law, thus this solution introduces a feedback loop on the parameters, which is not to be recommended. A second solution consists in defining the predictors of '" and y from the given pOlynomial ~ and _ from the unbiased identified coefficients A and B (see Appendix). Now, the block diagram of the whole control law can be represented (Fig.l) : r"------, u PROCESS
~k-n-l"'Z;k-2n uk_I .•. uk_nl
(33)
Now, the dimension of zk is 3n, but it can be easily shown, from appendix, that in the deterministic case, there exist a vector t and a matrix T satisfying the equa1ities T Yk = zK t ,
T
tk
=
T zk T
(37)
(39)
From the two above fundamental options, many variantes and extensions exist. Some of those permit to identify the polynomial C(q-I), or even a more general model (such as y = (B/A)u + (C/D)e). Others are designed in order to optimize the identification procedure. Having in mind these previous solutions, we propose here an instrument adapted to the closed loop identification problem, and capable to satisfy the above stability condition «32). From an arbitrarily given stable transfer function K-l (q-l), the following signals are defined -I
More precisely
(31-b)
(31-b) is convenient for closed loop identification, provided that the controller order, and also the order of C(q-l) be lower than n.
-1
D beeing any given Hurwitz polynomial. Then, the following control law is applied
It is clear that the independance of zk and w is k not longer verified when using (38)
(31-a)
uk - 1 ·· .uk-J or :
(35)
(36)
(ii) Instruments based on delayed observations, such that :
= K (q
--I
S/A
The independance of zk and w is met, in open loop, if u and w are independant~ without any hypothek k sis o~eE the order of C(q-l). If B/A is unstable, then B/A cannot approximate B/A, and in closep loop, zk and w are not independent. Then, the above zk (eq.30) k~s not convenient here.
~k
-I
(29)
wk
There exists classical approaches in order to derive some instrumental vectors zk'
Y
-
Suppose that a model ~_(q ), B(q ) is recursively identified. Then, t~e classical approach of the indirect adaptive control, via pole placement, consist into solving at each time k the diophantine equation :
(34)
which was not possible with instruments (30 or 31). See also Young (1979), which does similar analysis. On the other hand, the choice of K(q-l) permits to optimize the method, by setting K = C in the sto-
(see appendix) n step ahead I •. Id
PREDICTORS (H, F, G) ahead I. ·11
P. De Larminat
50';!
It is clear that the whole feedback uly is equivalent to a transfer function such as M(q
-1
)u
-n-l
k
= q N(q
(40)
)Yk
Where M and N are of order 2n-l and n-l respective ly. The factor q-n results from the presence of q-n in : (q-n/K)Yk
f;k-n
On the other hand, the characteristic polynomial of the whole closed loop system is given by the product of D(q-l)(desired) by K(q-l) (introduced by the predictors). Then, the third and finally retained solution consists in determining directly the polynomials M a~d N, defining the global feedback (40), by solvlng the equation :
AM
+
- -n
Bq
N = DK
(41)
Under this last form (40), it is clear that using IV identification yields necessarily a loss of performances, since a time lag must be introduced in the feedback control. That is the price which is to he paid if one want benefit of the advantages of the IV method presented here : - unbiased estimates in the stochastic case stability properties proved in the deterministic case,(conjectured in the stochastic case) due to an over-dimensioned instrument.
'
An other point concerns the dimension (3n) of the instrument : a natural suggestion could be derived from the classical choice (31-b), which preserve the independence condition
In the case of time invariant control, T
u k = f ~k + qoYk
(43)
The existence of t and T would be restored. Thinking now in terms of adaptive control, it appears that if convergence occurs, then (43) asymptotically holds, from (38). However, this solution introduces again a parametric feedback and then a risk of unstability. Thus, it seems finally that an over dimensioned vector is really a necessity when using IV for adaptive control. (In relation it is to be mentioned that all the other classic;l methods (ELS, RML, EKF) are also, at least, 3n dimensioned). Then, it was necessary to introduce an IV recursive version, able to work with overdimensioned vector. And, when dealing with a sensitive problem, such as adaptive control, an exact solution is highly preferable to any approximate methods, even if computationally simpler. VII - CONCLUSION The goal of this paper was to define how to implement the IV algorithm in the case of adaptive control. A preliminary question is : why IV would be prefered to other classical methods ? We will not risk a definitive response, only some arguments : in the general case, RLS is to be rejected because of its bias (~xcept in some specific cases, such as when used
1n
connection with minimal variance
control). Concerning ELS, it is known that a positivity condition over C(q-l) must be satisfied. On the other hand, RML and EKF are based on a simultaneous identification of a . , b . , and c .• Then, the data u and Yk are filter~d b9 l/C, ekplicitely in k RML, implicitely in ELS and EKF. The convergence problems of ELS results from this parametric loop. An other inconvenient is that filtering by C- 1 makes more difficult the stability analysis of the adaptive systems based on this method, and a moni-
toring of C is sometimes necessary in order to ensure the stability. An other major objection to the methods based on the simultaneous estimation of C lies in the principle itself of modelling the processes by the ARMAX model. This modelling is commonly accepted, but is often rather inadequate : ~he non-determinism of the noise does not implies lpSO facto that a rational spectrum model be adequate. On the other hand, it is commonly recognized that the identification of C is generally difficult : unfo~tunately, noises are generally large enough to blase a RLS identification, but too small for a correct identification of C. Then, it is obviously not recommended that the identification of C be strongly integrated in the method as in ELS RML or EKF. " For this reason, we do not follow the authors which are looking for IV refinements, in order to identify C : in fact, it is just the main interest of IV, to be able to provide unbiased estimates of A and B, without estimating C ! ~onversely, it is interesting to preserve lmp~ovements by using a priori knowledge,
possible if given. I~ lS the reason why we introduced the filters K l(q-l) in the method. REFERENCES Basmann R.L. (1957) "A generalized classical method of linear estimation of coefficients in a structural equation" Econometrica, 25, 77 Bierman (1977) "Factorization methods for discrete sequential estimation", Academic Press, N.Y. De Larminat P. (1982) "On the stabilizability condition in indirect adaptive control", 6th IFAC symposium on identification and system parameter estlmatlon. Washlngton, June 1982 Dhrymes P.J. (1970) Econometrics-statistical foundations and applications, New-York, Harper and Rowe Hs ia T. C. (1981) "A two stage least squares procedure for system identification", IEEE trans .A.C., vol. AC-26 Isermann R. and Baur U. (1974) Comparison of six on-line identification and parameter estimation methods, Automatica, vol.l0, pp. 69-79, Jan. Leyva-Montiel J.L. and De Larminat Ph. (1984) "Une nouvelle formulation recursive de l'identification par variable instrumentale" rapport technique LR2I, Fevrier 1984 ' Landau 1.0. (1976) "Unbiased recursive identification using model reference adaptive techniques", IEEE trans.A.C. AC-21, 194 Mayne D.Q. (1967) "A method for estimating discrete time transfer functions", IEEE, conference on computers in control, University of Bristol, paper n"C-2 Mendel J.M. (1975) Multi-stage least squares parameter estimators IEEE, trans.Aut.Cont . AC-20 n06 Pandya R. N. and Pagurek ~. (1973) IIIdentification and system parameter estimation"
edited by P. Eykhoff (New-York: American Elsevier; Amsterdam: North Holland), pp. 701 710 Solo v. (1981) "The convergence of an instrumental-variablelike recursion", Automatica, vol.17, n03, pp.
545-547, 1981 Theil H. (1958) "Economic forecaste and policy", (Amsterdam
North-Holland publishing CO.)
Adaptive Control Wong K.Y. and Polak E. (1967) "Identification of linear discrete time systems
using the instrumental variable method", IEEE Trans., AC-12, 707 Young P.C. (1967) "An instrumental variable method for real time identification of noisy processes", Fourth IFAC Congress, Warsaw, paper n026.6 . Young P.C. (1979) "Refined instrumental variable methods of recursive time-ser1cs analysis, part I, single input,
single output systems", Int.J.Control, vol.29, N°1, 1-30. Young P.C. (1979) "Self Adaptive Kalman Filter", Electronic Letters, 15, 358-360 Young P. and Jakeman A. (1980) "Refined instrumental variable methods of recursive time-series analysis, part Ill, Extensions" Int.J.Control, vol.31, N°4, 741-764 APPENDIX - d-step ahead predictors Write the process equation (24) as : Aon Y = B1n u + Cone Where the first index is for the first non nul coefficient of the polynomials, and the second index is for the order. Recall the filtered data (32)
r;=if- F,=-ion
on
We look for a d-step ahead predictor, based on the input data, until present time k, and on the output data, only until k-d-1 : Yk = h 1u k _ 1+·· .+hduk_d+f1r;k_d_1+···+fn1;k_d_n +glF,k-d-1+" ·+gnF,k-d-n Using the above notation, it follows -
y = H
1d u + q
-d F 1n
~ u
+ q
-d G1n K Y
on
on
optimality will be reached if the prediction error Yk is only function of e - ... e - (unprek 1 k d k dictable from the output known until k-d-1) :
Y
Thus the following equality must hold -d F 1n -d G1n y = H1d u + q ~ u + q ~ y + Lode on
on
which leads to (K
It
1S
on
- G q 1n
-d
)y
clear that this equality hold if :
K L
on on
For that, set first : K = C on on
Then, solve with respect to Land G the diophantine equation
A L
on od
+
q-d G = K 1n on
Then, solve with respect to Hand F K H on 1n
+
q-d F
1n
= B L
1n od
603
For the given polynomials order, the solution always exists. If C is not given, take any stable K . The predicto~nis not longer optimal : but b8nnded e leads to bounded prediction error, and in the deterministic case, the prediction error is
asymptotically zero. From the existence of the above predictors, existence of t and T for the proposed instrument (32), (33) is straightforward.