- Email: [email protected]
Unbiased parameter estimation of linear systems with colored noises
Pergamon
0
PII: sooo5-1098(%)00246-4
Auromorica, Vol. 33, No. 5, pp. %9-913, 1997 1997 Elsevier Science Ltd. All rights reserved Printed in Great Britain Kxls-1098/97 $17.00 + 0.00
Brief Paper
Unbiased
Parameter
Estimation of Linear Systems with Colored Noises*
YING ZHANGt Key Words-Least-squares identification.
method;
and CHUN-BO
consistency:
Abstract-We report a study of the problem of consistent parameter estimation in the general case where the measurement noise is unknown and correlated not only with the system output but also with the system input. A new method is presented for eliminating the noise-induced biases in the least-squares (LS) estimates. In the proposed method, two filters are introduced to filter the system input and output signals, respectively, so that an augmented system with some known zeros and poles is obtained. By shifting the space of the parameters to be estimated and using the known zeros and poles of the parameters to be estimated and using the known zeros and poles, a sufficient number of linear constraints are constructed to determine the noise-induced biases. After eliminating the biases in the ordinary LS consistent estimates of the parameters are estimates, obtained. Analysis shows that the results of the proposed method are independent of the noise model used. 0 1997 Elsevier Science Ltd.
parameter
FENGS
estimation;
colored
noise;
system
restrictions on the noise model. However, at present there is no efficient way of constructing a suitable instrumental variable for a given case of identification. It is noticeable that there are many systems in use, such as the closed-loop systems and the error-in-variables (EV) systems, in which the noise is correlated not only with the system output but also with the system input. In such cases direct application of the methods developed for open-loop systems will usually give unsatisfactory results. In the existing methods for those kinds of system, it is still necessary that the noise be modeled accurately (see e.g. Gustavsson et al., 1977; Soderstiim, 1981; Anderson, 1985; Zhang and Feng, 1992). Feng and Zheng (1991) and Feng and Zhang (1995) have proposed a bias-eliminating least-squares (BELS) method for treating the bias problem without modeling the noise. However, it is assumed in the BELS method that the noise at the output terminal should not depend statistically upon the system input signal. Without this assumption it is difficult to construct a sufficient number of linear equations for determining the noise-induced biases. Therefore the BELS method cannot be applicable to the general case where the system input and output are both corrupted with colored noise. However, the idea of the BELS method is of potential in treating the noise-induced biases. In this paper we extend the idea of the BELS method to the identification of svstems , in the more general case. The essence of this paper is to construct a sufficient number of linear equations to determine the unknown noise-induced biases in the ordinary LS estimates. To this end, two digital filters are inserted into the system at the input and output terminals so that the augmented system will have some known zeros and poles, which can be used to construct a batch of linear equalities about the biases. Using the information obtained from these known zeros and poles, another batch of linear equalities can also be constructed in a new parameter space. Then the noise-induced biases in the LS estimates can be determined from these two batches of equalities. Finally, with the biases removed, consistent estimates of the parameters are obtained.
1. Introduction The main problem in doing parameter estimation using the least-squares (LS) methods is how to achieve consistent estimation in the presence of noise. In the past decades much effort has been devoted to this problem. Various modified least-squares methods have been developed for the identification of open-loop systems (see e.g. Ljung, 1987; Siiderstrom and Stoica, 1989). In these methods the parameters of the noise model and the system parameters are estimated simultaneously. Thus the results of these methods are inevitably dependent upon an accurate model of the noise. In addition, certain strictly positive-real (SPR) conditions on the noise model must be satisfied in these methods in order to obtain consistent parameter estimates. Detailed studies on the performance of these methods have been reported by Soderstrom et al. (1978) Ljung (1987) and Ljung and Siiderstbm (1983). In practical circumstances, it is usually difficult to model the noise exactly. It is difficult to know beforehand whether or not the SPR conditions on the noise model are satisfied. Fortunately, advances have been made on techniques which can overcome the above-mentioned limitations (see e.g. Solo, 1984, Guo ef al., 1988; Moore et al., 1990). A promising method is the instrumental variables (IV) method originally proposed by Siiderstriim and Stoica (1983). In principle, the results of this method should not depend upon any
2. Problem statement ZN’ Assume that the observation sequence {u(k), y(k)}&‘=, is generated from a linear system which can be modeled by A(r--‘)y(k)
* Received 24 May 1995; revised 29 April 1996; received in final form 6 November 1996. This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor J. Bokor under the direction of Editor Torsten Sederstrom. Corresponding author Dr Ying Zhang. Tel. +657991366: Fax +65 7912687; E-mail [email protected]. t School of Electrical & Electronic Engineering, Nanyang Technological University, Nanyang Avenue, Singapore 639798. $ Research Institute of Automation, Southeast University, Nanjing, 210096, P. R. China.
= B(z-r)u(k)
+ u(k),
(1)
where A(z-‘)=l+a,z~‘+...+a,,,z”.,
(2)
B(zr’)=b,,+b,z-‘+...+b,,$‘*,
(3)
and y(k) and u(k) are the system input and output, respectively. u(k) is a colored noise of an unspecified character, which may be correlated with the sytem input and the system output. Denote the parameter vector f3= by [a,, a?, . , a,,,,;b,,. b,, , b,J. Then the identification of 969
970
Brief Papers
the system is to estimate the parameter vector 8. Of course, a consistent estimate is preferred. As will be shown later, the identification results of the method to be presented do not depend upon the noise model. Thus the model given by (1) takes into account the models of closed-loop systems and EV systems. For convenience in illustrating the development of the method, it is assumed that n, = nh + 1 = n and that aflUf 0. In fact the system described by (1) can always be transformed in to a system in which this assumption is valid by using the following procedures. Without loss of generality, let us assume that n, > n,, + 1. Denote n = max (n,, n,, + 1). Then n = n,. Filtering the system input signal by a designed filter l/[F’(z-‘)I, we get
the system at the input and output terminals, respectively, we obtain an augmented system expressed by
A(z-‘)y(k)
= B’(z-‘)u’(k)
+ u(k),
= b;, + b;z-’
+.
+/( = [-y(k - l),
, -y(k -n);
u(k),
(7) The ordinary LS estimate of 0 and its asymptotical property are given by @LS@')
= K&.(W,,(~)~
^ o,,(N)=
I&
f3 +
&+,;b; - l),
&Jr,
(20)
. , -y(k - 2n - 1): Z(k),
, u(k - 2n)lT. (21)
‘&s(N) = R&(N)R&N),
(22)
iLS(N) = 6 + R;;R+,
(23)
whereR+g,Qy, R+ and their estimates at the time instant N are similarly defined by (lo)-(12). R&u = [r+(l),
In particular,
, r_Cu(2n+ 1); r,iu(0), ,r,&2n)lT.
(24)
There are 4n + 2 unknowns r*(i) and r&) (i = 1,2, . ,2n + 1;j = 0, 1,. , 2n) to be determined. It is noticeable that the system (16) has some known poles and zeros which are just the same as the zeros of the filters F,(z-‘) and Fz(z-I). Let A, and s, (i = 1,2,. , n + 1) denote the zeros of &(z-I) and F,(z-‘), respectively. They satisfy the following conditions: and
A, #A,
Ih,l
for
s, # s,
(9)
where
(18) +b*,1z-2tz, (19)
+.
Similar to (8) and (9), we get
(8)
R;$,Rd,,
,,...)
;ift
, u(k -n + l)]‘.
+ &,+,z~(“‘+‘),
and i&,+, =fntlan f0. Define
t b,‘_,z”-‘.
It can be seen that for the system (5) the assumption on n, = nh + 1 = n is valid. Therefore, in the remainder of this paper, we take n, = n and nh = n - 1. Define the regressive signal vector by
(17)
= 1 + a,~~’ +
B(z-‘)=B(z-‘)F,(z~‘)=b,,+b,z-’
(5)
(6)
(16)
u(k) = F,(z-,)U(k)
A(z-‘) = A(z-‘)F,(z-‘)
6, = [-y(k
where ~‘(~-1) = B(Z~‘)F’(r-‘)
+ u(k),
1
J(k) = 16(z_‘)y(k)’
e=[a ,,..., where F’(z-‘) = 1 +fi)r-’ +. +fr~,,--nh~l~n~--nh-‘. Now the original system can be rewritten as follows:
= B(z-l&(k)
A(z-‘)y(k) where
Is,\<1
and
if i=l,2
i #j, ,...,
(25) n+l.
(26)
It follows from (18) and (19) that A@-‘) = 1 + :,A,-’ +
+ ifz,,+,A,~‘B’+” = 0, i=l,2
B(s,-l)=b,,+b,s,~‘+...+bz,,s,~*“=O,
,..., i=l,2
n+l,
(27)
,..., n+l. (28)
R,, g E[&u(k)]
= lim R,,(N)
fv-r
= [ryv(l), r,,(2),
=
Introducting the following matrix and vector,
. , rdn); r,,,(O).r,,,(l),
A:”
, c,.(n - 111’. (12)
0 A’”
Equations (9) and (12) reveal that the estimate a,,(N) is biased, and that the bias R&R+, depends upon the correlation functions between the noise and the delayed system input and output signals. To obtain an unbiased estimate, these unknown correlation functions have to be determined and extracted. In the following section we discuss the method for determining these unknowns.
H, =
,1+1 A::; ’
In this section we develop a method for estimating the R,,. To this end, two digital filters are inserted into the system so as to add some known poles and zeros to the system. Assume that the two filters are F,‘(z-‘) and F;‘(z-‘), where
F,(z-‘) = 1 +f,z -’ +. F*(z_‘) = 1 +f,,+,z-’
+j,+,z-(“+I), +.
+ f*,1+2Zm(‘8+‘).
(14) (15)
These filters are designed to be stable. Connecting them into
1
$1
s,2‘1 I
1
6’: I
SF+<’
1 I
0
E R(h+2)X(h+Z) p, = [-A:“+‘,
3. New method for consistent parameter estimation From (9) we can see that if the consistent estimate of R,, is R&N), then the consistent estimate of 8 can be reached by '&L,(N)= ~LS(N)-~,~(W,~W (13)
1
1
A:“_’
(29)
-A:“+‘,
, -Af::;‘:O,,
, OITE R(2,1+2)x’, (30)
equations (27) and (28) can be rewritten compact form: H,e=p,.
in the following (31)
Multiplying HI on both sides of (23) and using (31) we get H, lili= GLS(N)
-
p, = H, R&R,,
(32)
This equation provides (2n + 2) linear constraints about the (4n + 3) unknowns, i.e. rFu(i) and r&) (i = 0, 1, ,2n + 1: j = 0, 1, ,2n). However, in order to determine all the unknowns, we need an additional (2n + 1) linear constraints. We will now discuss the method of constructing these additional constraints.
971
Brief Papers Consider parameter estimation in the new parameter space given by 1 r,+ [ e2n+r
g=
a2n+r
& ,...,_ -&_._ ,_
__6,
,...,
* ] . (33)
h+l
a2n+l a2n+l
If M is invertible, the unknown vector Rb determined uniquely by the equation R,
in (23) can be
HI Jo= &d”) -PI
= QM-’
(47)
0
System (16) can be expressed by y(k) pji(k
where 4, = [-y(k),
-y(k
-2n
- l),
u(k) aa+1
(34)
, u(k -h)]*.
(35)
- 1) = $18 +:,
. . , -y(k
- 2n);
ii(k),
The LS estimate of 6 and its asymptotical property are given by -^ e.,(N) = R;;(N)R;Y(N), (36) lim iLS(N) = fi + Ri$Ra,,
(37)
N-x
where R.& R,+j, and Ra., and their estimates at time instan_t N are similarly defined by (lo)-(12) replacing I$ and y by 4 and jr, respectively. In particular,
-&
R&= _
(38)
H2=
A: .
A?+?’
A2” tit,
I
Al . .
0 s,2n-I
...
0
.
..
1
...
s?+; ’
ct,
1
E R(2,,+2)X(4n+2)
(39)
. , -LO,.
. ,O] E R’=‘+2’x’.
(40)
n;l
Multiplying by H2 on both sides of (37) gives H2 lili% i&N)
- pz = HzR.$$Rmy
[a,, 02,
I
H,R& = [PI, P2,.
, (Y.,~+~] E R(=‘+2)x(4n+2),
(42)
(2n+2)x(4n+2) , P4n+21E R ?
(43)
where ai, pi E R WI+~)~’ for i = 1,2,. Denoting x = [aZ,,+,;r+(O), r_%(l),
M=
0 -H,~~~&sW~+P~
HI W2,. . ., &,,+I, 0, Pzn+2,. . .r Ant2lR4~
0 P, Q2n
0
a,
, c&n)lT. (44 .
aZn-, P2n
Pz azn+l
An+1
... .”
1
Q4n+2 ) (45)
P‘h+2
equations (32) and (41) can be written in a compact form as follows: (46)
1
(48)
are mutually independent 0 0 S
}2n+2 }2n ]2
H, P q
ir 2-4nz-
’
, I
where S is an invertible matrix in R2x2 and all elements of P are linear combinations of the elements of pi, i = 1,2,. . ,4n + 2 and R.++. From the definition of /ii and the spectral analysis, it can be seen that each element of P is a function of degree &1 + 4 with respect to the 2n + 2 coefficients of &(z-‘), i = 1,2. Therefore, P can be assigned by adjusting the coefficients of fi(z-‘) so that the matrix [HT; PT] is full ranked, and consequently M is invertible. For example, we can assign P
YF
.
y, 1 yz 1
y:” . . p=p*P
rt:
‘.’
. :
0
:
Yn 1 h:”
,,,
q:” .
0
:*n “’ 11,,
I! where yi and vi, i = 1,2,.
~,
1
(50)
1): 1 .‘. : : VI,? 1 1
, n are designed such that
Vi=1,2,...,n: ‘yiE IA,, AZ,.
>An+,)
71,E(s,, s2.. . , %+,I.
,4n + 2.
, rjd2n + 1); kd% h(l),
As the vectors p, and H,Ri$Rh in general, it follows that
(41)
It is known from (38) that (41) gives (2n +2) linear constraints about the unknowns r&), dj) (i = 0, 1, . ,2n + 1; j =0, 1, . ,h) and Z2,,+,. Together with (32), (4n + 4) linear constraints have been obtained for determining the (4n + 3) unknowns r+(i), r,i”(j) (i = O,l,..., 2n+l; j=O,l,..., 2n) and the introduced new unknown rS2,,+,. Now all the unknowns can be determined. For clarity, we rewrite the matrices H,R& and H2R$ in the following forms: H,R&=
/3,
rank(M) =rank (M’) =rank
A“+I ST
p2 = [-1, -1,.
-H2R$Rh
1
We define the following matrix and vector: .,.
0
0 z
. . 9rd~n)l’ . . , r&k): k,(O),
hiA0)~rdlh
A:“+’
where Q = [O;14n+2]E R(4n+2)x(4n+4) and 14n+2is an identity matrix in R(” +2)X(&+2) The following analysis will show that the invertibility of the matrix M can be guaranteed by selecting suitable coefficients of F,(z-r) and F2(z-r). As discussed in Ljung (1987) and Sijderstriim and Stoica (1989) R& and R;f exist if the system input u(r) is a persistently exciting signal of order 4n + 2. Thus it is valid from (29), (32), and (41) that the matrix M has the same rank as that of the matrix M’, given by
(51)
It is obvious from the property of the Vandermonde matrix that the matrix [HT; Pa is invertible and thus M is invertible. Equation (50) provides 2n X (4n + 2) equations for determining the 2n + 2 coefficients of &(z-‘), i = 1,2. As each of those equations is of degree 8n + 4 with respect to the 2n + 2 variables, it is known from the theory of nonlinear algebra equations that the 2n X (4n + 2) nonlinear equations have a feasible solution with respect to the 2n + 2 variables, which reveals that we can always find suitable filters F;(z-‘), i = 1,2 such that M is invertible. Rigorously speaking, we can regard the existence of M-’ as an identifiability condition. In practice, such a condition is usually satisfied even if little attention is paid on selecting the filters. A large number of numerical studies have also verified this conclusion. As the noise-induced biases can be asymptotically determined by (47) the consistent estimate of R+ can be
972
Brief Papers
obtained bv (52) where A, given by
It?,=
stands for the consistent
0
0
[ -H~&_sW) +PZ
6,
estimate of M and is
.
c?2n-,
8,^ 82
h,,
~2,
azn+,
0
B2n+,
1
&4”+2
“’
L+*
(53)
with 6, and bi (i = 1,2,. . ,4n + 2) being the columns of H, R&N) and H*Rif(N). Therefore, it is known from (13) the consistent estimate of 8 can be obtained by i&s(N)
= &s(N) - R&N)&(N)
(54)
The procedures presented above can be summarized by the following algorithm, which we will call the extended BELS method (EBELS). Algorithm 1.
(i) Design two nth filters Fi(z-‘) and F,(z-‘) such that (25), (26) and the identifiability condition are satisfied, and use them to filter the observed input and output data respectively so that an augmented system is obtained (see (16)). (ii) Estimate the parameters of the augn?_ent system using the standard LS method, which gives &s(N). (iii) Construct the matrix I$!, by using equations (53). (iv) Check the invertibility of a,,,; if fi, is invertible then go to next step else go to step (i) to redesign the filters. (v) Determine the corrected estimate iaELS(N) by C&&N)
= &(N)
- R,$(N)Qni,l[
HISLsr)
The design method for the stable filters F,(z-‘) and F2(z-‘) is the same as that discussed in Feng and Zheng (1991), and is thus omitted here. 4. Analysis of consistency It is known from (18) and (19) that A(z-‘) and B(z-r) can be uniquely determined if &z-l) and &z-r) are obtained. Thus the following proposition is self-evident. Proposition 1. If &e&N) (or &&I)) is consistent with the true value e, then the estimate &uLS(N) (or &e,,(t)) obtained using the above algorithms is consistent with 19.
Hence it i! sufficient to consider the consistency of &e,,(N) (or e,,,,(t)). The following two theorems,contirm the consistency of the estimate &,,(N) and OuELS(t), respectively. The proofs of the theorems are the same as those of the corresponding theorems in Feng and Zheng (1991), and thus they are omitted here. Theorem 1. When the size of the:sampled data tends to infinity, the estimated parameter Bems(N) -obtained with Algorithm 1 is consistent with the true value 0 namely,
lim S ems(N) = 3, N--r%
fiir .GBELS(I)= e,
The above algorithm can also be easily transformed into its recursive scheme.
(i) Design two nth-order filters F,(zz’) and F?(z-‘) such that (25) (26), and the identifiability co~ditidn are satisfied, and use them to filter the observed input and output data, respectively, so that the augmented system given by (16) is obtained.
recursive
to obtain
Q,,(r), E, and p,:
&,(I, = &,(t - 1) +&j,@(r) p,=p,-, &(t)
gLS(0), fi,, and
LS method
= &(I
-p,-,I$,(1
-
rnf&(l- 1))
+@p,_,$,))‘f$:E_,,
- 1) + r?,,d,(jr(r) - $:Q,,(r
p,= p,:-, - P,-,d,(l+
- 1))
df~,_,l$,)-‘i:~,,,.
(iv) Constru_ct the matrix I@,by using the columns of H,p, and H,P, (see (52)).
(vi) Perform the bias correction as follows:
(vii) Calculate the 8ams(t) from &&t)
1
system given by
where r(k) is a pseudorandom binary signal (PRBS) of unit magnitude. u(k) and w(k) are unknown colored noises, and are simulated by the following two MA processes: u(k) = e(k) + 0.5e(k - l), w(k) = e(k) + 1.5e(k - 1) + 0.56e(k - 2), where e(k) is a white noise with zero mean. The two filters used in this example are designed as F,(z-‘) = (1 - 0.6z-‘)(l - 0.7z-‘)(l - 0.8zz’), &(z-‘) = (1 - 0.3z --I)(1 - 0.4z_‘)(l - 0.5z_‘), so that the augmented system has known poles 0.6, 0.7, and 0.8, and known zeros 0.3,0.4, and 0.5. We used Algorithms 1 and 2 ten times, with the size of sampled data set N being 500 for each test. The mean values and standard deviations of the estimated parameters are listed in Table 1. The results show that the proposed EBELS method can achieve parameter estimation of high accuracy, even when the system is disturbed by unknown colored noise at the system input and output terminals. Example 2. To illustrate the robustness of the presented method with respect to the noise models, we changed the noise models and again considered the system given in Example 1. Here u(k) and w(k) were assumed to be
(v) Check the invertibility of fi,; if&f, is invertible then go to next step else go to step (i) to redesign the filters.
-PI iBELS(I) =i,,(r) - P,@G,-[ 4 k,(t) 0
(58)
Y(k) =x(k) + n(k), u(k) = r(k) + w(k),
Algorithm 2
i,,(t),
w.p.1.
(1 + 1.5~~’ + 0.8zmZ)x(k) = (1.3 +0.8z-‘)r(k),
(vi) Compute the estim=ate &e,,(N) of the original system parameter 0 from BeeLs(N) (see (18) and (19)).
(iii) Apply the standard
obtained
5. Simulation examples In this section, several examples of simulation will be given to illustrate the performance of the proposed method.
(55)
(ii) Set recursive initial values for &(O), 8,.
(57)
Theorem 2. When 1+ 2, the estimate &e&f) with Algorithm 2 is consistent with 8, i.e.
Example 1. Consider a discrete-time -‘I].
w.p.1.
(56)
via (18) and (19).
(viii) Repeat steps (iii)-(vi) until some convergence criterion has been reached.
Table 1. Simulation results for Example 1 Parameter
True value
aI a2 b,, b,
1.5 0.8 1.3 0.8
Algorithm 1
Algorithm 2
1.436 f 0.821 f 1.247 f 0.833 f
1.410 f 0.775 f 1.234 f 0.827 f
0.089 0.094 0.064 0.065
0.078 0.081 0.072 0.063
Brief Papers Table 2. Simulation results for Example 2 Parameter 0, a2 b,, b,
True value 1.5 0.8 1.3 0.8
Algorithm 1
Algorithm 2
1.414 f 0.766 f 1.277 f 0.841 f
1.407 f 0.758 f 1.268 f 0.839 f
0.093 0.102 0.060 0.071
_
0.088 0.096 0.073 0.070
generated by the following equations: u(k) + 1.4u(k - 1) + 0.48u(k - 2) = e(k) + 0.5e(k - l), w(k) = e(k) + 0.6e(k - 1). Repeating the same experiments as in Example 1 gave the results shown in Table 2. It can be seen that the proposed EBELS method is quite robust with regard to the noise models. 6. Conclusion We have studied the problem of consistent parameter estimation in the case where the measurement noise is unknown and correlated with system input and output signals. The idea of the BELS method proposed in Feng and Zheng (1991) has been extended to establish a unified approach for eliminating the noise-induced bias in the parameter estimates. In the method presented in this paper, two designed filters are inserted simultaneously into the input and output terminals of the system to be identified so that the augmented system has some known zeros and poles. Using the same procedures as those reported in Feng and Zheng (1991), a set of linear equations about the unknown noise-induced biases is obtained. In order to obtain a sufficient number of linear equations to determine the biases, the problem of parameter estimation is again considered in a new shifted-parameter space. Using the information obtained from the known zeros and poles, complementary set of linear equations is constructed, and the noise-induced biases are determined from these two sets of linear equations. With the biases removed, a consistent parameter estimate is reached. Theoretical analysis and simulation examples show that the method described in this paper has inherited all the merits of the previously established BELS method; in particular, the results of identification do not depend upon the noise model. In comparison with the method of Feng and Zheng (1991), the newly established method has the ability to eliminate the
noise-induced biases, even when the colored noise is correlated with the system input. Therefore the present method can deal with the identification of a wider variety of systems, including open-loop systems, closed-loop systems, and error-in-variables systems. Acknowledgement-This Grant No. 6934011.
work
was supported
by NSFC
References Anderson, B. D. 0. (1985) Identification of scale errors-in-variables methods with dynamics. Automaticu Ut, 709-716. Feng, C.-B. and Zhang, Y. (1995) Unbiased identification of systems with nonparametric uncertainty. IEEE Trans. Autom. Control AC-40,933-936. Feng, C.-B. and Zheng, W.-X. (1991) Robust identification of stochastic systems with correlated noise. IEE Proc., PC D 138,48&492. Guo, L., Xia, L. and Moore, J. B. (1988) Robust recursive identification of multidimensional linear regressive models. Int. J. Control 48, %l-979. Gustavsson, I., Ljung, L. and SiiderstrGm, T. (1977) Identification of processes in closed-loop: identifiability and accuracy aspects. Automnticu 13,59-69. Ljung, L. (1987) System Identification: Theory for Users. Prentice-Hall, Englewood Cliffs, NJ. Ljung, L. and SGderstiim, T. (1983) Theory and Practice of Recursive Identification. MIT Press, Cambridge, MA. Moore, J. B., Niedzwiecki, M. and Xia, L. (1990) Identification prediction algorithms for ARMAX models with relaxed positive real conditions. ht. J. Adaptive Control Sig. Process. 4, 49-67. Siiderstriim, T. (1981) Identification of stochastic linear systems in the presence of input noise. Automatica 17, 713-725. Siiderstrtim, T. and Stoica, P. (1983) Instrumental Variable Methods for System Identification. Springer-Verlag, Berlin. SGderstrGm, T. and Stoica, P. (1989) System Identification. Prentice-Hall, London. SGderstrBm, T., Ljung, L. and Gustavsson, I. (1978) A theoretical analysis of recursive identification methods. Automatica 14, 231-244. Solo, V. (1984) Adaptive spectral factorization. Preprint, Harvard University. Zheng, W.-X. and Feng, C.-B. (1992) Identification of a class of dynamic error-in-variables models. inc. 1. Adaptive Control Sig. Process. 6, 431-440.
0
PII: sooo5-1098(%)00246-4
Auromorica, Vol. 33, No. 5, pp. %9-913, 1997 1997 Elsevier Science Ltd. All rights reserved Printed in Great Britain Kxls-1098/97 $17.00 + 0.00
Brief Paper
Unbiased
Parameter
Estimation of Linear Systems with Colored Noises*
YING ZHANGt Key Words-Least-squares identification.
method;
and CHUN-BO
consistency:
Abstract-We report a study of the problem of consistent parameter estimation in the general case where the measurement noise is unknown and correlated not only with the system output but also with the system input. A new method is presented for eliminating the noise-induced biases in the least-squares (LS) estimates. In the proposed method, two filters are introduced to filter the system input and output signals, respectively, so that an augmented system with some known zeros and poles is obtained. By shifting the space of the parameters to be estimated and using the known zeros and poles of the parameters to be estimated and using the known zeros and poles, a sufficient number of linear constraints are constructed to determine the noise-induced biases. After eliminating the biases in the ordinary LS consistent estimates of the parameters are estimates, obtained. Analysis shows that the results of the proposed method are independent of the noise model used. 0 1997 Elsevier Science Ltd.
parameter
FENGS
estimation;
colored
noise;
system
restrictions on the noise model. However, at present there is no efficient way of constructing a suitable instrumental variable for a given case of identification. It is noticeable that there are many systems in use, such as the closed-loop systems and the error-in-variables (EV) systems, in which the noise is correlated not only with the system output but also with the system input. In such cases direct application of the methods developed for open-loop systems will usually give unsatisfactory results. In the existing methods for those kinds of system, it is still necessary that the noise be modeled accurately (see e.g. Gustavsson et al., 1977; Soderstiim, 1981; Anderson, 1985; Zhang and Feng, 1992). Feng and Zheng (1991) and Feng and Zhang (1995) have proposed a bias-eliminating least-squares (BELS) method for treating the bias problem without modeling the noise. However, it is assumed in the BELS method that the noise at the output terminal should not depend statistically upon the system input signal. Without this assumption it is difficult to construct a sufficient number of linear equations for determining the noise-induced biases. Therefore the BELS method cannot be applicable to the general case where the system input and output are both corrupted with colored noise. However, the idea of the BELS method is of potential in treating the noise-induced biases. In this paper we extend the idea of the BELS method to the identification of svstems , in the more general case. The essence of this paper is to construct a sufficient number of linear equations to determine the unknown noise-induced biases in the ordinary LS estimates. To this end, two digital filters are inserted into the system at the input and output terminals so that the augmented system will have some known zeros and poles, which can be used to construct a batch of linear equalities about the biases. Using the information obtained from these known zeros and poles, another batch of linear equalities can also be constructed in a new parameter space. Then the noise-induced biases in the LS estimates can be determined from these two batches of equalities. Finally, with the biases removed, consistent estimates of the parameters are obtained.
1. Introduction The main problem in doing parameter estimation using the least-squares (LS) methods is how to achieve consistent estimation in the presence of noise. In the past decades much effort has been devoted to this problem. Various modified least-squares methods have been developed for the identification of open-loop systems (see e.g. Ljung, 1987; Siiderstrom and Stoica, 1989). In these methods the parameters of the noise model and the system parameters are estimated simultaneously. Thus the results of these methods are inevitably dependent upon an accurate model of the noise. In addition, certain strictly positive-real (SPR) conditions on the noise model must be satisfied in these methods in order to obtain consistent parameter estimates. Detailed studies on the performance of these methods have been reported by Soderstrom et al. (1978) Ljung (1987) and Ljung and Siiderstbm (1983). In practical circumstances, it is usually difficult to model the noise exactly. It is difficult to know beforehand whether or not the SPR conditions on the noise model are satisfied. Fortunately, advances have been made on techniques which can overcome the above-mentioned limitations (see e.g. Solo, 1984, Guo ef al., 1988; Moore et al., 1990). A promising method is the instrumental variables (IV) method originally proposed by Siiderstriim and Stoica (1983). In principle, the results of this method should not depend upon any
2. Problem statement ZN’ Assume that the observation sequence {u(k), y(k)}&‘=, is generated from a linear system which can be modeled by A(r--‘)y(k)
* Received 24 May 1995; revised 29 April 1996; received in final form 6 November 1996. This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor J. Bokor under the direction of Editor Torsten Sederstrom. Corresponding author Dr Ying Zhang. Tel. +657991366: Fax +65 7912687; E-mail [email protected]. t School of Electrical & Electronic Engineering, Nanyang Technological University, Nanyang Avenue, Singapore 639798. $ Research Institute of Automation, Southeast University, Nanjing, 210096, P. R. China.
= B(z-r)u(k)
+ u(k),
(1)
where A(z-‘)=l+a,z~‘+...+a,,,z”.,
(2)
B(zr’)=b,,+b,z-‘+...+b,,$‘*,
(3)
and y(k) and u(k) are the system input and output, respectively. u(k) is a colored noise of an unspecified character, which may be correlated with the sytem input and the system output. Denote the parameter vector f3= by [a,, a?, . , a,,,,;b,,. b,, , b,J. Then the identification of 969
970
Brief Papers
the system is to estimate the parameter vector 8. Of course, a consistent estimate is preferred. As will be shown later, the identification results of the method to be presented do not depend upon the noise model. Thus the model given by (1) takes into account the models of closed-loop systems and EV systems. For convenience in illustrating the development of the method, it is assumed that n, = nh + 1 = n and that aflUf 0. In fact the system described by (1) can always be transformed in to a system in which this assumption is valid by using the following procedures. Without loss of generality, let us assume that n, > n,, + 1. Denote n = max (n,, n,, + 1). Then n = n,. Filtering the system input signal by a designed filter l/[F’(z-‘)I, we get
the system at the input and output terminals, respectively, we obtain an augmented system expressed by
A(z-‘)y(k)
= B’(z-‘)u’(k)
+ u(k),
= b;, + b;z-’
+.
+/( = [-y(k - l),
, -y(k -n);
u(k),
(7) The ordinary LS estimate of 0 and its asymptotical property are given by @LS@')
= K&.(W,,(~)~
^ o,,(N)=
I&
f3 +
&+,;b; - l),
&Jr,
(20)
. , -y(k - 2n - 1): Z(k),
, u(k - 2n)lT. (21)
‘&s(N) = R&(N)R&N),
(22)
iLS(N) = 6 + R;;R+,
(23)
whereR+g,Qy, R+ and their estimates at the time instant N are similarly defined by (lo)-(12). R&u = [r+(l),
In particular,
, r_Cu(2n+ 1); r,iu(0), ,r,&2n)lT.
(24)
There are 4n + 2 unknowns r*(i) and r&) (i = 1,2, . ,2n + 1;j = 0, 1,. , 2n) to be determined. It is noticeable that the system (16) has some known poles and zeros which are just the same as the zeros of the filters F,(z-‘) and Fz(z-I). Let A, and s, (i = 1,2,. , n + 1) denote the zeros of &(z-I) and F,(z-‘), respectively. They satisfy the following conditions: and
A, #A,
Ih,l
for
s, # s,
(9)
where
(18) +b*,1z-2tz, (19)
+.
Similar to (8) and (9), we get
(8)
R;$,Rd,,
,,...)
;ift
, u(k -n + l)]‘.
+ &,+,z~(“‘+‘),
and i&,+, =fntlan f0. Define
t b,‘_,z”-‘.
It can be seen that for the system (5) the assumption on n, = nh + 1 = n is valid. Therefore, in the remainder of this paper, we take n, = n and nh = n - 1. Define the regressive signal vector by
(17)
= 1 + a,~~’ +
B(z-‘)=B(z-‘)F,(z~‘)=b,,+b,z-’
(5)
(6)
(16)
u(k) = F,(z-,)U(k)
A(z-‘) = A(z-‘)F,(z-‘)
6, = [-y(k
where ~‘(~-1) = B(Z~‘)F’(r-‘)
+ u(k),
1
J(k) = 16(z_‘)y(k)’
e=[a ,,..., where F’(z-‘) = 1 +fi)r-’ +. +fr~,,--nh~l~n~--nh-‘. Now the original system can be rewritten as follows:
= B(z-l&(k)
A(z-‘)y(k) where
Is,\<1
and
if i=l,2
i #j, ,...,
(25) n+l.
(26)
It follows from (18) and (19) that A@-‘) = 1 + :,A,-’ +
+ ifz,,+,A,~‘B’+” = 0, i=l,2
B(s,-l)=b,,+b,s,~‘+...+bz,,s,~*“=O,
,..., i=l,2
n+l,
(27)
,..., n+l. (28)
R,, g E[&u(k)]
= lim R,,(N)
fv-r
= [ryv(l), r,,(2),
=
Introducting the following matrix and vector,
. , rdn); r,,,(O).r,,,(l),
A:”
, c,.(n - 111’. (12)
0 A’”
Equations (9) and (12) reveal that the estimate a,,(N) is biased, and that the bias R&R+, depends upon the correlation functions between the noise and the delayed system input and output signals. To obtain an unbiased estimate, these unknown correlation functions have to be determined and extracted. In the following section we discuss the method for determining these unknowns.
H, =
,1+1 A::; ’
In this section we develop a method for estimating the R,,. To this end, two digital filters are inserted into the system so as to add some known poles and zeros to the system. Assume that the two filters are F,‘(z-‘) and F;‘(z-‘), where
F,(z-‘) = 1 +f,z -’ +. F*(z_‘) = 1 +f,,+,z-’
+j,+,z-(“+I), +.
+ f*,1+2Zm(‘8+‘).
(14) (15)
These filters are designed to be stable. Connecting them into
1
$1
s,2‘1 I
1
6’: I
SF+<’
1 I
0
E R(h+2)X(h+Z) p, = [-A:“+‘,
3. New method for consistent parameter estimation From (9) we can see that if the consistent estimate of R,, is R&N), then the consistent estimate of 8 can be reached by '&L,(N)= ~LS(N)-~,~(W,~W (13)
1
1
A:“_’
(29)
-A:“+‘,
, -Af::;‘:O,,
, OITE R(2,1+2)x’, (30)
equations (27) and (28) can be rewritten compact form: H,e=p,.
in the following (31)
Multiplying HI on both sides of (23) and using (31) we get H, lili= GLS(N)
-
p, = H, R&R,,
(32)
This equation provides (2n + 2) linear constraints about the (4n + 3) unknowns, i.e. rFu(i) and r&) (i = 0, 1, ,2n + 1: j = 0, 1, ,2n). However, in order to determine all the unknowns, we need an additional (2n + 1) linear constraints. We will now discuss the method of constructing these additional constraints.
971
Brief Papers Consider parameter estimation in the new parameter space given by 1 r,+ [ e2n+r
g=
a2n+r
& ,...,_ -&_._ ,_
__6,
,...,
* ] . (33)
h+l
a2n+l a2n+l
If M is invertible, the unknown vector Rb determined uniquely by the equation R,
in (23) can be
HI Jo= &d”) -PI
= QM-’
(47)
0
System (16) can be expressed by y(k) pji(k
where 4, = [-y(k),
-y(k
-2n
- l),
u(k) aa+1
(34)
, u(k -h)]*.
(35)
- 1) = $18 +:,
. . , -y(k
- 2n);
ii(k),
The LS estimate of 6 and its asymptotical property are given by -^ e.,(N) = R;;(N)R;Y(N), (36) lim iLS(N) = fi + Ri$Ra,,
(37)
N-x
where R.& R,+j, and Ra., and their estimates at time instan_t N are similarly defined by (lo)-(12) replacing I$ and y by 4 and jr, respectively. In particular,
-&
R&= _
(38)
H2=
A: .
A?+?’
A2” tit,
I
Al . .
0 s,2n-I
...
0
.
..
1
...
s?+; ’
ct,
1
E R(2,,+2)X(4n+2)
(39)
. , -LO,.
. ,O] E R’=‘+2’x’.
(40)
n;l
Multiplying by H2 on both sides of (37) gives H2 lili% i&N)
- pz = HzR.$$Rmy
[a,, 02,
I
H,R& = [PI, P2,.
, (Y.,~+~] E R(=‘+2)x(4n+2),
(42)
(2n+2)x(4n+2) , P4n+21E R ?
(43)
where ai, pi E R WI+~)~’ for i = 1,2,. Denoting x = [aZ,,+,;r+(O), r_%(l),
M=
0 -H,~~~&sW~+P~
HI W2,. . ., &,,+I, 0, Pzn+2,. . .r Ant2lR4~
0 P, Q2n
0
a,
, c&n)lT. (44 .
aZn-, P2n
Pz azn+l
An+1
... .”
1
Q4n+2 ) (45)
P‘h+2
equations (32) and (41) can be written in a compact form as follows: (46)
1
(48)
are mutually independent 0 0 S
}2n+2 }2n ]2
H, P q
ir 2-4nz-
’
, I
where S is an invertible matrix in R2x2 and all elements of P are linear combinations of the elements of pi, i = 1,2,. . ,4n + 2 and R.++. From the definition of /ii and the spectral analysis, it can be seen that each element of P is a function of degree &1 + 4 with respect to the 2n + 2 coefficients of &(z-‘), i = 1,2. Therefore, P can be assigned by adjusting the coefficients of fi(z-‘) so that the matrix [HT; PT] is full ranked, and consequently M is invertible. For example, we can assign P
YF
.
y, 1 yz 1
y:” . . p=p*P
rt:
‘.’
. :
0
:
Yn 1 h:”
,,,
q:” .
0
:*n “’ 11,,
I! where yi and vi, i = 1,2,.
~,
1
(50)
1): 1 .‘. : : VI,? 1 1
, n are designed such that
Vi=1,2,...,n: ‘yiE IA,, AZ,.
>An+,)
71,E(s,, s2.. . , %+,I.
,4n + 2.
, rjd2n + 1); kd% h(l),
As the vectors p, and H,Ri$Rh in general, it follows that
(41)
It is known from (38) that (41) gives (2n +2) linear constraints about the unknowns r&), dj) (i = 0, 1, . ,2n + 1; j =0, 1, . ,h) and Z2,,+,. Together with (32), (4n + 4) linear constraints have been obtained for determining the (4n + 3) unknowns r+(i), r,i”(j) (i = O,l,..., 2n+l; j=O,l,..., 2n) and the introduced new unknown rS2,,+,. Now all the unknowns can be determined. For clarity, we rewrite the matrices H,R& and H2R$ in the following forms: H,R&=
/3,
rank(M) =rank (M’) =rank
A“+I ST
p2 = [-1, -1,.
-H2R$Rh
1
We define the following matrix and vector: .,.
0
0 z
. . 9rd~n)l’ . . , r&k): k,(O),
hiA0)~rdlh
A:“+’
where Q = [O;14n+2]E R(4n+2)x(4n+4) and 14n+2is an identity matrix in R(” +2)X(&+2) The following analysis will show that the invertibility of the matrix M can be guaranteed by selecting suitable coefficients of F,(z-r) and F2(z-r). As discussed in Ljung (1987) and Sijderstriim and Stoica (1989) R& and R;f exist if the system input u(r) is a persistently exciting signal of order 4n + 2. Thus it is valid from (29), (32), and (41) that the matrix M has the same rank as that of the matrix M’, given by
(51)
It is obvious from the property of the Vandermonde matrix that the matrix [HT; Pa is invertible and thus M is invertible. Equation (50) provides 2n X (4n + 2) equations for determining the 2n + 2 coefficients of &(z-‘), i = 1,2. As each of those equations is of degree 8n + 4 with respect to the 2n + 2 variables, it is known from the theory of nonlinear algebra equations that the 2n X (4n + 2) nonlinear equations have a feasible solution with respect to the 2n + 2 variables, which reveals that we can always find suitable filters F;(z-‘), i = 1,2 such that M is invertible. Rigorously speaking, we can regard the existence of M-’ as an identifiability condition. In practice, such a condition is usually satisfied even if little attention is paid on selecting the filters. A large number of numerical studies have also verified this conclusion. As the noise-induced biases can be asymptotically determined by (47) the consistent estimate of R+ can be
972
Brief Papers
obtained bv (52) where A, given by
It?,=
stands for the consistent
0
0
[ -H~&_sW) +PZ
6,
estimate of M and is
.
c?2n-,
8,^ 82
h,,
~2,
azn+,
0
B2n+,
1
&4”+2
“’
L+*
(53)
with 6, and bi (i = 1,2,. . ,4n + 2) being the columns of H, R&N) and H*Rif(N). Therefore, it is known from (13) the consistent estimate of 8 can be obtained by i&s(N)
= &s(N) - R&N)&(N)
(54)
The procedures presented above can be summarized by the following algorithm, which we will call the extended BELS method (EBELS). Algorithm 1.
(i) Design two nth filters Fi(z-‘) and F,(z-‘) such that (25), (26) and the identifiability condition are satisfied, and use them to filter the observed input and output data respectively so that an augmented system is obtained (see (16)). (ii) Estimate the parameters of the augn?_ent system using the standard LS method, which gives &s(N). (iii) Construct the matrix I$!, by using equations (53). (iv) Check the invertibility of a,,,; if fi, is invertible then go to next step else go to step (i) to redesign the filters. (v) Determine the corrected estimate iaELS(N) by C&&N)
= &(N)
- R,$(N)Qni,l[
HISLsr)
The design method for the stable filters F,(z-‘) and F2(z-‘) is the same as that discussed in Feng and Zheng (1991), and is thus omitted here. 4. Analysis of consistency It is known from (18) and (19) that A(z-‘) and B(z-r) can be uniquely determined if &z-l) and &z-r) are obtained. Thus the following proposition is self-evident. Proposition 1. If &e&N) (or &&I)) is consistent with the true value e, then the estimate &uLS(N) (or &e,,(t)) obtained using the above algorithms is consistent with 19.
Hence it i! sufficient to consider the consistency of &e,,(N) (or e,,,,(t)). The following two theorems,contirm the consistency of the estimate &,,(N) and OuELS(t), respectively. The proofs of the theorems are the same as those of the corresponding theorems in Feng and Zheng (1991), and thus they are omitted here. Theorem 1. When the size of the:sampled data tends to infinity, the estimated parameter Bems(N) -obtained with Algorithm 1 is consistent with the true value 0 namely,
lim S ems(N) = 3, N--r%
fiir .GBELS(I)= e,
The above algorithm can also be easily transformed into its recursive scheme.
(i) Design two nth-order filters F,(zz’) and F?(z-‘) such that (25) (26), and the identifiability co~ditidn are satisfied, and use them to filter the observed input and output data, respectively, so that the augmented system given by (16) is obtained.
recursive
to obtain
Q,,(r), E, and p,:
&,(I, = &,(t - 1) +&j,@(r) p,=p,-, &(t)
gLS(0), fi,, and
LS method
= &(I
-p,-,I$,(1
-
rnf&(l- 1))
+@p,_,$,))‘f$:E_,,
- 1) + r?,,d,(jr(r) - $:Q,,(r
p,= p,:-, - P,-,d,(l+
- 1))
df~,_,l$,)-‘i:~,,,.
(iv) Constru_ct the matrix I@,by using the columns of H,p, and H,P, (see (52)).
(vi) Perform the bias correction as follows:
(vii) Calculate the 8ams(t) from &&t)
1
system given by
where r(k) is a pseudorandom binary signal (PRBS) of unit magnitude. u(k) and w(k) are unknown colored noises, and are simulated by the following two MA processes: u(k) = e(k) + 0.5e(k - l), w(k) = e(k) + 1.5e(k - 1) + 0.56e(k - 2), where e(k) is a white noise with zero mean. The two filters used in this example are designed as F,(z-‘) = (1 - 0.6z-‘)(l - 0.7z-‘)(l - 0.8zz’), &(z-‘) = (1 - 0.3z --I)(1 - 0.4z_‘)(l - 0.5z_‘), so that the augmented system has known poles 0.6, 0.7, and 0.8, and known zeros 0.3,0.4, and 0.5. We used Algorithms 1 and 2 ten times, with the size of sampled data set N being 500 for each test. The mean values and standard deviations of the estimated parameters are listed in Table 1. The results show that the proposed EBELS method can achieve parameter estimation of high accuracy, even when the system is disturbed by unknown colored noise at the system input and output terminals. Example 2. To illustrate the robustness of the presented method with respect to the noise models, we changed the noise models and again considered the system given in Example 1. Here u(k) and w(k) were assumed to be
(v) Check the invertibility of fi,; if&f, is invertible then go to next step else go to step (i) to redesign the filters.
-PI iBELS(I) =i,,(r) - P,@G,-[ 4 k,(t) 0
(58)
Y(k) =x(k) + n(k), u(k) = r(k) + w(k),
Algorithm 2
i,,(t),
w.p.1.
(1 + 1.5~~’ + 0.8zmZ)x(k) = (1.3 +0.8z-‘)r(k),
(vi) Compute the estim=ate &e,,(N) of the original system parameter 0 from BeeLs(N) (see (18) and (19)).
(iii) Apply the standard
obtained
5. Simulation examples In this section, several examples of simulation will be given to illustrate the performance of the proposed method.
(55)
(ii) Set recursive initial values for &(O), 8,.
(57)
Theorem 2. When 1+ 2, the estimate &e&f) with Algorithm 2 is consistent with 8, i.e.
Example 1. Consider a discrete-time -‘I].
w.p.1.
(56)
via (18) and (19).
(viii) Repeat steps (iii)-(vi) until some convergence criterion has been reached.
Table 1. Simulation results for Example 1 Parameter
True value
aI a2 b,, b,
1.5 0.8 1.3 0.8
Algorithm 1
Algorithm 2
1.436 f 0.821 f 1.247 f 0.833 f
1.410 f 0.775 f 1.234 f 0.827 f
0.089 0.094 0.064 0.065
0.078 0.081 0.072 0.063
Brief Papers Table 2. Simulation results for Example 2 Parameter 0, a2 b,, b,
True value 1.5 0.8 1.3 0.8
Algorithm 1
Algorithm 2
1.414 f 0.766 f 1.277 f 0.841 f
1.407 f 0.758 f 1.268 f 0.839 f
0.093 0.102 0.060 0.071
_
0.088 0.096 0.073 0.070
generated by the following equations: u(k) + 1.4u(k - 1) + 0.48u(k - 2) = e(k) + 0.5e(k - l), w(k) = e(k) + 0.6e(k - 1). Repeating the same experiments as in Example 1 gave the results shown in Table 2. It can be seen that the proposed EBELS method is quite robust with regard to the noise models. 6. Conclusion We have studied the problem of consistent parameter estimation in the case where the measurement noise is unknown and correlated with system input and output signals. The idea of the BELS method proposed in Feng and Zheng (1991) has been extended to establish a unified approach for eliminating the noise-induced bias in the parameter estimates. In the method presented in this paper, two designed filters are inserted simultaneously into the input and output terminals of the system to be identified so that the augmented system has some known zeros and poles. Using the same procedures as those reported in Feng and Zheng (1991), a set of linear equations about the unknown noise-induced biases is obtained. In order to obtain a sufficient number of linear equations to determine the biases, the problem of parameter estimation is again considered in a new shifted-parameter space. Using the information obtained from the known zeros and poles, complementary set of linear equations is constructed, and the noise-induced biases are determined from these two sets of linear equations. With the biases removed, a consistent parameter estimate is reached. Theoretical analysis and simulation examples show that the method described in this paper has inherited all the merits of the previously established BELS method; in particular, the results of identification do not depend upon the noise model. In comparison with the method of Feng and Zheng (1991), the newly established method has the ability to eliminate the
noise-induced biases, even when the colored noise is correlated with the system input. Therefore the present method can deal with the identification of a wider variety of systems, including open-loop systems, closed-loop systems, and error-in-variables systems. Acknowledgement-This Grant No. 6934011.
work
was supported
by NSFC
References Anderson, B. D. 0. (1985) Identification of scale errors-in-variables methods with dynamics. Automaticu Ut, 709-716. Feng, C.-B. and Zhang, Y. (1995) Unbiased identification of systems with nonparametric uncertainty. IEEE Trans. Autom. Control AC-40,933-936. Feng, C.-B. and Zheng, W.-X. (1991) Robust identification of stochastic systems with correlated noise. IEE Proc., PC D 138,48&492. Guo, L., Xia, L. and Moore, J. B. (1988) Robust recursive identification of multidimensional linear regressive models. Int. J. Control 48, %l-979. Gustavsson, I., Ljung, L. and SiiderstrGm, T. (1977) Identification of processes in closed-loop: identifiability and accuracy aspects. Automnticu 13,59-69. Ljung, L. (1987) System Identification: Theory for Users. Prentice-Hall, Englewood Cliffs, NJ. Ljung, L. and SGderstiim, T. (1983) Theory and Practice of Recursive Identification. MIT Press, Cambridge, MA. Moore, J. B., Niedzwiecki, M. and Xia, L. (1990) Identification prediction algorithms for ARMAX models with relaxed positive real conditions. ht. J. Adaptive Control Sig. Process. 4, 49-67. Siiderstriim, T. (1981) Identification of stochastic linear systems in the presence of input noise. Automatica 17, 713-725. Siiderstrtim, T. and Stoica, P. (1983) Instrumental Variable Methods for System Identification. Springer-Verlag, Berlin. SGderstrGm, T. and Stoica, P. (1989) System Identification. Prentice-Hall, London. SGderstrBm, T., Ljung, L. and Gustavsson, I. (1978) A theoretical analysis of recursive identification methods. Automatica 14, 231-244. Solo, V. (1984) Adaptive spectral factorization. Preprint, Harvard University. Zheng, W.-X. and Feng, C.-B. (1992) Identification of a class of dynamic error-in-variables models. inc. 1. Adaptive Control Sig. Process. 6, 431-440.