- Email: [email protected]
The Frisch Identification Scheme: Properties of the Solution in the Dynamic Case
Copyright e IFAC System Identification. Copenhagen. Denmark. 1994
THE FRISCH IDENTIFICATION SCHEME: PROPERTIES OF THE SOLUTION IN THE DYNAMIC CASE S. Beghelli, P. Castaldi, R.P. Guidorzi and U. Soverini Univusilii di Bologna. EI~lIronica. Informalica ~ Sisl~mislica. dd Risorgim~nlo 2. 40136 Bologna. Italy
Dipanim~nlO
Vial~
di
Abstract. llris paper investigates some of the many algebraic properties of the solution of the Frisch identification scheme applied to dynamic systems. These properties are relaled to the design of a robust selection criterion leading to a single model also when the assumptions of the scheme are not fulfilled. Key Words. System identifiCalion. errors-in-variables. stochastic systems. linear systems. parameter estimation.
I. INTRODUCTION
some of these properties to a robust selection criterion that has been recently proposed.
The identification of dynamic errors-in-variables models has been investigated by several authors in recent years (Anderson, 1985; Anderson and Deistler. 1984; Deistler and Anderson, 1989). In Beghelli et al. (1990) the problem has been analysed under the assumptions that the variables are corrupted by additive and uncorrelated white noises and the analysis has been based on a deterministic approach which constitutes an extension of the Frisch static scheme to the dynamic case. The result obtained is that, in general, it is possible to determine the actual model of the system which has generated the data and the variances of the additive noises. The existence, in general, of a single solution has been proved in Picd et al. (1993) using a completely different approach.
The content is organized as follows. Section 2 describes how the solution can be extracted when the assumptions are satisfied while Section 3 analyses some properties of this solution and shows how they are linked to the model selection criterion described in Beghelli et al. (1993), which has shown remarkable robustness properties. Some concluding remarks are finally given in Section 4.
2. THE DYNAMIC FRISCH SCHEME Let us consider two scalar discrete-time real zeromean stationary random sequences Wl (t), W2(t) with rational power spectrum and related by a linear timeinvariant relation of the type
Despite the flexibility due to the mild assumptions of the Frisch scheme, its applications to real problems have been modest even if suitable numerical algorithms can reduce the computational burden associated to the identification of real multivariable dynamic systems (Beghelli et al., 1994). When the assumptions behind the scheme are violated, the determination of a single model in the family of possible ones requires the introduction of a suitable criterion. This paper investigates some of the many algebraical properties of the solution of this scheme and links
where P~(z) and P;(z) are relatively prime polynomials (z = forward-shift operator) with degrees nl and n2 respectively. We are interested in finding the linear difference equation which links the sequences Wl (t) and W2(t) from noisy measurements of the type Wl(t)
=Wl(t) + Wl(t)
W2(t) = W2(t)
+ Ui:z(t) ,
(2.2)
where Wl (t) and Ui:z(t) are the errors in the variables. In the scheme here considered Wl (t) and W2(t) are assumed to be zero-mean white noise processes with unknown variances O'~tul , O'~tu, , mutually uncorrelated and uncorrelated with the true signals Wl (t), W2(t).
This research has been supported by the Ministry for University and Scientific and Technologic Research, Rome, under project Model Identification. Colltro! Systems and Signal Processing 889
=
(i 0, ... , k) of the power spectrum of the signal Wl(t)
This scheme can be considered as an extension of the Frisch static scheme (Frisch. 1934) to dynamic systems. Representations of this type are usually called enurs-in-variables models and play an important role when the interests of the analysis concern the research of the true system behind the data rather than the choice of a model for predictive purposes. Note that the variables are treated in a symmetric way. so that the scheme allows identification of both causal and non-causaJ dynamic systems. Relation (2.1) can be written. without loss of generality, in the form RI
.tk(WlWl)
,=0
LP,~(t+i)=O.
roUl (0)
r oU1 (I)
rtiJl (k)
rtiJ 1(1)
r oU1 (0)
r';'l(k - I)
(~ -
[ rtiJl:(k)
rtiJ 1
I)
r oU1 (0)
(2.9) and the (k + I) x (k + I) symmetric Toeplitz matrix Ek(~~). built with the samples roU,(i). (i = 0, ... ,k) of the power spectrum of the signal W2(t)
n2
La,wl(t+i)+
=
(2.3)
E k (W2 W2)
,=0
=
r,;,,(O)
a"
where the time-invariant parameters Pi are the coefficients of the polynomials P;(z) and P;(z) respectively. with P;(z) monic and nl ~ n2. Denote now with 4J w1 (z) the power spectrum of the signal
r,;,,(l)
r,;"
r,;,,(O)
r,;,,(l)
r,;,,(~ -
[ r,;,,:(k)
Wl(t)
(k)
r,;,,(k - I)
I) (2.10)
In the same way. the (k+ I) x (k+ I) Toeplitz matrix Ek(W2Wd = E'[(WlW2) can be obtained with the samples r""';'1 (i), (i = -k, ... , k)
+00 4JW1 (z)=
L
r w1 (i)z-' ,
i=-CX)
(2.4)
where r w1 (i) = E[Wl(t)Wl(t + i») is the autocorrelation of the signal Wl(t): the power spectrum of Wl (t) is
E k (W2 Wl) = r"""'I(O) r"""'I(-I)
+00
4J w1 (z)
=
L
r"'l (i) z-,
+ U~1
r"""'1 (1) r"""'1 (0)
...
:::
r"""'1 (k)
r"""'I~k -
r"""'I(-k+l)...
r"""'I(-k)
r""", 1 (0)
(2.11 )
We can now define the (2k + 2) x (2k + 2) symmetric matrix t:k partitioned in the following way
In the same way. for the signals W2(t) and W2(t) we have +00 L
.
[
(2.5)
i=-oo
4J..,,(z)=
]
I)
•
Ek
r..,,(i)z-' ,
_ [Ek(WlWl) -.
_=-00
E k (W2 w d
Ek(WlW2)] M
•
(2.12)
L-k(W2 W2)
(2.6)
When the errors in the variables are absent it is possible to inspect the sequence of increasing dimension matrices (2.13)
and +00 -4Jw ,(z)=
L
r..,,(i)z-'+u~,
(2.7)
i=-oo
and it can be easily proved that Ek becomes singular (non-negative definite) when k is equal to or greater than nl.
where rw,(i) = E[~(t)W2(t+i») is the autocorrelation of the signal W2(t). The cross-spectrum between the signals Wl (t) and W2(t) is given by
Moreover
where
P~1 =[ao, ... ,an1-l,I,po, ... ,Pnl]T
+00
=- ~~:~ 4Jw, (z) =,~oo r W,Wl (i) z-i
(2.14)
is t~e (2nl + 2) vector whose entries are the coefficients of relation (2.3). Of course. when nl > n2 the coefficients Pn,+lt ... , Pn l are equal to zero.
,
(2.8)
where r"" oU l(i) = E[W2(t)Wl(t+i)] is the crosscorrelation between Wl(t) and ~(t). Since the white noises Wl (t) and ~(t) are uncorrelated with the signals Wl(t) and ~(t) and mutually uncorrelated. 41"""'1 (z)
En1 P~1 =O.
When the data are corrupted by additive noises the matrices E k • obtained with the noisy sequences Wl (t) and W2(t). are non-singular (positive definite) for any k. Under the previous assumptions. the matrices Ek satisfy the following relation
=4JtiJ, tiJ l (z).
Let us consider the (k+ I) x (k+ I) symmetric ToeplilZ matrix Ek (Wl wd. built with the samples r w1 (i ),
(2.15)
890
where
greater than a certain value (n1) have a single common point, then this point corresponds to the actual variances (O'ti;" 0';,) of the errors in the variables and the coefficients (p~, of the relation linking the data can be exactly evaluated. In Picci et al. (1993) it has been proved that, with the exception of singular cases, the common point is unique.
(2.16)
The degrees and the coefficients of P~(z) and P;(z) can be identified in the following way: - for a given k determine all matrices E" = [O'wJ"+l' O'w,I"+l]' i.e. all couples (0',;;" O'w,). such that E" is non-negative definite
3. PROPERTIES OF TIlE SOLUTION The point belonging to all curves described in Section 2 has some interesting properties. With reference to the curve associated to polynomials of degree (n1 + I), the following result can be proved.
- analyse how the solution set obtained at the previous step varies with k.
Theorem 3.1 - The vectors belonging to the subspace Im [P~,+l P:,+tllead to a pair of polynomials Q1(Z) and Q2(Z) with Q2(Z)/Q1(Z) = P;'(z)/Pi(z).
With reference to the first step in Beghelli et al. (1990) it has been proved that. for a given value of k. the solution set of relation (2.17) describes in the first quadrant of the plane (O'w" O'w,) a convex curve whose concavity faces the origin. Every point (0'w, , 0'w,) on the curve is associated to a couple of polynomials P 1(z), P 2 (z) with deg [P1(z)] = k. The coefficients Pie = [o~, ... , 0Z_l' I, p~ ,... ,p;]T of such polynomials can be obtained by means of the equation
Proof - A vector of coefficients given by linear combinations of the vectors P~'+l and P:,+l is of the type q = k 1 P~,+l + k 2 P~'+l = [k 1 0 0 , k 1 0
1
+ k2 0 0 ,
.. .k 1 + k 2 0 n ,-1, k 2 , k 1 Po, k 1 P1 + k 2 Po, ... ,k 1Pn, + k 2Pn,-1' k 2Pn,] (3.1)
and corresponds to the pair of polynomials
=(k 1z+ k2 )P;(z) Q2(Z) =(k 1 z + k 2 )P;(z) , Q1(Z)
Moreover, in Beghelli et al. (1990) it has been proved that the curves associated to increasing values of k approach the origin of the noise plane (0'w, , 0'W2) and each curve includes all subsequent ones.
and this proves the theorem.
When W1(t) and W2(t) are linked by relation (2.1) and are corrupted by additive white noises satisfying the previous assumptions. the point corresponding to the actual variances (O'~" O'~,) of the noises belongs to the curve associated to polynomials of degree k = n1' The linear relation obtained at this point is characterized by the actual coefficients p~, .
A similar property holds when curves associated to
polynomials of greater degrees are considered. With reference to a matrix E" it is also worth analysing the solutions of the equation det [E Ie diag [O'w, 1,,+1> O'w, /"+1]] = in the first quadrant of the plane (O'w"O'w,). Fig. I shows the solution set with reference to a matrix with k = I. The curve nearest to the origin leads to a non-negative definite matrix El expressed by relation (2.17). the next curve is associated to a non-definite matrix El with only one eigenvaIue less than zero, the number of negative eigenvaIues increases in a unit from one curve to the next. Consequently the following property holds.
°
It can easily be verified that this noise point is common also to the curve related to polynomials of degree (n1 + I); in fact the rank of matrix E n ,+l -diag [O'~, I n ,+2' 0'~Jn,+2] is equal to 2n1 + 2 and
Ker [E n ,+l - diag [O'~, I n ,+2, 0':U,In,+2]]= Im [P~,+l P~,+l] ,
(2.19)
Property 3.1 - Let us consider, with reference to k = I, a generic point A (O'~, O'~) in the noise plane; the straight line through A with unitary slope intersects the curves previously defined at points Al. A 2 , A 3 and A•. ThevaIues(O'~;-O'~)(i=1, ... ,4) are the eigenvaIues of matrix El -diag [O'~ 12 , O'~ 12 ] since, by subtracting from this matrix diag [(O'~; 0'~)12, (O'~i - 0'~)12]' all eigenvalues are shifted by (O'~i - O'~). Of course this property can be generalized for a generic value of k.
where
P~,+l = [00,"" 0n,-l, 1,0, Po, P~,+l = [O,oo, ... ,on,_l,I,O,po,
(3.2)
,Pn"O]T
,Pn,]T . (2.20) A similar property holds for all curves related to polynomials with greater degrees. In conclusion, from an identification point of view, when all curves associated to polynomials of degrees equal to or 891
CHI. The same property holds for KIC and CH2 •
A3 A2 A 1/
Let us consider a point Al (u~ I • u~ I) on the first curve of fig. 2 and let A 2 (U~, I u~,) be the point obtained by intersecting the second curve with the straight line from Al with unitary slope. The two smallest eigenvalues of matrix E n d I diag [UA;/nIH. uA;,In l + 2 ] are 0 and (UA; - UA;) respectively; the corresponding eigenvectors are orthogonal since they belong to a symmetric matrix. The entries of the eigenvector associated to the null eigenvalue are, if properly nonnalized, the coefficients of relation (2.3); thus the following conjecture can be fonnulated.
.' //
.4.-.. - - - - - - - - - - 1
...
./
Fig. 1. Solution set in the noise plane; x-axis: input noise variance; y-axis: output noise variance
Conjecture 3.2 - When point A I moves from K I C to CHI, crossing C, a discontinuity in the coefficients of relation (2.3) can be observed.
On the basis of the previous considerations the following property can also be stated.
When the assumptions of this identification scheme are not satisfied (time-invariance and linearity of the processes, uncorrelation of the additive white noises, etc.) no common point can be observed and a single solution can be obtained only by introducing suitable selection criteria. Analysis of the previous properties can be considered a starting point for the design of these criteria. Under mild assumption violations, different consistent algorithms have been tested and compared in Beghelli et al. (1993, 1994) for several signal-to-noise ratios. The best robustness has been exhibited by the Shifted Relation criterion, based on the peculiar rank deficiency properties of matrix E ndI described by relations (2.19) and (2.20).
Property 3.2 - The two curves nearest to the origin of the solution set of equation det [Enl +1 diag [UWI 1n l +2, U w, 1nl +21]= 0 share the point (u~ I ' u~,) since at this point the non-negative definite matrix Enl + I has two null eigenvalues.
4. CONCLUSIONS
Fig.
2.
In this paper the identification of linear relations linking two scalar discrete-time stationary random sequences with rational power spectrum has been considered. The identification procedure here considered is an extension to dynamic systems of the algebraic Frisch scheme. Some properties of the solution of this identification problem have been related to the design of a robust selection criterion leading to a single solution also when the assumptions of the scheme are not fulfilled.
Solution set in the noise plane; x-axis: input !l0ise variance; y-axis: output noise variance
Fig. 2 shows this situation: the curve characterized by eigenvalues greater than or equal to zero (End 1 ~ 0) is composed of the arcs KIC and CHI; in the same way, the curve characterized by an eigenvalue less than zero is composed of the arcs K 2 C, C H 2 • Note that no discontinuity in the derivative can be observed by going to K 2 C from CHI and to KIC from CH2 ; the arcs K 2 C and KIC can thus be considered the natural extensions of CHI and C H 2 • Moreover, numerical simulations induce the following conjecture.
5. REFERENCES Anderson, B.D.O. (1985). Identification of scalar errors-in-variables models with dynamics. Automatica, 21, 709-716. Anderson B.D.O. and M. Deistler (1984). Identifiability of dynamic errors-in-variables models. J. of TIme Series Analysis,S, 1-13.
Conjecture 3.1 - The eigenvector associated to the null eigenvalue of matrix E n d I diag [U w.!nl+2, U w,Ind2 ] shows no discontinuities when point (UW1l u w,) moves on K 2 C and
Beghelli, S., RP. Guidorzi and U. Soverini (1990). The Frisch scheme in dynamic system identification. AUlOmatica, 26, 171-176. 892
Beghelli. S., P. Castaldi. R.P. Guidorzi and U. Soverini (1993). A robust criterion for model selection in identification from noisy data. Proc. of the Int. Conf. on Systems Science. pp. 480-484. Las Vegas. USA.
intensive Methods in Control and Signal Processing. Prague.
BegheUi. S.• P. CastaIdi. R.P. Guidorzi and U. Soverini (1994). A comparison between different model selection criteria in Frisch scheme identification. Systems Science J.. 20, n.I.
Frisch. R. (1934). Statistical confluence analysis by means of complete regression systems, Pub. No. 5. Economics Institute. University of Oslo.
Deistler, M. and B.D.O. Anderson (1989). Linear dynamic errors-in-variables models: some structure theory. 1. of Econometrics. 41. 39-63.
Picci. G.• F. Gei and S. Pinzoni (1993). Identification of errors-in-variables with white measurement errors. Proc. of the second European Control Conference. pp. 2154-2158. Groningen. The Netherlands.
Beghelli, S.• P. CastaIdi, R.P. Guidorzi. U. Soverini and A. Stoian (1994). Practical implementation of the Frisch scheme in the identification of dynamical systems. IEEE European Workshop on Computer-
893
THE FRISCH IDENTIFICATION SCHEME: PROPERTIES OF THE SOLUTION IN THE DYNAMIC CASE S. Beghelli, P. Castaldi, R.P. Guidorzi and U. Soverini Univusilii di Bologna. EI~lIronica. Informalica ~ Sisl~mislica. dd Risorgim~nlo 2. 40136 Bologna. Italy
Dipanim~nlO
Vial~
di
Abstract. llris paper investigates some of the many algebraic properties of the solution of the Frisch identification scheme applied to dynamic systems. These properties are relaled to the design of a robust selection criterion leading to a single model also when the assumptions of the scheme are not fulfilled. Key Words. System identifiCalion. errors-in-variables. stochastic systems. linear systems. parameter estimation.
I. INTRODUCTION
some of these properties to a robust selection criterion that has been recently proposed.
The identification of dynamic errors-in-variables models has been investigated by several authors in recent years (Anderson, 1985; Anderson and Deistler. 1984; Deistler and Anderson, 1989). In Beghelli et al. (1990) the problem has been analysed under the assumptions that the variables are corrupted by additive and uncorrelated white noises and the analysis has been based on a deterministic approach which constitutes an extension of the Frisch static scheme to the dynamic case. The result obtained is that, in general, it is possible to determine the actual model of the system which has generated the data and the variances of the additive noises. The existence, in general, of a single solution has been proved in Picd et al. (1993) using a completely different approach.
The content is organized as follows. Section 2 describes how the solution can be extracted when the assumptions are satisfied while Section 3 analyses some properties of this solution and shows how they are linked to the model selection criterion described in Beghelli et al. (1993), which has shown remarkable robustness properties. Some concluding remarks are finally given in Section 4.
2. THE DYNAMIC FRISCH SCHEME Let us consider two scalar discrete-time real zeromean stationary random sequences Wl (t), W2(t) with rational power spectrum and related by a linear timeinvariant relation of the type
Despite the flexibility due to the mild assumptions of the Frisch scheme, its applications to real problems have been modest even if suitable numerical algorithms can reduce the computational burden associated to the identification of real multivariable dynamic systems (Beghelli et al., 1994). When the assumptions behind the scheme are violated, the determination of a single model in the family of possible ones requires the introduction of a suitable criterion. This paper investigates some of the many algebraical properties of the solution of this scheme and links
where P~(z) and P;(z) are relatively prime polynomials (z = forward-shift operator) with degrees nl and n2 respectively. We are interested in finding the linear difference equation which links the sequences Wl (t) and W2(t) from noisy measurements of the type Wl(t)
=Wl(t) + Wl(t)
W2(t) = W2(t)
+ Ui:z(t) ,
(2.2)
where Wl (t) and Ui:z(t) are the errors in the variables. In the scheme here considered Wl (t) and W2(t) are assumed to be zero-mean white noise processes with unknown variances O'~tul , O'~tu, , mutually uncorrelated and uncorrelated with the true signals Wl (t), W2(t).
This research has been supported by the Ministry for University and Scientific and Technologic Research, Rome, under project Model Identification. Colltro! Systems and Signal Processing 889
=
(i 0, ... , k) of the power spectrum of the signal Wl(t)
This scheme can be considered as an extension of the Frisch static scheme (Frisch. 1934) to dynamic systems. Representations of this type are usually called enurs-in-variables models and play an important role when the interests of the analysis concern the research of the true system behind the data rather than the choice of a model for predictive purposes. Note that the variables are treated in a symmetric way. so that the scheme allows identification of both causal and non-causaJ dynamic systems. Relation (2.1) can be written. without loss of generality, in the form RI
.tk(WlWl)
,=0
LP,~(t+i)=O.
roUl (0)
r oU1 (I)
rtiJl (k)
rtiJ 1(1)
r oU1 (0)
r';'l(k - I)
(~ -
[ rtiJl:(k)
rtiJ 1
I)
r oU1 (0)
(2.9) and the (k + I) x (k + I) symmetric Toeplitz matrix Ek(~~). built with the samples roU,(i). (i = 0, ... ,k) of the power spectrum of the signal W2(t)
n2
La,wl(t+i)+
=
(2.3)
E k (W2 W2)
,=0
=
r,;,,(O)
a"
where the time-invariant parameters Pi are the coefficients of the polynomials P;(z) and P;(z) respectively. with P;(z) monic and nl ~ n2. Denote now with 4J w1 (z) the power spectrum of the signal
r,;,,(l)
r,;"
r,;,,(O)
r,;,,(l)
r,;,,(~ -
[ r,;,,:(k)
Wl(t)
(k)
r,;,,(k - I)
I) (2.10)
In the same way. the (k+ I) x (k+ I) Toeplitz matrix Ek(W2Wd = E'[(WlW2) can be obtained with the samples r""';'1 (i), (i = -k, ... , k)
+00 4JW1 (z)=
L
r w1 (i)z-' ,
i=-CX)
(2.4)
where r w1 (i) = E[Wl(t)Wl(t + i») is the autocorrelation of the signal Wl(t): the power spectrum of Wl (t) is
E k (W2 Wl) = r"""'I(O) r"""'I(-I)
+00
4J w1 (z)
=
L
r"'l (i) z-,
+ U~1
r"""'1 (1) r"""'1 (0)
...
:::
r"""'1 (k)
r"""'I~k -
r"""'I(-k+l)...
r"""'I(-k)
r""", 1 (0)
(2.11 )
We can now define the (2k + 2) x (2k + 2) symmetric matrix t:k partitioned in the following way
In the same way. for the signals W2(t) and W2(t) we have +00 L
.
[
(2.5)
i=-oo
4J..,,(z)=
]
I)
•
Ek
r..,,(i)z-' ,
_ [Ek(WlWl) -.
_=-00
E k (W2 w d
Ek(WlW2)] M
•
(2.12)
L-k(W2 W2)
(2.6)
When the errors in the variables are absent it is possible to inspect the sequence of increasing dimension matrices (2.13)
and +00 -4Jw ,(z)=
L
r..,,(i)z-'+u~,
(2.7)
i=-oo
and it can be easily proved that Ek becomes singular (non-negative definite) when k is equal to or greater than nl.
where rw,(i) = E[~(t)W2(t+i») is the autocorrelation of the signal W2(t). The cross-spectrum between the signals Wl (t) and W2(t) is given by
Moreover
where
P~1 =[ao, ... ,an1-l,I,po, ... ,Pnl]T
+00
=- ~~:~ 4Jw, (z) =,~oo r W,Wl (i) z-i
(2.14)
is t~e (2nl + 2) vector whose entries are the coefficients of relation (2.3). Of course. when nl > n2 the coefficients Pn,+lt ... , Pn l are equal to zero.
,
(2.8)
where r"" oU l(i) = E[W2(t)Wl(t+i)] is the crosscorrelation between Wl(t) and ~(t). Since the white noises Wl (t) and ~(t) are uncorrelated with the signals Wl(t) and ~(t) and mutually uncorrelated. 41"""'1 (z)
En1 P~1 =O.
When the data are corrupted by additive noises the matrices E k • obtained with the noisy sequences Wl (t) and W2(t). are non-singular (positive definite) for any k. Under the previous assumptions. the matrices Ek satisfy the following relation
=4JtiJ, tiJ l (z).
Let us consider the (k+ I) x (k+ I) symmetric ToeplilZ matrix Ek (Wl wd. built with the samples r w1 (i ),
(2.15)
890
where
greater than a certain value (n1) have a single common point, then this point corresponds to the actual variances (O'ti;" 0';,) of the errors in the variables and the coefficients (p~, of the relation linking the data can be exactly evaluated. In Picci et al. (1993) it has been proved that, with the exception of singular cases, the common point is unique.
(2.16)
The degrees and the coefficients of P~(z) and P;(z) can be identified in the following way: - for a given k determine all matrices E" = [O'wJ"+l' O'w,I"+l]' i.e. all couples (0',;;" O'w,). such that E" is non-negative definite
3. PROPERTIES OF TIlE SOLUTION The point belonging to all curves described in Section 2 has some interesting properties. With reference to the curve associated to polynomials of degree (n1 + I), the following result can be proved.
- analyse how the solution set obtained at the previous step varies with k.
Theorem 3.1 - The vectors belonging to the subspace Im [P~,+l P:,+tllead to a pair of polynomials Q1(Z) and Q2(Z) with Q2(Z)/Q1(Z) = P;'(z)/Pi(z).
With reference to the first step in Beghelli et al. (1990) it has been proved that. for a given value of k. the solution set of relation (2.17) describes in the first quadrant of the plane (O'w" O'w,) a convex curve whose concavity faces the origin. Every point (0'w, , 0'w,) on the curve is associated to a couple of polynomials P 1(z), P 2 (z) with deg [P1(z)] = k. The coefficients Pie = [o~, ... , 0Z_l' I, p~ ,... ,p;]T of such polynomials can be obtained by means of the equation
Proof - A vector of coefficients given by linear combinations of the vectors P~'+l and P:,+l is of the type q = k 1 P~,+l + k 2 P~'+l = [k 1 0 0 , k 1 0
1
+ k2 0 0 ,
.. .k 1 + k 2 0 n ,-1, k 2 , k 1 Po, k 1 P1 + k 2 Po, ... ,k 1Pn, + k 2Pn,-1' k 2Pn,] (3.1)
and corresponds to the pair of polynomials
=(k 1z+ k2 )P;(z) Q2(Z) =(k 1 z + k 2 )P;(z) , Q1(Z)
Moreover, in Beghelli et al. (1990) it has been proved that the curves associated to increasing values of k approach the origin of the noise plane (0'w, , 0'W2) and each curve includes all subsequent ones.
and this proves the theorem.
When W1(t) and W2(t) are linked by relation (2.1) and are corrupted by additive white noises satisfying the previous assumptions. the point corresponding to the actual variances (O'~" O'~,) of the noises belongs to the curve associated to polynomials of degree k = n1' The linear relation obtained at this point is characterized by the actual coefficients p~, .
A similar property holds when curves associated to
polynomials of greater degrees are considered. With reference to a matrix E" it is also worth analysing the solutions of the equation det [E Ie diag [O'w, 1,,+1> O'w, /"+1]] = in the first quadrant of the plane (O'w"O'w,). Fig. I shows the solution set with reference to a matrix with k = I. The curve nearest to the origin leads to a non-negative definite matrix El expressed by relation (2.17). the next curve is associated to a non-definite matrix El with only one eigenvaIue less than zero, the number of negative eigenvaIues increases in a unit from one curve to the next. Consequently the following property holds.
°
It can easily be verified that this noise point is common also to the curve related to polynomials of degree (n1 + I); in fact the rank of matrix E n ,+l -diag [O'~, I n ,+2' 0'~Jn,+2] is equal to 2n1 + 2 and
Ker [E n ,+l - diag [O'~, I n ,+2, 0':U,In,+2]]= Im [P~,+l P~,+l] ,
(2.19)
Property 3.1 - Let us consider, with reference to k = I, a generic point A (O'~, O'~) in the noise plane; the straight line through A with unitary slope intersects the curves previously defined at points Al. A 2 , A 3 and A•. ThevaIues(O'~;-O'~)(i=1, ... ,4) are the eigenvaIues of matrix El -diag [O'~ 12 , O'~ 12 ] since, by subtracting from this matrix diag [(O'~; 0'~)12, (O'~i - 0'~)12]' all eigenvalues are shifted by (O'~i - O'~). Of course this property can be generalized for a generic value of k.
where
P~,+l = [00,"" 0n,-l, 1,0, Po, P~,+l = [O,oo, ... ,on,_l,I,O,po,
(3.2)
,Pn"O]T
,Pn,]T . (2.20) A similar property holds for all curves related to polynomials with greater degrees. In conclusion, from an identification point of view, when all curves associated to polynomials of degrees equal to or 891
CHI. The same property holds for KIC and CH2 •
A3 A2 A 1/
Let us consider a point Al (u~ I • u~ I) on the first curve of fig. 2 and let A 2 (U~, I u~,) be the point obtained by intersecting the second curve with the straight line from Al with unitary slope. The two smallest eigenvalues of matrix E n d I diag [UA;/nIH. uA;,In l + 2 ] are 0 and (UA; - UA;) respectively; the corresponding eigenvectors are orthogonal since they belong to a symmetric matrix. The entries of the eigenvector associated to the null eigenvalue are, if properly nonnalized, the coefficients of relation (2.3); thus the following conjecture can be fonnulated.
.' //
.4.-.. - - - - - - - - - - 1
...
./
Fig. 1. Solution set in the noise plane; x-axis: input noise variance; y-axis: output noise variance
Conjecture 3.2 - When point A I moves from K I C to CHI, crossing C, a discontinuity in the coefficients of relation (2.3) can be observed.
On the basis of the previous considerations the following property can also be stated.
When the assumptions of this identification scheme are not satisfied (time-invariance and linearity of the processes, uncorrelation of the additive white noises, etc.) no common point can be observed and a single solution can be obtained only by introducing suitable selection criteria. Analysis of the previous properties can be considered a starting point for the design of these criteria. Under mild assumption violations, different consistent algorithms have been tested and compared in Beghelli et al. (1993, 1994) for several signal-to-noise ratios. The best robustness has been exhibited by the Shifted Relation criterion, based on the peculiar rank deficiency properties of matrix E ndI described by relations (2.19) and (2.20).
Property 3.2 - The two curves nearest to the origin of the solution set of equation det [Enl +1 diag [UWI 1n l +2, U w, 1nl +21]= 0 share the point (u~ I ' u~,) since at this point the non-negative definite matrix Enl + I has two null eigenvalues.
4. CONCLUSIONS
Fig.
2.
In this paper the identification of linear relations linking two scalar discrete-time stationary random sequences with rational power spectrum has been considered. The identification procedure here considered is an extension to dynamic systems of the algebraic Frisch scheme. Some properties of the solution of this identification problem have been related to the design of a robust selection criterion leading to a single solution also when the assumptions of the scheme are not fulfilled.
Solution set in the noise plane; x-axis: input !l0ise variance; y-axis: output noise variance
Fig. 2 shows this situation: the curve characterized by eigenvalues greater than or equal to zero (End 1 ~ 0) is composed of the arcs KIC and CHI; in the same way, the curve characterized by an eigenvalue less than zero is composed of the arcs K 2 C, C H 2 • Note that no discontinuity in the derivative can be observed by going to K 2 C from CHI and to KIC from CH2 ; the arcs K 2 C and KIC can thus be considered the natural extensions of CHI and C H 2 • Moreover, numerical simulations induce the following conjecture.
5. REFERENCES Anderson, B.D.O. (1985). Identification of scalar errors-in-variables models with dynamics. Automatica, 21, 709-716. Anderson B.D.O. and M. Deistler (1984). Identifiability of dynamic errors-in-variables models. J. of TIme Series Analysis,S, 1-13.
Conjecture 3.1 - The eigenvector associated to the null eigenvalue of matrix E n d I diag [U w.!nl+2, U w,Ind2 ] shows no discontinuities when point (UW1l u w,) moves on K 2 C and
Beghelli, S., RP. Guidorzi and U. Soverini (1990). The Frisch scheme in dynamic system identification. AUlOmatica, 26, 171-176. 892
Beghelli. S., P. Castaldi. R.P. Guidorzi and U. Soverini (1993). A robust criterion for model selection in identification from noisy data. Proc. of the Int. Conf. on Systems Science. pp. 480-484. Las Vegas. USA.
intensive Methods in Control and Signal Processing. Prague.
BegheUi. S.• P. CastaIdi. R.P. Guidorzi and U. Soverini (1994). A comparison between different model selection criteria in Frisch scheme identification. Systems Science J.. 20, n.I.
Frisch. R. (1934). Statistical confluence analysis by means of complete regression systems, Pub. No. 5. Economics Institute. University of Oslo.
Deistler, M. and B.D.O. Anderson (1989). Linear dynamic errors-in-variables models: some structure theory. 1. of Econometrics. 41. 39-63.
Picci. G.• F. Gei and S. Pinzoni (1993). Identification of errors-in-variables with white measurement errors. Proc. of the second European Control Conference. pp. 2154-2158. Groningen. The Netherlands.
Beghelli, S.• P. CastaIdi, R.P. Guidorzi. U. Soverini and A. Stoian (1994). Practical implementation of the Frisch scheme in the identification of dynamical systems. IEEE European Workshop on Computer-
893