- Email: [email protected]
Signal space geometry
INFORMATION
SCIENCES
40, 15- 82 (1986)
75
Signal Space Geometry*+ R. S. BUCY University
of Southern California, Los Angeles, California 90007
ABSTRACT
The purpose of this paper is to show the usefulness of the concepts of differential geometry for the problem of multiple frequency determination or multiple frequency demodulation. The exterior algebra of Grassmann provides the tool which is natural for analysing the problem. We describe a distance between k-dimensional subspaces in R” and a geometric “least squares” which allows us to uniquely associate with each r-dimensional subspace determined by the observations an r-dimensional subspace which determines the r unknown frequencies.
1.
PROBLEM We consider
FORMULATION a signal
Z(r)=
i
a,elL’ + n ( t ) ,
a=1
where n(t) is a complex Gaussian zero mean process. The a, are zero mean complex Gaussian random variables, independent of the values of n (1) and each other. The f, are unknown frequencies to be estimated on the basis of observations of { Z(t,)},=, ,..., ,,. The { Z(t,)},=,,, ,1are independently observed N times. Now this situation can be described mathematically as
Z’ = xal,d( f,) +n’,
i=l
,.... N,
*This research was supported in part by Thomson CSF-DTAS. University of Nice, Nice, France. + This paper is dedicated to the memory of Richard Bellman. OElsevier Science Publishing Co., Inc. 1986 52 Vanderbilt Ave., New York. NY 10017
Valbonne
France,
and the
0020-0255/‘86j$O3.50
76
R. S. BUCY
where
and i denotes
the i th observation.
We assume that
En’nJ* = S,,cI
(1.0)
where I is the identity matrix and * denotes conjugate
is a sufficient
statistic for estimating,
Eii = R$iEZZ*
= L(f) i’:y
fi,.
transpose.
It is clear that
. , f,. We note that
‘..
E,;r,lL*(f)+cI,
(1.1)
where
L(f) = [d(fd~-~d(f,)l DEFINITION1. The problem
observable
2.
represented by (1.0) will be called K frequency iff the map f + L(f) is injective when the domain is f, E K c R.
GEOMETRY
Given a n-dimensional real space V, recall that Ap( I’), the set of antisymtensors of weight p, are called p-vectors, while the set A, (I’) of antisymmetric covariant tensors of weight p are called r-forms. See [l] and [2] for exterior algebra details.
metric contravariant
DEFINITION2. An r-vector k is simple iff 3a,,. . ,a, E V 3 A . . . A a,. A denotes exterior product.
k = a, A a,
SIGNAL
SPACE GEOMETRY
77
DEFINITION 3. G,,,, is the set of all linear subspaces of V of dimension
k.
NOTATION. Let
4=(Y) ( II combined
a=q-1
r),
For a E R, [a] is the greatest integer not greater than u. REMARK. G,,, is isomorphic to the set of simple r-vectors; see Cartan [2]. Using Plucker coordinates G,, r can be identified as a closed algebraic variety in P”.
EXAMPLE. G4,2 z P’( R) (7 {
Ps( R) is Sdimensional
x0x5
=
x,x4
-
x2.x3}.
projective space with coordinates
Notice that G,,, is isomorphic to a subspace of P” and to the set of all n x r matrices of rank r with the equivalence relation A - B iff 3C nonsingular r x r such that A = BC. In the last isomorphism A E G,, I corresponds to all matrices with columns which span A. DEFINITION 4. { t, .
, t, } is symmetric iff 3 T 3
where t; = t, - T We remark that equally spaced sampling produces symmetric
THEOREM 1. If {t, },=1, ,,,,, is symmetric, eigenvector of R, then
then yR.9
if
n is even ,
if
nisodd.
e=
where I E C[n’21, p = 0 or 1.
sets:
= R, and if e is an
R. S. BUCY
78 at { ti }i = 1,, (n result in the same covariance
Proof. Observations tions at { t:}I_l,..
as observa-
,n:
R 1.m = EZ(t;)Z(r:,)*
= EZ( - t;+l_,)Z*(
= -@(L-,)~*(~;+,-,
- t;+l_m)
= L-l,n+lhn~
where we assume that {r;} and {t;} are indexed with respect to the order of the elements. This proves the first assertion. Now if e is an eigenvector of R, then Re = Xe, with X real, as R = R*. Since JV = I, YRY9e
= Me,
or Rye = Me, RAi = AA?; hence J% = ye. But ee* = 1 or IyI = 1, and so e can be chosen so that the second conclusion follows, since complex eigenvectors are unique up to an arbitrary phase factor.
If { t, } is symmetric,
and eigenvectors
then
of R, e in C”, where
d,( t) = elk’, can be represented
by a real vector as
n even,
1 RL,
Red&r)
Im 1,
ImA i, = R4$3) I=%&) where j3 = [n/2].
,
7 Rk la Im 1,
n odd,
SIGNAL
SPACE GEOMETRY
LEMMA1. Suppose i,
79
are linearly independent.
Span[e,,...,e,]
If
=Span[d(f,),...,d(f,)]
in the vector space C” over the complex field, then
Span[C, ,..., &] =Span[ci(f,)
,..., i(f,)]
in the vector space R[(“’ ‘)/*I over the real field. Proof.
If
.@ai= i: y,ITd(f,), where IId is the projection on the first [n/2] components of d(f,), which annihilates the last [(n + 1)/2] components. Consequently y, is real, as we assume IId are linearily independent over the reals. This shows inclusion one way, and a symmetric argument proves the equality. Let A and Z be two r-planes through the origin in R” and hence elements a,], where Span[a, ,..., a,] = A, and Y = of G,,.,; and let X=[a,,a,,..., b,], where Span[b,,. . . ,br] = Z. Now we can define the distance P+,bl,..., between A and I: as
(2.0) where X, are roots of
Notice that A, are canonical correlation coefficients, and cos-‘& are the angles that the r-planes through the origin determined by A and Z make with each other. REMARK. d( A, 2) is the same as the invariant G/H=SO(n)/SO(r)xSO(n-r),
metric on
80
R. S. BUCY
where SO(n) is the special orthogonal
group and
(dr)* = trace B’B. Now B is determined
by
1
B’ 0 /XI
’
of g - h, where B is Ix r and I = n - r. Further, Lie algebras of G and H respectively; see [5]. an element
g and h are the
LEMMA 2. Let X be an n X r matrix with columns r< n.
x’,...,xr, Then
det( X’X) = ]]x’ AX* A
.
Ax’lj2dzfa.a
where a is the uector of Plucker coordinates of x1 A x2 A ’ . . A xr in Rq. Proof.
By the Shur relations (see [4]),
W=detX’X=I
_“X
yI=deta,,.
Now
where II is a permutation of (1,2,3,. . . , n + m). Note that nonvanishing terms in W correspond to ff’s for which II]~t,,,,,,,) is a l-l function into (m+l,..., into (m+l,...,n+m)fl{jlj=II(i), i= n + m) and qm+l,...,n+m) 1 ,..., m }’ U { 1,2,3,. . . , m }. Each nonzero term in is the square of a Plucker coordinate. Let X and Y be n X r of rank r. Then THEOREM 2.
)Kvx-
X,Y( Y'Y)-lYxI
=0
(2.1)
SIGNAL
81
SPACE GEOMETRY
is the po!vnomial
c
(X*+1)‘-“W,t
< 11,< a, <
I1<
lQi,
,1=,
R”, where
Y’=[I,.y,~O,“_.,,x,].
,i~,
a,_l,
r+l
a, ~~are the Plucker coordinates
of X in the coordinate
system of
Proof.
where P is the projection of R” onto the subspace spanned Y. Notice that if I - P = Q then
p ,,
let
Z = (X1 + \/X2 + 1 Q),
and
of
P.
(iZ+JX2+iQ)‘=A*Z-
now
by the columns
if the Plucker
coordinates
of
X are
,kcr,. ar_k, then those of Z are
+1y
(2 Consequently
k)/2AkP,,
,1(~, (I, *_
our results follows from the lemma.
See [3] for the special case when r = 2, and geometric insight. 3.
ESTIMATING
THE FREQUENCIES
As N + co the eigenvalues x,(N) of i have the property that “x,(N) + c, cx=l , , n - r, while x,(N) 4 k, > c, a > n - r, where the X, are indexed in nondecreasing order. The estimation procedure consists of finding min d(A,Z)
=d(A.x,,).
XEDK
D,={ZEG,,,,)3f,EK;
A = span[e,_,+,,...,&] Here e, are the eigenvectors
Z=M},
and
M=
spa&$
fi),...,i(
Z=sp~[~(f,)....,~(f,)].
of f corresponding
to x,(N).
f,,].
82
R. S. BUCY
Of c:urse the minimum is achieved, as D, is compact. We assume that 9& = R, symmetrizing if necessary. Notice that A may fall outside DK because of errors due to the discrepancy between A and R and/or numerical errors in finding the eigenvalues. These results can be summarized as: 3. Suppose that { t, } is symmetric and K observable, and further
THEOREM
that d( t,) are linearly independent termined
correspon& (2.0). 4.
over the reals.
by the eigenvectors corresponding to a unique point of D,,
Further,
Then every r-plane A de-
to the r largest eigenvalues of D,
the closest point of DK to A in the distance
this point of DK determines 2,.
CONCLUSIONS
We have described a geometric least squares method for estimating multiple frequencies, by finding the shortest distance from a point to a set in a Grassmannian with a coordinate free metric. Other metrics are possible, and one such is being investigated in [6], where-interestingly-Shubert cycles arise. Various robustness questions are natural and remain to be clarified. It is clear that Bayesian estimates are possible, and we intend to devote a future paper to this subject. Another possible generalization is to symmetric spaces. The author has benefited from conversations with Professor Mark Green of U. C. L.A.
REFERENCES 1. W. Thirring, Classical Dynamical Systems, Springer, New York, 1978. 2. E. Cartan, Les Sysrkmes Diffbentiels ExtCrieurs et Leurs Applicaiions Gomefriques. Hermann, Paris, 1971. 3. H. Hotelling, Relations between two sets of variables, Biometrika 28:321-377 (1936). 4. F. R. Gantmacher, Matrix Theov, Chelsea, New York, 1959. 5. S. Helgason, Differential GeometT, Lie Groups. and Symmefric Spaces, Academic, New York, 1978. 6. S. Leung, Thesis, Aerospace Engineering Dept., Univ. of Southern California, to appear. 7. R. Schmidt, Thesis, Electrical Engineering Dept., Stanford Univ., 1981. Received 3 April 1986; revised I5 April 1986.
SCIENCES
40, 15- 82 (1986)
75
Signal Space Geometry*+ R. S. BUCY University
of Southern California, Los Angeles, California 90007
ABSTRACT
The purpose of this paper is to show the usefulness of the concepts of differential geometry for the problem of multiple frequency determination or multiple frequency demodulation. The exterior algebra of Grassmann provides the tool which is natural for analysing the problem. We describe a distance between k-dimensional subspaces in R” and a geometric “least squares” which allows us to uniquely associate with each r-dimensional subspace determined by the observations an r-dimensional subspace which determines the r unknown frequencies.
1.
PROBLEM We consider
FORMULATION a signal
Z(r)=
i
a,elL’ + n ( t ) ,
a=1
where n(t) is a complex Gaussian zero mean process. The a, are zero mean complex Gaussian random variables, independent of the values of n (1) and each other. The f, are unknown frequencies to be estimated on the basis of observations of { Z(t,)},=, ,..., ,,. The { Z(t,)},=,,, ,1are independently observed N times. Now this situation can be described mathematically as
Z’ = xal,d( f,) +n’,
i=l
,.... N,
*This research was supported in part by Thomson CSF-DTAS. University of Nice, Nice, France. + This paper is dedicated to the memory of Richard Bellman. OElsevier Science Publishing Co., Inc. 1986 52 Vanderbilt Ave., New York. NY 10017
Valbonne
France,
and the
0020-0255/‘86j$O3.50
76
R. S. BUCY
where
and i denotes
the i th observation.
We assume that
En’nJ* = S,,cI
(1.0)
where I is the identity matrix and * denotes conjugate
is a sufficient
statistic for estimating,
Eii = R$iEZZ*
= L(f) i’:y
fi,.
transpose.
It is clear that
. , f,. We note that
‘..
E,;r,lL*(f)+cI,
(1.1)
where
L(f) = [d(fd~-~d(f,)l DEFINITION1. The problem
observable
2.
represented by (1.0) will be called K frequency iff the map f + L(f) is injective when the domain is f, E K c R.
GEOMETRY
Given a n-dimensional real space V, recall that Ap( I’), the set of antisymtensors of weight p, are called p-vectors, while the set A, (I’) of antisymmetric covariant tensors of weight p are called r-forms. See [l] and [2] for exterior algebra details.
metric contravariant
DEFINITION2. An r-vector k is simple iff 3a,,. . ,a, E V 3 A . . . A a,. A denotes exterior product.
k = a, A a,
SIGNAL
SPACE GEOMETRY
77
DEFINITION 3. G,,,, is the set of all linear subspaces of V of dimension
k.
NOTATION. Let
4=(Y) ( II combined
a=q-1
r),
For a E R, [a] is the greatest integer not greater than u. REMARK. G,,, is isomorphic to the set of simple r-vectors; see Cartan [2]. Using Plucker coordinates G,, r can be identified as a closed algebraic variety in P”.
EXAMPLE. G4,2 z P’( R) (7 {
Ps( R) is Sdimensional
x0x5
=
x,x4
-
x2.x3}.
projective space with coordinates
Notice that G,,, is isomorphic to a subspace of P” and to the set of all n x r matrices of rank r with the equivalence relation A - B iff 3C nonsingular r x r such that A = BC. In the last isomorphism A E G,, I corresponds to all matrices with columns which span A. DEFINITION 4. { t, .
, t, } is symmetric iff 3 T 3
where t; = t, - T We remark that equally spaced sampling produces symmetric
THEOREM 1. If {t, },=1, ,,,,, is symmetric, eigenvector of R, then
then yR.9
if
n is even ,
if
nisodd.
e=
where I E C[n’21, p = 0 or 1.
sets:
= R, and if e is an
R. S. BUCY
78 at { ti }i = 1,, (n result in the same covariance
Proof. Observations tions at { t:}I_l,..
as observa-
,n:
R 1.m = EZ(t;)Z(r:,)*
= EZ( - t;+l_,)Z*(
= -@(L-,)~*(~;+,-,
- t;+l_m)
= L-l,n+lhn~
where we assume that {r;} and {t;} are indexed with respect to the order of the elements. This proves the first assertion. Now if e is an eigenvector of R, then Re = Xe, with X real, as R = R*. Since JV = I, YRY9e
= Me,
or Rye = Me, RAi = AA?; hence J% = ye. But ee* = 1 or IyI = 1, and so e can be chosen so that the second conclusion follows, since complex eigenvectors are unique up to an arbitrary phase factor.
If { t, } is symmetric,
and eigenvectors
then
of R, e in C”, where
d,( t) = elk’, can be represented
by a real vector as
n even,
1 RL,
Red&r)
Im 1,
ImA i, = R4$3) I=%&) where j3 = [n/2].
,
7 Rk la Im 1,
n odd,
SIGNAL
SPACE GEOMETRY
LEMMA1. Suppose i,
79
are linearly independent.
Span[e,,...,e,]
If
=Span[d(f,),...,d(f,)]
in the vector space C” over the complex field, then
Span[C, ,..., &] =Span[ci(f,)
,..., i(f,)]
in the vector space R[(“’ ‘)/*I over the real field. Proof.
If
.@ai= i: y,ITd(f,), where IId is the projection on the first [n/2] components of d(f,), which annihilates the last [(n + 1)/2] components. Consequently y, is real, as we assume IId are linearily independent over the reals. This shows inclusion one way, and a symmetric argument proves the equality. Let A and Z be two r-planes through the origin in R” and hence elements a,], where Span[a, ,..., a,] = A, and Y = of G,,.,; and let X=[a,,a,,..., b,], where Span[b,,. . . ,br] = Z. Now we can define the distance P+,bl,..., between A and I: as
(2.0) where X, are roots of
Notice that A, are canonical correlation coefficients, and cos-‘& are the angles that the r-planes through the origin determined by A and Z make with each other. REMARK. d( A, 2) is the same as the invariant G/H=SO(n)/SO(r)xSO(n-r),
metric on
80
R. S. BUCY
where SO(n) is the special orthogonal
group and
(dr)* = trace B’B. Now B is determined
by
1
B’ 0 /XI
’
of g - h, where B is Ix r and I = n - r. Further, Lie algebras of G and H respectively; see [5]. an element
g and h are the
LEMMA 2. Let X be an n X r matrix with columns r< n.
x’,...,xr, Then
det( X’X) = ]]x’ AX* A
.
Ax’lj2dzfa.a
where a is the uector of Plucker coordinates of x1 A x2 A ’ . . A xr in Rq. Proof.
By the Shur relations (see [4]),
W=detX’X=I
_“X
yI=deta,,.
Now
where II is a permutation of (1,2,3,. . . , n + m). Note that nonvanishing terms in W correspond to ff’s for which II]~t,,,,,,,) is a l-l function into (m+l,..., into (m+l,...,n+m)fl{jlj=II(i), i= n + m) and qm+l,...,n+m) 1 ,..., m }’ U { 1,2,3,. . . , m }. Each nonzero term in is the square of a Plucker coordinate. Let X and Y be n X r of rank r. Then THEOREM 2.
)Kvx-
X,Y( Y'Y)-lYxI
=0
(2.1)
SIGNAL
81
SPACE GEOMETRY
is the po!vnomial
c
(X*+1)‘-“W,t
< 11,< a, <
I1<
lQi,
,1=,
R”, where
Y’=[I,.y,~O,“_.,,x,].
,i~,
a,_l,
r+l
a, ~~are the Plucker coordinates
of X in the coordinate
system of
Proof.
where P is the projection of R” onto the subspace spanned Y. Notice that if I - P = Q then
p ,,
let
Z = (X1 + \/X2 + 1 Q),
and
of
P.
(iZ+JX2+iQ)‘=A*Z-
now
by the columns
if the Plucker
coordinates
of
X are
,kcr,. ar_k, then those of Z are
+1y
(2 Consequently
k)/2AkP,,
,1(~, (I, *_
our results follows from the lemma.
See [3] for the special case when r = 2, and geometric insight. 3.
ESTIMATING
THE FREQUENCIES
As N + co the eigenvalues x,(N) of i have the property that “x,(N) + c, cx=l , , n - r, while x,(N) 4 k, > c, a > n - r, where the X, are indexed in nondecreasing order. The estimation procedure consists of finding min d(A,Z)
=d(A.x,,).
XEDK
D,={ZEG,,,,)3f,EK;
A = span[e,_,+,,...,&] Here e, are the eigenvectors
Z=M},
and
M=
spa&$
fi),...,i(
Z=sp~[~(f,)....,~(f,)].
of f corresponding
to x,(N).
f,,].
82
R. S. BUCY
Of c:urse the minimum is achieved, as D, is compact. We assume that 9& = R, symmetrizing if necessary. Notice that A may fall outside DK because of errors due to the discrepancy between A and R and/or numerical errors in finding the eigenvalues. These results can be summarized as: 3. Suppose that { t, } is symmetric and K observable, and further
THEOREM
that d( t,) are linearly independent termined
correspon& (2.0). 4.
over the reals.
by the eigenvectors corresponding to a unique point of D,,
Further,
Then every r-plane A de-
to the r largest eigenvalues of D,
the closest point of DK to A in the distance
this point of DK determines 2,.
CONCLUSIONS
We have described a geometric least squares method for estimating multiple frequencies, by finding the shortest distance from a point to a set in a Grassmannian with a coordinate free metric. Other metrics are possible, and one such is being investigated in [6], where-interestingly-Shubert cycles arise. Various robustness questions are natural and remain to be clarified. It is clear that Bayesian estimates are possible, and we intend to devote a future paper to this subject. Another possible generalization is to symmetric spaces. The author has benefited from conversations with Professor Mark Green of U. C. L.A.
REFERENCES 1. W. Thirring, Classical Dynamical Systems, Springer, New York, 1978. 2. E. Cartan, Les Sysrkmes Diffbentiels ExtCrieurs et Leurs Applicaiions Gomefriques. Hermann, Paris, 1971. 3. H. Hotelling, Relations between two sets of variables, Biometrika 28:321-377 (1936). 4. F. R. Gantmacher, Matrix Theov, Chelsea, New York, 1959. 5. S. Helgason, Differential GeometT, Lie Groups. and Symmefric Spaces, Academic, New York, 1978. 6. S. Leung, Thesis, Aerospace Engineering Dept., Univ. of Southern California, to appear. 7. R. Schmidt, Thesis, Electrical Engineering Dept., Stanford Univ., 1981. Received 3 April 1986; revised I5 April 1986.