- Email: [email protected]
3 Geometry of Hilbert Space
3
Geometrw of Hilbert Space
3.1 The notion of Banach space abstracts many of the important properties of finite-dimensional linear spaces. The geometry of a Banach space can, however, be quite different from that of Euclidean n-space; for example, the unit ball of a Banach space may have corners, and closed convex sets need not possess a unique vector of smallest norm. The most important geometrical property absent in general Banach spaces is a notion of perpendicularity or orthogonality. In the study of analytic geometry we recall that the orthogonality of two vectors was determined analytically by considering their inner (or dot) product. In this chapter we introduce the abstract notion of an inner product and show how a linear space equipped with an inner product can be made into a normed linear space. If the linear space is complete in the metric defined by this norm, then it is said to be a Hilbert space. This chapter is devoted to studying the elementary geometry of Hilbert spaces and to showing that such spaces possess many of the more pleasant properties of Euclidean nspace. We will show, in fact, that a finite dimensional Hilbert space is isomorphic to Euclidean n-space for some integer n.
3.2 Definition An inner product on a complex linear space Y is a function cp from Y x 9 to d= such that: (1) c p ( ~ ~ f ~ + ~ z f ~ , g ) = ~ ~ r p ( f ~ y g )for + ~ za lcypa (2 finZ ,6 ~ )and fbfZY9
in 9 ;
63
64
3 Geomerry of Hilbert Space
(4) q ( f , f ) >, 0 forfin Y and q ( J f ) = 0 if and only i f f = 0.
A linear space equipped with an inner product is said to be an inner product space. The following lemma contains a useful polarization identity, the importance of which lies in the fact that the value of the inner product p is expressed solely in terms of the values of the associated quadratic form defined by +(f)= q ( f , f ) forfin 9.
+
3.3 Lemma If
Y is an inner product space with the inner product
p, then
for f and g in 9. Proof Compute.
An inner product is usually denoted (,), that is, ( A g ) = cp(f,g) for f and g in 9. 3.4 Definition If Y is an inner product space, then the norm 11 1 on 2’ associated with the inner product is defined by llfll = (f,f)”forfin 9.
The following inequality is basic in the study of inner product spaces. We show that the norm just defined has the required properties of a norm after the proof of this inequality.
3.5 Proposition (Cauchy-Schwarz Inequality) I f f and g are in the inner product space 9, then I(f,g)I
IlPll Ilsll.
Proof For f and g in Y and I in @, we have
14’ 119112+ 2ReCJ(f,g)l + llfll’
= (f+Ig,f+&d
= Ilf+IgJ122 0.
Setting 2 = te“, where t is real and eie is chosen such that e - ” ( J g ) 2 0, we obtain the inequality llgl12t2+ 21(Ag)lt
+ llfll’
2 0.
The Cauchy-Schwarz Inequality
65
Hence the quadratic equation llg112~2 + 2l(f,g)lt
+ llfll’ = 0
in f has at most one real root, and therefore its discriminant must be nonpositive. Substituting we obtain C2I(f,9)1I2- 4 I M 2 Ilfl12 < 0,
from which the desired inequality follows.
3.6 Observe that the property in the preceding proof. 3.7 Proposition If on 9.
(Lf)= 0 implies that f = 0 was not needed
Y is an inner product space, then II.11 defines a norm
Proof We must verify properties (1)-(3) of Definition 1.3. The fact that = 0 if and only i f f = 0 is immediate from (4) (Definition 3.2) and thus (1) holds. Since
/if11
lll.fll = (l..If)” = ( I x ( f , f ) ) ” =
1,Il llfil
for I in C and f i n 9,
we see that (2) holds. Lastly, using the Cauchy-Schwarz inequality, we have llf+gll = Cf+g, f + g ) =
(Lf)+ (h9) + ( s , f >+ tg9 s)
llfil’ + 119Il’ + 2Re(f,g) IlfI12+ 1I91l2+ 21(f,g)l illfll2 + lI91l2 + 2 llfllI1911 G (Ilfll + 11g11)2 for f and g in 9. Thus (3) holds and I[. 11 is a norm. =
3.8 Proposition In an inner product space, the inner product is continuous. Proof Let Y be an inner product space and {fa}ueAand {ga}uoAbe = f a n d limGEAga = g. Then nets in 9 such that limaEAf&
I(f,
9) - ( Agu)l G
and hence limaEA(A,ga) = (L 9).
I(f-fh, g>l+ g-gu>I Ilf-Lll 1 1 ~ 1 1+ IlfaII llg-gull~
3.9 Definition ,In the inner product space Y two Yectors f and g are said to be orthogonal, denoted fL g, if df, g) = 0. A subset 9’of Y is said to be orthogonal if f lg for f and g in Y and orthonormal if, in addition, llfll = I forfin 9’.
66
3 Geometry of Hilbert Space
This notion of orthogonality generalizes the usual one in Euclidean space. It is now possible to extend various theorems from Euclidean geometry to inner product spaces. We give two that will be useful. The first is the familiar Pythagorean theorem, while the second is the result relating the lengths of the sides of a parallelogram to the lengths of the diagonals. 3.10 Proposition (Pythagorean Theorem) If {fl,fi,. ..&} is an orthogonal subset of the inner product space 9, then
Proof Computing, we have
3.11 Proposition (Parallelogram Law) I f f and g are in the inner product space 9, then llf+g112 + Ilf-gll’
= 211fl12 + 211gl12
Proof Expand the left-hand side in terms of inner products. As in the case of normed linear spaces the deepest results are valid only if the space is complete in the metric induced by the norm.
3.12 Definition A Hilbert space is a complex linear space which is complete in the metric induced by the norm. In particular, a Hilbert space is a Banach space. 3.13 Examples We now consider some examples of Hilbert spaces. For n a positive integer let @“ denote the collection of complex ordered n-tuples {x : x = (xI,xz, ...,x,), xi E C}. Then C“ is a complex linear space for the coordinate-wise operations. Define the inner product (,) on @” such that (x, y ) = C;= x i ji.The properties of an inner product are easily verified and the associated norm is the usual Euclidean norm I / x / I 2= (C;=I 1.~~1’)”. To verify completeness suppose { x ’ ’ } ~ = ~is a Cauchy sequence in C”. Then
Examples of Hilbert Spaces: C", 12, L2,and H a 67
since Jxjl'-xim) < I(x~-x'"J~~, it follows that {x:}?=~ is a Cauchy sequence in @ for I < i < n. If we set x = ( x , , x2,.. ., xk), where xi = limk-,a xjl', then x is in @" and limk-,a 2 = x in the norm of 63". Thus @" is a Hilbert space. The space @" is the complex analog of real Euclidean n-space. We show later in this chapter, in a sense to be made precise, that the @"'s are the only finite-dimensional Hilbert spaces. 3.14 We next consider the "union" of the V's. Let 8 be the collection of complex functions on Z+ which take only finitely many nonzero values. With respect to pointwise addition and scalar multiplication, Y is a complex linear space. Moreover, (f,g) = x.,"=of(n)s(n)defines an inner product on 9, where the sum converges, since all but finitely many terms are zero. Is Y a Hilbert space? I t is if 8 is complete with respect to the metric induced by the norm llfl12 = (xF=o lf(n)I2)". Consider the sequence {h}F==l contained in 9, where
One can easily show that {A}?=, is Cauchy but does not converge to an element of 9. We leave this as an exercise for the reader. Thus 8 is not a Hilbert space. The space 8 is not a Hilbert space because it is not large enough. Let us enlarge it to obtain our first example of an infinite-dimensional Hilbert space. (This example should be compared to Example I . 15.) Let 1 2 ( Z ' ) denote the collection of all complex functions cp on Z+ such that Z:,"=o Icp(n)12 < co.Then 12(Z') is a complex linear space, since
3.15
I(f+g)(n)l2 G 2 If(n>I2+ 2 I s(n)12.
-
F o r f a n d g in 12(Z'), define ( J g ) = ~ . , " = o f ( n ) g ( n )Does . this make sense, that is, does the sum converge? For each N in Z+, the n-tuples
FN = (If(O)I, If(l)l,
..., IfCWI)
and
G N = (Is(O)l,Is(l)l, ..., lg(N)l)
lie in CN.Applying the Cauchy-Schwarz inequality, we have
c If(n)s(n>I N
fl=O
= I(FN,GN)I G
IIFNII
IlGNlI
68
3 Geometry of Hilbert Space
-
Thus the series C:=of(n)g(n) converges absolutely. That ( ,) is an inner product follows easily. To establish the completeness of /2(Z') in the metric given by the norm I) ]I2, suppose If'})= is a Cauchy sequence in /'(Z'). Then for each n in Z', we have
If'W -f'(n)I
< Ilfk-f'112
and hence {fk(~)}:=l is a Cauchy sequence in C for each n in H'. Define the function f on E + to be f ( n ) = limk+ .f"(n). Two things must be shown : that f i s in /'(Z') and that limk+,wlif-fk]12 = 0. Since {fk}k",, is a Cauchy sequence, there exists an integer K such that for k 3 K we have l f ' - , f K I( < 1. Thus we obtain
< and hence f is in /2(Z+). Moreover, given E > 0, choose M such that k , j 1, M implies llj'k-f'll < E . Then for k 2 M and any N , we have N
C
n=O
If(n>-fk(n)12
=
<
iim
N
1 Ifj(n)-fk(n)12
j-r n=O
l i m ~ u p I / f j - f ~ ( (<~ '.e2. j- m
Since N is arbitrary, this proves that lif-fkl12 a Hilbert space.
and therefore
/'(Z+)is
3.16 The Space L2 In Section 1.44 we introduced the Banach spaces L' and L" based on a measure space ( X , 9 , p ) . We now consider the corresponding L2 space, which happens to be a Hilbert space. We begin by letting 9'denote the set of all measurable complex functions f on X which satisfy J x dp < co. Since the inequality lf+gl' < 21f12+ 219l2 is valid for arbitrary functionsfand g on A', we see that 2" is a linear space for pointwise addition and scalar multiplication. Let N 2 be the subspace of functions f in Y 2for which jxIfl' dp = 0, and let L2 denote the quotient linear space 2 ' 2 / ~ ~ 2 .
If/'
Examples of Hilberr Spaces: C", 12, L2,and H 2
69
Iff and g are in 9', then the identity
If4 = !?{(If1 + lsl)'
fa
-
Ifl' - 191'1
shows that the function is integrable. If we define cp(J g) = Jx,fijdp for f a n d g in Y 2 ,then cp has all the properties of an inner product except one; namely, cp(f,f) = 0 does not necessarily implyf= 0. By the remark following the proof of the Cauchy-Schwarz inequality, that inequality holds for cp. Thus, i f L f ' , g, and g' are functions in 2" such thatf-f' and g-g' belong to .N2,then
+ Icp(f'99-9')l cp (f-f't f - f ' ) cp (999)
Icp(L9) - cp(SI,g')I G Icp(f-SI99)I
G
+ cp (f', f')cp (9- g'9 9 - 8')= 0. Therefore, cp is a well-defined function on L2. Moreover, if cp([f], [f])= 0, then Jx dp = 0 and hence [f]= [O]. Thus cp is an inner product on L2 and we will denote it from now o n in the usual manner. Furthermore, the associated norm on L2 is defined by l i [ f ] l 1 2 = (Jx l f 1 2 dp)". The only problem remaining before we can conclude that L2 is a Hilbert space is the question of its completeness. This is slightly trickier than in the case of L'. We begin with a general inequality. Takefin P2and define g on X such that g(x) =f(x)/lf(x)I iff(x) # 0 and g ( x ) = 1 otherwise. Then g is measurable, Jg(x)J = I , and = Moreover, applying the Cauchy-Schwarz inequality, we have
If]'
fa If/.
llftll
=
J' If1 4 J' fa& X
=
X
= I(L9)I
G llfll211g112 =
llfll2.
Therefore, llfll G llfllz forfin Y 2 . We now prove that Lz is complete using Corollary 1.10. Let {[fJ}."=l be a sequence in Lz such that C,"= Ii[fn]112 < M < CO. By the precedin,g inequality l [ f. ] l l < M , and hence, by the proof in Section 1.44, there exists f in 2 '' such that C , " = l f , ( x )= f ( x ) for almost all x in X . Moreover
x:,"=I
and since IimN+RI ~ , , " = , f . ( x ) 1=2 If(x)12 for almost all x in X , it follows from Fatou's lemma that is integrable and hencefis in 9''. Moreover,
'/fI
70
3 Geometry of Hilbert Space
since the sequence that k = IirnN+=
{(x.n”= =;}’)l.fI
(xy=If, I)’ I
I
is monotonically increasing, it follows is an integrable function. Therefore,
Jx;=,,,+
by the Lebesgue dominated convergence theorem, since I f,I2 ,< k for all N and lim,,,,, fn(x)12 = 0 for almost all x in X . Thus L2 is a Hilbert space. Lastly, we henceforth adopt the convention stated in Section 1.44 for the elements of L2; namely, we shall treat them as functions.
3.17 The Space H 2 Let T denote the unit circle, p the normalized Lebesgue measure on T, and L2(T) the Hilbert space defined with respect to p . The corresponding Hardy space H 2 is defined as the closed subspace
where
xm is the function x,,(e”) = ein‘. A slight variation of this definition ( ~ E L ’ ( T ) (AX,,) : =0
is
for n = - I , -2, - 3 ,... }.
3.18 Whereas in Chapter 1 after defining a Banach space we proceeded to determine the conjugate space, this is unnecessary for Hilbert spaces since we show in this chapter that the conjugate space of a Hilbert space can be identified with the space itself. This will be the main result of this chapter. We begin by extending a result on the distance to a convex set to subsets of Hilbert spaces. Although most proofs of this result for Euclidean spaces make use of the compactness of closed and bounded subsets, completeness actually suffices.
3.19 Theorem If X is a nonempty, closed, and convex subset of the Hilbert space H , then there exists a unique vector in X of smallest norm. Proof If S = inf{ Ilfll :f E X ) , then there exists a sequence in X such that limn+z 11f.1 = 6. Applying the parallelogram law to the vectorsf,/2 and fm/2, we obtain
Since X is convex, (f,+fm)/2 is in .X and hence li(f,+fm)/2(/22 6’. There-
Examples oJHilberr Spaces: C, 12, L2,and
H 2 71
/I' < 2 ilf,(I'+ 2 ) I f r n I/'- 46', which implies < 26' + 26' - 46' = 0. Iim sup /if,-f,/I'
fore, we have //J, -f ,
n.m-3o
Thus { fn},"=, is a Cauchy sequence in .X and from the completeness of SEP and the fact that X is a closed subset of if we obtain a vector f in .X such that Iirnn+*f, = J Moreover, since the norm is continuous, we have Having proved the existence of a vector in .X of smallest norm we now consider its uniqueness. Suppose f and g are in X with / / f l l = 1/g//= 6. Again using the parallelogram law, we have
since
I\(f + g ) / 2 ( (>, 6. Therefore, f
= g and uniqueness is proved.
If II is a plane and / is a line in three-space perpendicular to n and both IT and I contain the origin, then each vector in the space can be written uniquely as the sum of a vector which lies in n and a vector in the direction of 1. We extend this idea to subspaces of a Hilbert space in the theorem following the definition. 3.20 Definition If -& is a subset of the Hilbert space %',then the orthogonal complement of denoted A'', is the set of vectors in if orthogonal to every vector in A'.
Clearly ~ i % "is a closed subspace of #, possibly consisting ofjust the zero vector. However, if is not the subspace ( 0 ) consisting of the zero vector alone, then A" F X . 3.21 Theorem If dl is a closed subspace of the Hilbert space %' and j' is a vector in #, then there exist unique vectors g in J" and h in ,KL such that
f = g+h. Proof If we set X = i f - k : k E A!}, then .X is a nonempty, closed and convex subset of X . Let /7 be the unique element of .X with smallest norm whose existence is given by the previous theorem. If k is a unit vector in A', then It-(/i,k)k is in X , and hence ))/i)/'
G /lh-(/i,k)kl[2 =
<
/ / / i ) [ '-
G ) ( h , k ) - (/i,k)(h,)+ ( / i , k ) ~ ) I \ k \ l ' .
Therefore, /(h,k)(' 0, which implies (I7,k) = 0, and hence 11 is in A".Since 17 is in X , there exists g in A such tha tf= g + / i and the existence is proved.
72
3 Geometry of Hilbert Space
Suppose now t h a t f = g,+ h , = g2 +h,, where g 1 and g, are in A" while h , and 11, are in A'. Then g2-g1 = h , -h, is in A n A', and hence g2-g1 is orthogonal to itself. Therefore, Ijgz-g,j(2 = (g,-g,,g,-g,) =0 which implies g, = 9,. Finally, h , = h, and the proof is complete.
3.22 Corollary If A is a subspace of the Hilbert space Y?, then Jk" = clos JH. Proof That c l o s d c 4" follows immediately for any subset 4 of 2. Iff is in A"'', then by the theorem f = g + h, where g is in clos A' and /7 is in dl. Sincefis in A"', we have
0 = (f,h) = (g+h,h)
=
(h,h)
=
i/h//'.
Therefore, h = 0 and hence f is in clos A.
If g is a vector in the Hilbert space 8 ,then the complex functional defined = ( J g ) forfin 3' is clearly linear. Moreover, since Iq,(f)/ < ilfll \lg\l for all f in X , it follows that 'p, is bounded and that IIcpgJl < l]g]l. Since 11gIl2= cpg(g) G I/cpgl\ Iigll we have Ilg11 < llcpgII and hence IIcpg/I = Ilgll. The following theorem states that every bounded linear functional on 3% is of this form. cp,(f)
3.23 Theorem (Riesz Representation Theorem) If cp is a bounded linear functional on 2 , then there exists a unique y in ,Hsuch that c p ( f ) = ( L g ) for f in 8. Proof Let L% be the kernel of cp, that is, X = { f8 ~ : q ( f )= 0 ) . Since q is continuous, X is a closed subspace of 2. If X = 2. then q ( f ) = ( f , O ) for f i n X and the theorem is proved. If X # X , then there exists a unit vector h orthogonal to X by the remark following Definition 3.20. Sinceh is not in .X, then cp (l 7 ) # 0. Forfin &'the vectorf-(cp(f)/cp(h))h is in X since q(f-(cp(f)/q(h))h) = 0. Therefore, we have
fo r f i n &', and hence cp(f) = ( L g ) for g = cp(h)h.
The Riesz Representation Theorem
73
If (f,g,)= (f,g2)for f in X , then, in particular, (g,- g 2 , g ,- g 2 ) = 0 and hence g , = g 2 . Therefore, q ( f )= ( Lg) for a unique g in &. Thus we see that the mapping from 2 to 2*defined g -,‘pg is not only norm preserving but onto. Moreover, a straightforward verification shows that this map is conjugate linear, that is, q u l g+ al2 9 2 = ti, qgl+ti, qg2for z, and z2 in C and g, and g, in 2”. Thus for most purposes it is possible to identify &* with X by means of this map. 3.24 I n the theory of complex linear spaces, a linear space is characterized up to an isomorphism by its algebraic dimension. While this is not true for Banach spaces, it is true for Hilbert spaces with an appropriate and different definition of dimension. Before giving this definition we need an extension of the Pythagorean theorem to infinite orthogonal sets. 3.25 Theorem If { . f a ) u E A is an orthogonal subset of the Hilbert space 2 , then converges in 2 if and only if ~ ~ f < a ~00~ and 2 in this case
xuEaf,
IIxdfuIIZ =
xusA
A \lful12.
L
xaeAfa
Proof Let 9 denote the collection of finite subsets of A . I f converges, then by Definition 1.8, the continuity of the norm, and the Pythagorean theorem we have
Therefore, if fa converges, then C z El lA j71i2< 00. Conversely, suppose lls,i12< 00. Given E > 0, there exists F, in 9 such that F 2 F, implies lif,ii2- ZaEfo lifai12< E ’ . Thus, for F , and F, in 9 such that F , , F, 3 F,, we have
xuoA xuEF
1
c s, - c f ,
keFI
UEF,
2
=
c c
01 E F , \F2
llfl!2 + a
ueF, u F 2
c c llfll’ <
E F2 \ F I
l l f l I Z - aeFo
llful!2
E27
where the first equality follows from the Pythagorean theorem. Therefore, the net { ~ a E F s is , }Cauchy, F E S and hence converges by definition.
xueAfu
3.26 Corollary If { e a } a E Ais an orthonormal subset of the Hilbert space
74
3 Geometry of Hilbert Space
X and A is the smallest closed subspace of X containing the set {e, : 2 E A ) , then
P r o o f Let 9 denote the directed set of all finite subsets of A. If is a choice of complex numbers such that < 00, then { A a C u } , E A is an orthogonal set and l]Azeu\12< 03. Thus l U e uconverges to a vector f in H by the theorem and since f = l i m F E 9 ~ u E f l ~ athe e u vector , is seen to lie in
xuEA1Au12 xaeA
xasA
Since A’ contains {e, : 2 E A } , the proof will be complete once we show that N is a closed subspace of H.If { A a } a e A and { p n } u E Asatisfy xaeAIAa12 < co and LA bUl2< 00, then
C
acA
IAU+PUl2
G 2
C 1A,I2
aeA
-I- 2
C IPzl2 <
asA
Hence A’ is a linear subspace of 2 . Now suppose {p}:=, is a Cauchy sequence contained in A’ and that f” = x a e A A r ) e a Then . for each x in A we have
LA
)Yi =llf”-f“ll?
pc)-Ap)l < 1 IAh“)-A;”y
and hence Aa = limn+mA?) exists. Moreover, for F i n 9 we have
xaEA
Hence,f= Aueuis well defined and an element of A’”.Now given E > 0, if we choose N such that n , m 2 N implies Ilf”-f’”l < E , then for F in 9, we have
C lAu-Ap)12
UEF
=
lim
C IA?)-A?)~’
m-m a e F
<
lim supllfm-f”l12 rn- x
< E’.
Therefore, for n > N , we have
and hence N is closed.
3.27 Definition A subset { e Z J u E Aof the Hilbert space i9f is said to be an
orthonormal basis if it is orthonormal and the smallest closed subspace containing it is 2.
The Existence of Orthonornial Bases
75
An orthonormal basis has especially pleasant properties with respect to representing the elements of the space.
3.28 Corollary If (c,},,~ is an orthonormal basis for the Hilbert space 3f and f is a vector in X , then there exist unique Fourier coefficients { A e J Z e A contained in CC such t h a t f = ~ u e A E L u eMoreover, u. Au = (Lee)for r in A . Proof That { A u j a e A exists such that f = I z E A A a e , follows from the preceding corollary and definition. Moreover, if 5 denotes the collection of finite subsets of A and p is in A , then
(The limit is unaffected since the subsets of A containing fl are cofinal in 9.) Therefore the set is unique, where 2, = (f,eu) for 2 in A .
3.29 Theorem Every Hilbert space ( # (0)) possesses an orthonormal basis. Proof Let B be the collection of orthonormal subsets E of X with the partial ordering E , < E, if E , c E , . We want to use Zorn's lemma to assert the existence of a maximal orthonormal subset and then show that it is a basis. To this end let {EA}AE,, be an increasing chain of orthonormal subsets of 2. Then clearly U A E , , E lis an orthonormal subset of 3y" and hence is in 8.Therefore, each chain has a maximal element and hence R itself has a maximal element EM.Let A! be the smallest closed subspace of H containing E M . I f A4 = H ,then EM is an orthonormal basis. If A' # 3'7, then for e a unit vector in A'', the set EM u { e } is an orthonormal subset of 2 greater than -EM.This contradiction shows that dl = H and E M is the desired orthonormal basis.
Although there is nothing unique about an orthonormal basis, that is, there always exist infinitely many if X # (01, the cardinality of an orthonormal basis is well defined.
3.30 Theorem If { e a } a e A and {fa)pes are orthonormal bases for the Hilbert space 2,then card A = card B.
Proof If either of A and B is finite, then the result follows from the theory of linear algebra. Assume, therefore, that card A >, No and card B 2 KO.
76
3 Geometry of Hilbert Space
For s1 in A set B, = {BE B : (e&) # O}. Since e, = & E B ( e m , f f l ) f fby l Corollary 3.28 and 1 = /)e,//'= &EB/(e,,fs)lZ by Theorem 3.25, it follows that card B, < KO. Moreover, since f e = xaEA(e,,fe)e,, it follows that (e&) # 0 for some c( in A . Therefore, B = U a e B, A and hence card B < C a Ecard A B, < K O = card A since card A 2 K O . From symmetry we obtain the reverse inequality and hence card A = card B.
xmEA
3.31 Definition If 2 is a Hilbert space, then the dimension of X , denoted dim X , is the cardinality of any orthonormal basis for 2. The dimension of a Hilbert space is well defined by the previous two theorems. We now show that two Hilbert spaces 2 and X of the same dimension are isomorphic, that is, there exists an isometrical isomorphism from X onto X which preserves the inner product.
3.32 Theorem Two Hilbert spaces are isomorphic if and only if their dimensions are equal. Proof If 2 and X are Hilbert spaces such that d i m 2 = d i m X , then there exist orthonormal bases {ea},eA and { f a } a r A for X and X , respectively. Define the map CD from 2 to .X such that for g in 2, we set CDg = C a e(g,e,)f,. A Since g = (g,e,)e, by Corollary 3.28, it follows from Theorem 3.25 that X a e AI(g,e,)12 = /1g1/'. Therefore, CDg is well defined and
xaaA
ll@gl12=
c
l(s?e,)i2 = ll9ll2.
UEA
That CD is linear is obvious. Hence, Q, is an isometrical isomorphism of X to . X . Thus, CDX is a closed subspace of X which contains { f , : a E A } and by the definition of basis, must therefore be all of X . Lastly, since (g,g) = //g/l2 = /(CDg//'= ((Dg,CDg) for g in X , it follows from the polarization identity that (9, h) = (CDg, ah) for g and h in 2 .
3.33 We conclude this chapter by computing the dimension of Examples 3.13, 3.15, 3.16, and 3.17. For @" it is clear that the n-tuples {(l,O,...,O),(O, 1,...,0), ..., (O,O,..., 1)) form an orthonormal basis, and therefore dim@"= n. Similarly, it is easy to see that the functions in /'(Z+) defined by e,(rn) = 1 if n = rn and 0 otherwise, form an orthonormal subset of /'(Z'). Moreover, sincef in /'(Z') can be writtenf= z : = , f ( n ) e n , it follows that is an orthonormal basis for r2(Zf) and hence that dim[/2(Zf)] = KO.
Notes
77
In the Hilbert space L2([0,l]), the set {e2ninx},,EZ is orthonormal since dx = I if n = 0 and 0 otherwise. Moreover, from the StoneWeierstrass theorem it follows that C([O, 11) is contained in the uniform closure of the subspace spanned by the set {e2ninx : n E Z} and hence C([O, 11) is contained in the smallest closed subspace of L 2 ( [ 0 ,1)) containing them. For f in L2([0,11) it follows from the Lebesgue dominated convergence theorem that limk.+mIlf-fkli2 = 0, where
Since C([O, 13) is dense in L'([O, I]) in the L'-norm, there exists for each k in qk in C([O, I]) such that I q k ( X ) ( < k for .Y in [0, 11 and IIfk-(Pk//1 < l/k2. Hence
6+,a function
< limsup k-a
-
\ kI > "
=
0.
Thus, C([O, 13) is a dense subspace of L 2 ( [ 0 ,11) and hence the smallest closed subspace of L2([0,11) containing the functions {e2ninx: n E h} is L2([0,11). Therefore, {e2ninx},,EZis an orthonormal basis for L2([0,I]). Hence, dim { L 2 ( [ 0 11)) , = K O and therefore despite their apparent difference, 1 2 ( Z ' ) and L2([0,13) are isomorphic Hilbert spaces. Similarly, since a change of variables shows that { x , , } , , ~is~ an orthonormal basis for L2(U), we see that { x , , } , , ~ ~ + is an orthonormal basis for H 2 and hence d i m H z = No also. We indicate in the exercises how to construct an example of a Hilbert space for all dimensions.
Notes The definition of a Hilbert space is due to von Neumann and he along with Hilbert, Riesz, Stone, and others set forth the foundations of the subject. An introduction to the geometry of Hilbert space can be found in many textbooks on functional analysis and, in particular, in Stone [ 1041, Halmos [SS], Riesz and Sz.-Nagy [92], and Akhieser and' Glazman [2].
78
3 Geometry of Hilbert Space
Exercises 3.1
Let A be a nonempty set and let
Show that ] ' ( A ) is a Hilbert space with the pointwise operations and with the inner product (Jg) = ~,,,f(a)g~).Show that dim12(A) = cardA. 3.2 Let 9 be a normed linear space for which the conclusion of the parallelogram law is valid. Show that an inner product can be defined on 9 for which the associated norm is the given norm.
3.3
Show that the completion of an inner product space is a Hilbert space.
3.4 Show that C([O,11) is not a Hilbert space, that is, there is no inner product on C(C0, 11) for which the associated norm is the supremum norm.
3.5 Show that C([O,11) is not homeomorphically isomorphic to a Hilbert space.
*
3.6 Complete the proof began in Section 3.14 that the space 9 defined there is not complete.
3.7 Give an example of a finite dimensional space containing a closed convex set which contains more than one point of smallest norm.*
3.8 Give an example of an infinite dimensional Banach space and a closed convex set having no point of smallest norm.* 3.9 Let cp be a bounded linear functional on the subspace 4 of the Hilbert space X . Show that there exists a unique extension of cp to X havingthe same norm.
3.10 Let X and X be Hilbert spaces and X @ X denote the algebraic direct sum. Show that
((hi, ki), ( h z , k z ) ) = (hi,h2) + (ki,kz) defines an inner product on X @ X ,that X @ X is complete with respect to this inner product, and that the subspaces X 0 (0)and { 0 } 0 X are closed and orthogonal in A@ 8 X . 3.11 Show that each vector of norm one is an extreme point of the unit ball of a Hilbert space.
3.12 Show that the w*-closure of the unit sphere in an infinite-dimensional Hilbert space is the entire unit ball.
Exercises
79
3.13 Show that every orthonormal subset of the Hilbert space &' is contained in an orthonormal basis for &'. 3.14
dim&
Show that if 4 is a closed subspace of the Hilbert space #, then < dim&'.
(Gram-Schmidt) Let {f,}."=l be a subset of the Hilbert space 2 whose closed linear span is A?. Set el =fl/llf l I( and assuming { e k } i = lto have been defined, set
3.15
where en+ is taken to be the zero vector if
c n
fn+l
=
k= 1
(fn+l,ek)ek.
Show that {en},", is an orthonormal basis for 3. 3.16 Show that L2([0,13) has an orthonormal basis {en},"=osuch that en is a polynomial of degree n.
Let Y be a dense linear subspace of the separable Hilbert space &'. Show that Y contains an orthonormal basis for 2. Consider the same question for nonseparable #.*
3.17
Give an example of two closed subspace 4 and Jlr of the Hilbert space &' for which the linear span
3.18
A + N ={f+g:f€M,g€N}
fails to be closed.* (Hint: Take 4 to be the graph of an appropriately chosen bounded linear transformation from X to .f and Jt' to be X@{O},where H =X @ X . ) Show that no Hilbert space has linear dimension KO.(Hint: Use the Baire category theorem.)
3.19
If &' is an infinite-dimensional Hilbert space, then d i m 2 coincides with the smallest cardinal of a dense subset of &'.
3.20
3.21 Let &' and X be Hilbert spaces and let 3 8 X denote the algebraic a
tensor product of &' and X considered as linear spaces over @. Show that
80
3 Geometry of Hilbert Space
defines an inner product on X @ X .Denote the completion of this inner a
product space by S O X XShow . that if { e J a P Aand { f s } s E B are orthonormal bases for X and X , respectively, then { e u @ f D } ( u , B ) E Aisx Ban orthonormal basis for X @ X . 3.22 Let ( X , Y , p ) be a measure space with p finite and cp be a bounded linear functional on L' ( X ) . Show that the restriction of cp to L2( X ) is a bounded linear functional on L 2 ( X ) and hence there exists g in L2(X) such that cp(f) = dp for f in Lz(X). Show further that g is in L"O(X)and hence obtain the characterization of L'(X)* as Lm(X).(Neither the result obtained in Chapter 1 nor the Radon-Nikodym theorem is to be used in this problem.)
lxfa
3.23 (uon Neumann) Let p and v be positive finite measures on (A',Y) such that v is absolutely continuous with respect to p. Show that f+ S,fdp is well defined and a bounded linear functional on L2(p + v). If cp is the function in L2(p+ v) satisfying Jxfcp d ( p v) = J , f d p , then (1 - q)/q is in L' (p) and
+
for E in 9. 3.24 Interpret the results of Exercises 1.30 and 1.31 under the assumption that S is a Hilbert space.
Geometrw of Hilbert Space
3.1 The notion of Banach space abstracts many of the important properties of finite-dimensional linear spaces. The geometry of a Banach space can, however, be quite different from that of Euclidean n-space; for example, the unit ball of a Banach space may have corners, and closed convex sets need not possess a unique vector of smallest norm. The most important geometrical property absent in general Banach spaces is a notion of perpendicularity or orthogonality. In the study of analytic geometry we recall that the orthogonality of two vectors was determined analytically by considering their inner (or dot) product. In this chapter we introduce the abstract notion of an inner product and show how a linear space equipped with an inner product can be made into a normed linear space. If the linear space is complete in the metric defined by this norm, then it is said to be a Hilbert space. This chapter is devoted to studying the elementary geometry of Hilbert spaces and to showing that such spaces possess many of the more pleasant properties of Euclidean nspace. We will show, in fact, that a finite dimensional Hilbert space is isomorphic to Euclidean n-space for some integer n.
3.2 Definition An inner product on a complex linear space Y is a function cp from Y x 9 to d= such that: (1) c p ( ~ ~ f ~ + ~ z f ~ , g ) = ~ ~ r p ( f ~ y g )for + ~ za lcypa (2 finZ ,6 ~ )and fbfZY9
in 9 ;
63
64
3 Geomerry of Hilbert Space
(4) q ( f , f ) >, 0 forfin Y and q ( J f ) = 0 if and only i f f = 0.
A linear space equipped with an inner product is said to be an inner product space. The following lemma contains a useful polarization identity, the importance of which lies in the fact that the value of the inner product p is expressed solely in terms of the values of the associated quadratic form defined by +(f)= q ( f , f ) forfin 9.
+
3.3 Lemma If
Y is an inner product space with the inner product
p, then
for f and g in 9. Proof Compute.
An inner product is usually denoted (,), that is, ( A g ) = cp(f,g) for f and g in 9. 3.4 Definition If Y is an inner product space, then the norm 11 1 on 2’ associated with the inner product is defined by llfll = (f,f)”forfin 9.
The following inequality is basic in the study of inner product spaces. We show that the norm just defined has the required properties of a norm after the proof of this inequality.
3.5 Proposition (Cauchy-Schwarz Inequality) I f f and g are in the inner product space 9, then I(f,g)I
IlPll Ilsll.
Proof For f and g in Y and I in @, we have
14’ 119112+ 2ReCJ(f,g)l + llfll’
= (f+Ig,f+&d
= Ilf+IgJ122 0.
Setting 2 = te“, where t is real and eie is chosen such that e - ” ( J g ) 2 0, we obtain the inequality llgl12t2+ 21(Ag)lt
+ llfll’
2 0.
The Cauchy-Schwarz Inequality
65
Hence the quadratic equation llg112~2 + 2l(f,g)lt
+ llfll’ = 0
in f has at most one real root, and therefore its discriminant must be nonpositive. Substituting we obtain C2I(f,9)1I2- 4 I M 2 Ilfl12 < 0,
from which the desired inequality follows.
3.6 Observe that the property in the preceding proof. 3.7 Proposition If on 9.
(Lf)= 0 implies that f = 0 was not needed
Y is an inner product space, then II.11 defines a norm
Proof We must verify properties (1)-(3) of Definition 1.3. The fact that = 0 if and only i f f = 0 is immediate from (4) (Definition 3.2) and thus (1) holds. Since
/if11
lll.fll = (l..If)” = ( I x ( f , f ) ) ” =
1,Il llfil
for I in C and f i n 9,
we see that (2) holds. Lastly, using the Cauchy-Schwarz inequality, we have llf+gll = Cf+g, f + g ) =
(Lf)+ (h9) + ( s , f >+ tg9 s)
llfil’ + 119Il’ + 2Re(f,g) IlfI12+ 1I91l2+ 21(f,g)l illfll2 + lI91l2 + 2 llfllI1911 G (Ilfll + 11g11)2 for f and g in 9. Thus (3) holds and I[. 11 is a norm. =
3.8 Proposition In an inner product space, the inner product is continuous. Proof Let Y be an inner product space and {fa}ueAand {ga}uoAbe = f a n d limGEAga = g. Then nets in 9 such that limaEAf&
I(f,
9) - ( Agu)l G
and hence limaEA(A,ga) = (L 9).
I(f-fh, g>l+ g-gu>I Ilf-Lll 1 1 ~ 1 1+ IlfaII llg-gull~
3.9 Definition ,In the inner product space Y two Yectors f and g are said to be orthogonal, denoted fL g, if df, g) = 0. A subset 9’of Y is said to be orthogonal if f lg for f and g in Y and orthonormal if, in addition, llfll = I forfin 9’.
66
3 Geometry of Hilbert Space
This notion of orthogonality generalizes the usual one in Euclidean space. It is now possible to extend various theorems from Euclidean geometry to inner product spaces. We give two that will be useful. The first is the familiar Pythagorean theorem, while the second is the result relating the lengths of the sides of a parallelogram to the lengths of the diagonals. 3.10 Proposition (Pythagorean Theorem) If {fl,fi,. ..&} is an orthogonal subset of the inner product space 9, then
Proof Computing, we have
3.11 Proposition (Parallelogram Law) I f f and g are in the inner product space 9, then llf+g112 + Ilf-gll’
= 211fl12 + 211gl12
Proof Expand the left-hand side in terms of inner products. As in the case of normed linear spaces the deepest results are valid only if the space is complete in the metric induced by the norm.
3.12 Definition A Hilbert space is a complex linear space which is complete in the metric induced by the norm. In particular, a Hilbert space is a Banach space. 3.13 Examples We now consider some examples of Hilbert spaces. For n a positive integer let @“ denote the collection of complex ordered n-tuples {x : x = (xI,xz, ...,x,), xi E C}. Then C“ is a complex linear space for the coordinate-wise operations. Define the inner product (,) on @” such that (x, y ) = C;= x i ji.The properties of an inner product are easily verified and the associated norm is the usual Euclidean norm I / x / I 2= (C;=I 1.~~1’)”. To verify completeness suppose { x ’ ’ } ~ = ~is a Cauchy sequence in C”. Then
Examples of Hilbert Spaces: C", 12, L2,and H a 67
since Jxjl'-xim) < I(x~-x'"J~~, it follows that {x:}?=~ is a Cauchy sequence in @ for I < i < n. If we set x = ( x , , x2,.. ., xk), where xi = limk-,a xjl', then x is in @" and limk-,a 2 = x in the norm of 63". Thus @" is a Hilbert space. The space @" is the complex analog of real Euclidean n-space. We show later in this chapter, in a sense to be made precise, that the @"'s are the only finite-dimensional Hilbert spaces. 3.14 We next consider the "union" of the V's. Let 8 be the collection of complex functions on Z+ which take only finitely many nonzero values. With respect to pointwise addition and scalar multiplication, Y is a complex linear space. Moreover, (f,g) = x.,"=of(n)s(n)defines an inner product on 9, where the sum converges, since all but finitely many terms are zero. Is Y a Hilbert space? I t is if 8 is complete with respect to the metric induced by the norm llfl12 = (xF=o lf(n)I2)". Consider the sequence {h}F==l contained in 9, where
One can easily show that {A}?=, is Cauchy but does not converge to an element of 9. We leave this as an exercise for the reader. Thus 8 is not a Hilbert space. The space 8 is not a Hilbert space because it is not large enough. Let us enlarge it to obtain our first example of an infinite-dimensional Hilbert space. (This example should be compared to Example I . 15.) Let 1 2 ( Z ' ) denote the collection of all complex functions cp on Z+ such that Z:,"=o Icp(n)12 < co.Then 12(Z') is a complex linear space, since
3.15
I(f+g)(n)l2 G 2 If(n>I2+ 2 I s(n)12.
-
F o r f a n d g in 12(Z'), define ( J g ) = ~ . , " = o f ( n ) g ( n )Does . this make sense, that is, does the sum converge? For each N in Z+, the n-tuples
FN = (If(O)I, If(l)l,
..., IfCWI)
and
G N = (Is(O)l,Is(l)l, ..., lg(N)l)
lie in CN.Applying the Cauchy-Schwarz inequality, we have
c If(n)s(n>I N
fl=O
= I(FN,GN)I G
IIFNII
IlGNlI
68
3 Geometry of Hilbert Space
-
Thus the series C:=of(n)g(n) converges absolutely. That ( ,) is an inner product follows easily. To establish the completeness of /2(Z') in the metric given by the norm I) ]I2, suppose If'})= is a Cauchy sequence in /'(Z'). Then for each n in Z', we have
If'W -f'(n)I
< Ilfk-f'112
and hence {fk(~)}:=l is a Cauchy sequence in C for each n in H'. Define the function f on E + to be f ( n ) = limk+ .f"(n). Two things must be shown : that f i s in /'(Z') and that limk+,wlif-fk]12 = 0. Since {fk}k",, is a Cauchy sequence, there exists an integer K such that for k 3 K we have l f ' - , f K I( < 1. Thus we obtain
< and hence f is in /2(Z+). Moreover, given E > 0, choose M such that k , j 1, M implies llj'k-f'll < E . Then for k 2 M and any N , we have N
C
n=O
If(n>-fk(n)12
=
<
iim
N
1 Ifj(n)-fk(n)12
j-r n=O
l i m ~ u p I / f j - f ~ ( (<~ '.e2. j- m
Since N is arbitrary, this proves that lif-fkl12 a Hilbert space.
and therefore
/'(Z+)is
3.16 The Space L2 In Section 1.44 we introduced the Banach spaces L' and L" based on a measure space ( X , 9 , p ) . We now consider the corresponding L2 space, which happens to be a Hilbert space. We begin by letting 9'denote the set of all measurable complex functions f on X which satisfy J x dp < co. Since the inequality lf+gl' < 21f12+ 219l2 is valid for arbitrary functionsfand g on A', we see that 2" is a linear space for pointwise addition and scalar multiplication. Let N 2 be the subspace of functions f in Y 2for which jxIfl' dp = 0, and let L2 denote the quotient linear space 2 ' 2 / ~ ~ 2 .
If/'
Examples of Hilberr Spaces: C", 12, L2,and H 2
69
Iff and g are in 9', then the identity
If4 = !?{(If1 + lsl)'
fa
-
Ifl' - 191'1
shows that the function is integrable. If we define cp(J g) = Jx,fijdp for f a n d g in Y 2 ,then cp has all the properties of an inner product except one; namely, cp(f,f) = 0 does not necessarily implyf= 0. By the remark following the proof of the Cauchy-Schwarz inequality, that inequality holds for cp. Thus, i f L f ' , g, and g' are functions in 2" such thatf-f' and g-g' belong to .N2,then
+ Icp(f'99-9')l cp (f-f't f - f ' ) cp (999)
Icp(L9) - cp(SI,g')I G Icp(f-SI99)I
G
+ cp (f', f')cp (9- g'9 9 - 8')= 0. Therefore, cp is a well-defined function on L2. Moreover, if cp([f], [f])= 0, then Jx dp = 0 and hence [f]= [O]. Thus cp is an inner product on L2 and we will denote it from now o n in the usual manner. Furthermore, the associated norm on L2 is defined by l i [ f ] l 1 2 = (Jx l f 1 2 dp)". The only problem remaining before we can conclude that L2 is a Hilbert space is the question of its completeness. This is slightly trickier than in the case of L'. We begin with a general inequality. Takefin P2and define g on X such that g(x) =f(x)/lf(x)I iff(x) # 0 and g ( x ) = 1 otherwise. Then g is measurable, Jg(x)J = I , and = Moreover, applying the Cauchy-Schwarz inequality, we have
If]'
fa If/.
llftll
=
J' If1 4 J' fa& X
=
X
= I(L9)I
G llfll211g112 =
llfll2.
Therefore, llfll G llfllz forfin Y 2 . We now prove that Lz is complete using Corollary 1.10. Let {[fJ}."=l be a sequence in Lz such that C,"= Ii[fn]112 < M < CO. By the precedin,g inequality l [ f. ] l l < M , and hence, by the proof in Section 1.44, there exists f in 2 '' such that C , " = l f , ( x )= f ( x ) for almost all x in X . Moreover
x:,"=I
and since IimN+RI ~ , , " = , f . ( x ) 1=2 If(x)12 for almost all x in X , it follows from Fatou's lemma that is integrable and hencefis in 9''. Moreover,
'/fI
70
3 Geometry of Hilbert Space
since the sequence that k = IirnN+=
{(x.n”= =;}’)l.fI
(xy=If, I)’ I
I
is monotonically increasing, it follows is an integrable function. Therefore,
Jx;=,,,+
by the Lebesgue dominated convergence theorem, since I f,I2 ,< k for all N and lim,,,,, fn(x)12 = 0 for almost all x in X . Thus L2 is a Hilbert space. Lastly, we henceforth adopt the convention stated in Section 1.44 for the elements of L2; namely, we shall treat them as functions.
3.17 The Space H 2 Let T denote the unit circle, p the normalized Lebesgue measure on T, and L2(T) the Hilbert space defined with respect to p . The corresponding Hardy space H 2 is defined as the closed subspace
where
xm is the function x,,(e”) = ein‘. A slight variation of this definition ( ~ E L ’ ( T ) (AX,,) : =0
is
for n = - I , -2, - 3 ,... }.
3.18 Whereas in Chapter 1 after defining a Banach space we proceeded to determine the conjugate space, this is unnecessary for Hilbert spaces since we show in this chapter that the conjugate space of a Hilbert space can be identified with the space itself. This will be the main result of this chapter. We begin by extending a result on the distance to a convex set to subsets of Hilbert spaces. Although most proofs of this result for Euclidean spaces make use of the compactness of closed and bounded subsets, completeness actually suffices.
3.19 Theorem If X is a nonempty, closed, and convex subset of the Hilbert space H , then there exists a unique vector in X of smallest norm. Proof If S = inf{ Ilfll :f E X ) , then there exists a sequence in X such that limn+z 11f.1 = 6. Applying the parallelogram law to the vectorsf,/2 and fm/2, we obtain
Since X is convex, (f,+fm)/2 is in .X and hence li(f,+fm)/2(/22 6’. There-
Examples oJHilberr Spaces: C, 12, L2,and
H 2 71
/I' < 2 ilf,(I'+ 2 ) I f r n I/'- 46', which implies < 26' + 26' - 46' = 0. Iim sup /if,-f,/I'
fore, we have //J, -f ,
n.m-3o
Thus { fn},"=, is a Cauchy sequence in .X and from the completeness of SEP and the fact that X is a closed subset of if we obtain a vector f in .X such that Iirnn+*f, = J Moreover, since the norm is continuous, we have Having proved the existence of a vector in .X of smallest norm we now consider its uniqueness. Suppose f and g are in X with / / f l l = 1/g//= 6. Again using the parallelogram law, we have
since
I\(f + g ) / 2 ( (>, 6. Therefore, f
= g and uniqueness is proved.
If II is a plane and / is a line in three-space perpendicular to n and both IT and I contain the origin, then each vector in the space can be written uniquely as the sum of a vector which lies in n and a vector in the direction of 1. We extend this idea to subspaces of a Hilbert space in the theorem following the definition. 3.20 Definition If -& is a subset of the Hilbert space %',then the orthogonal complement of denoted A'', is the set of vectors in if orthogonal to every vector in A'.
Clearly ~ i % "is a closed subspace of #, possibly consisting ofjust the zero vector. However, if is not the subspace ( 0 ) consisting of the zero vector alone, then A" F X . 3.21 Theorem If dl is a closed subspace of the Hilbert space %' and j' is a vector in #, then there exist unique vectors g in J" and h in ,KL such that
f = g+h. Proof If we set X = i f - k : k E A!}, then .X is a nonempty, closed and convex subset of X . Let /7 be the unique element of .X with smallest norm whose existence is given by the previous theorem. If k is a unit vector in A', then It-(/i,k)k is in X , and hence ))/i)/'
G /lh-(/i,k)kl[2 =
<
/ / / i ) [ '-
G ) ( h , k ) - (/i,k)(h,)+ ( / i , k ) ~ ) I \ k \ l ' .
Therefore, /(h,k)(' 0, which implies (I7,k) = 0, and hence 11 is in A".Since 17 is in X , there exists g in A such tha tf= g + / i and the existence is proved.
72
3 Geometry of Hilbert Space
Suppose now t h a t f = g,+ h , = g2 +h,, where g 1 and g, are in A" while h , and 11, are in A'. Then g2-g1 = h , -h, is in A n A', and hence g2-g1 is orthogonal to itself. Therefore, Ijgz-g,j(2 = (g,-g,,g,-g,) =0 which implies g, = 9,. Finally, h , = h, and the proof is complete.
3.22 Corollary If A is a subspace of the Hilbert space Y?, then Jk" = clos JH. Proof That c l o s d c 4" follows immediately for any subset 4 of 2. Iff is in A"'', then by the theorem f = g + h, where g is in clos A' and /7 is in dl. Sincefis in A"', we have
0 = (f,h) = (g+h,h)
=
(h,h)
=
i/h//'.
Therefore, h = 0 and hence f is in clos A.
If g is a vector in the Hilbert space 8 ,then the complex functional defined = ( J g ) forfin 3' is clearly linear. Moreover, since Iq,(f)/ < ilfll \lg\l for all f in X , it follows that 'p, is bounded and that IIcpgJl < l]g]l. Since 11gIl2= cpg(g) G I/cpgl\ Iigll we have Ilg11 < llcpgII and hence IIcpg/I = Ilgll. The following theorem states that every bounded linear functional on 3% is of this form. cp,(f)
3.23 Theorem (Riesz Representation Theorem) If cp is a bounded linear functional on 2 , then there exists a unique y in ,Hsuch that c p ( f ) = ( L g ) for f in 8. Proof Let L% be the kernel of cp, that is, X = { f8 ~ : q ( f )= 0 ) . Since q is continuous, X is a closed subspace of 2. If X = 2. then q ( f ) = ( f , O ) for f i n X and the theorem is proved. If X # X , then there exists a unit vector h orthogonal to X by the remark following Definition 3.20. Sinceh is not in .X, then cp (l 7 ) # 0. Forfin &'the vectorf-(cp(f)/cp(h))h is in X since q(f-(cp(f)/q(h))h) = 0. Therefore, we have
fo r f i n &', and hence cp(f) = ( L g ) for g = cp(h)h.
The Riesz Representation Theorem
73
If (f,g,)= (f,g2)for f in X , then, in particular, (g,- g 2 , g ,- g 2 ) = 0 and hence g , = g 2 . Therefore, q ( f )= ( Lg) for a unique g in &. Thus we see that the mapping from 2 to 2*defined g -,‘pg is not only norm preserving but onto. Moreover, a straightforward verification shows that this map is conjugate linear, that is, q u l g+ al2 9 2 = ti, qgl+ti, qg2for z, and z2 in C and g, and g, in 2”. Thus for most purposes it is possible to identify &* with X by means of this map. 3.24 I n the theory of complex linear spaces, a linear space is characterized up to an isomorphism by its algebraic dimension. While this is not true for Banach spaces, it is true for Hilbert spaces with an appropriate and different definition of dimension. Before giving this definition we need an extension of the Pythagorean theorem to infinite orthogonal sets. 3.25 Theorem If { . f a ) u E A is an orthogonal subset of the Hilbert space 2 , then converges in 2 if and only if ~ ~ f < a ~00~ and 2 in this case
xuEaf,
IIxdfuIIZ =
xusA
A \lful12.
L
xaeAfa
Proof Let 9 denote the collection of finite subsets of A . I f converges, then by Definition 1.8, the continuity of the norm, and the Pythagorean theorem we have
Therefore, if fa converges, then C z El lA j71i2< 00. Conversely, suppose lls,i12< 00. Given E > 0, there exists F, in 9 such that F 2 F, implies lif,ii2- ZaEfo lifai12< E ’ . Thus, for F , and F, in 9 such that F , , F, 3 F,, we have
xuoA xuEF
1
c s, - c f ,
keFI
UEF,
2
=
c c
01 E F , \F2
llfl!2 + a
ueF, u F 2
c c llfll’ <
E F2 \ F I
l l f l I Z - aeFo
llful!2
E27
where the first equality follows from the Pythagorean theorem. Therefore, the net { ~ a E F s is , }Cauchy, F E S and hence converges by definition.
xueAfu
3.26 Corollary If { e a } a E Ais an orthonormal subset of the Hilbert space
74
3 Geometry of Hilbert Space
X and A is the smallest closed subspace of X containing the set {e, : 2 E A ) , then
P r o o f Let 9 denote the directed set of all finite subsets of A. If is a choice of complex numbers such that < 00, then { A a C u } , E A is an orthogonal set and l]Azeu\12< 03. Thus l U e uconverges to a vector f in H by the theorem and since f = l i m F E 9 ~ u E f l ~ athe e u vector , is seen to lie in
xuEA1Au12 xaeA
xasA
Since A’ contains {e, : 2 E A } , the proof will be complete once we show that N is a closed subspace of H.If { A a } a e A and { p n } u E Asatisfy xaeAIAa12 < co and LA bUl2< 00, then
C
acA
IAU+PUl2
G 2
C 1A,I2
aeA
-I- 2
C IPzl2 <
asA
Hence A’ is a linear subspace of 2 . Now suppose {p}:=, is a Cauchy sequence contained in A’ and that f” = x a e A A r ) e a Then . for each x in A we have
LA
)Yi =llf”-f“ll?
pc)-Ap)l < 1 IAh“)-A;”y
and hence Aa = limn+mA?) exists. Moreover, for F i n 9 we have
xaEA
Hence,f= Aueuis well defined and an element of A’”.Now given E > 0, if we choose N such that n , m 2 N implies Ilf”-f’”l < E , then for F in 9, we have
C lAu-Ap)12
UEF
=
lim
C IA?)-A?)~’
m-m a e F
<
lim supllfm-f”l12 rn- x
< E’.
Therefore, for n > N , we have
and hence N is closed.
3.27 Definition A subset { e Z J u E Aof the Hilbert space i9f is said to be an
orthonormal basis if it is orthonormal and the smallest closed subspace containing it is 2.
The Existence of Orthonornial Bases
75
An orthonormal basis has especially pleasant properties with respect to representing the elements of the space.
3.28 Corollary If (c,},,~ is an orthonormal basis for the Hilbert space 3f and f is a vector in X , then there exist unique Fourier coefficients { A e J Z e A contained in CC such t h a t f = ~ u e A E L u eMoreover, u. Au = (Lee)for r in A . Proof That { A u j a e A exists such that f = I z E A A a e , follows from the preceding corollary and definition. Moreover, if 5 denotes the collection of finite subsets of A and p is in A , then
(The limit is unaffected since the subsets of A containing fl are cofinal in 9.) Therefore the set is unique, where 2, = (f,eu) for 2 in A .
3.29 Theorem Every Hilbert space ( # (0)) possesses an orthonormal basis. Proof Let B be the collection of orthonormal subsets E of X with the partial ordering E , < E, if E , c E , . We want to use Zorn's lemma to assert the existence of a maximal orthonormal subset and then show that it is a basis. To this end let {EA}AE,, be an increasing chain of orthonormal subsets of 2. Then clearly U A E , , E lis an orthonormal subset of 3y" and hence is in 8.Therefore, each chain has a maximal element and hence R itself has a maximal element EM.Let A! be the smallest closed subspace of H containing E M . I f A4 = H ,then EM is an orthonormal basis. If A' # 3'7, then for e a unit vector in A'', the set EM u { e } is an orthonormal subset of 2 greater than -EM.This contradiction shows that dl = H and E M is the desired orthonormal basis.
Although there is nothing unique about an orthonormal basis, that is, there always exist infinitely many if X # (01, the cardinality of an orthonormal basis is well defined.
3.30 Theorem If { e a } a e A and {fa)pes are orthonormal bases for the Hilbert space 2,then card A = card B.
Proof If either of A and B is finite, then the result follows from the theory of linear algebra. Assume, therefore, that card A >, No and card B 2 KO.
76
3 Geometry of Hilbert Space
For s1 in A set B, = {BE B : (e&) # O}. Since e, = & E B ( e m , f f l ) f fby l Corollary 3.28 and 1 = /)e,//'= &EB/(e,,fs)lZ by Theorem 3.25, it follows that card B, < KO. Moreover, since f e = xaEA(e,,fe)e,, it follows that (e&) # 0 for some c( in A . Therefore, B = U a e B, A and hence card B < C a Ecard A B, < K O = card A since card A 2 K O . From symmetry we obtain the reverse inequality and hence card A = card B.
xmEA
3.31 Definition If 2 is a Hilbert space, then the dimension of X , denoted dim X , is the cardinality of any orthonormal basis for 2. The dimension of a Hilbert space is well defined by the previous two theorems. We now show that two Hilbert spaces 2 and X of the same dimension are isomorphic, that is, there exists an isometrical isomorphism from X onto X which preserves the inner product.
3.32 Theorem Two Hilbert spaces are isomorphic if and only if their dimensions are equal. Proof If 2 and X are Hilbert spaces such that d i m 2 = d i m X , then there exist orthonormal bases {ea},eA and { f a } a r A for X and X , respectively. Define the map CD from 2 to .X such that for g in 2, we set CDg = C a e(g,e,)f,. A Since g = (g,e,)e, by Corollary 3.28, it follows from Theorem 3.25 that X a e AI(g,e,)12 = /1g1/'. Therefore, CDg is well defined and
xaaA
ll@gl12=
c
l(s?e,)i2 = ll9ll2.
UEA
That CD is linear is obvious. Hence, Q, is an isometrical isomorphism of X to . X . Thus, CDX is a closed subspace of X which contains { f , : a E A } and by the definition of basis, must therefore be all of X . Lastly, since (g,g) = //g/l2 = /(CDg//'= ((Dg,CDg) for g in X , it follows from the polarization identity that (9, h) = (CDg, ah) for g and h in 2 .
3.33 We conclude this chapter by computing the dimension of Examples 3.13, 3.15, 3.16, and 3.17. For @" it is clear that the n-tuples {(l,O,...,O),(O, 1,...,0), ..., (O,O,..., 1)) form an orthonormal basis, and therefore dim@"= n. Similarly, it is easy to see that the functions in /'(Z+) defined by e,(rn) = 1 if n = rn and 0 otherwise, form an orthonormal subset of /'(Z'). Moreover, sincef in /'(Z') can be writtenf= z : = , f ( n ) e n , it follows that is an orthonormal basis for r2(Zf) and hence that dim[/2(Zf)] = KO.
Notes
77
In the Hilbert space L2([0,l]), the set {e2ninx},,EZ is orthonormal since dx = I if n = 0 and 0 otherwise. Moreover, from the StoneWeierstrass theorem it follows that C([O, 11) is contained in the uniform closure of the subspace spanned by the set {e2ninx : n E Z} and hence C([O, 11) is contained in the smallest closed subspace of L 2 ( [ 0 ,1)) containing them. For f in L2([0,11) it follows from the Lebesgue dominated convergence theorem that limk.+mIlf-fkli2 = 0, where
Since C([O, 13) is dense in L'([O, I]) in the L'-norm, there exists for each k in qk in C([O, I]) such that I q k ( X ) ( < k for .Y in [0, 11 and IIfk-(Pk//1 < l/k2. Hence
6+,a function
< limsup k-a
-
\ kI > "
=
0.
Thus, C([O, 13) is a dense subspace of L 2 ( [ 0 ,11) and hence the smallest closed subspace of L2([0,11) containing the functions {e2ninx: n E h} is L2([0,11). Therefore, {e2ninx},,EZis an orthonormal basis for L2([0,I]). Hence, dim { L 2 ( [ 0 11)) , = K O and therefore despite their apparent difference, 1 2 ( Z ' ) and L2([0,13) are isomorphic Hilbert spaces. Similarly, since a change of variables shows that { x , , } , , ~is~ an orthonormal basis for L2(U), we see that { x , , } , , ~ ~ + is an orthonormal basis for H 2 and hence d i m H z = No also. We indicate in the exercises how to construct an example of a Hilbert space for all dimensions.
Notes The definition of a Hilbert space is due to von Neumann and he along with Hilbert, Riesz, Stone, and others set forth the foundations of the subject. An introduction to the geometry of Hilbert space can be found in many textbooks on functional analysis and, in particular, in Stone [ 1041, Halmos [SS], Riesz and Sz.-Nagy [92], and Akhieser and' Glazman [2].
78
3 Geometry of Hilbert Space
Exercises 3.1
Let A be a nonempty set and let
Show that ] ' ( A ) is a Hilbert space with the pointwise operations and with the inner product (Jg) = ~,,,f(a)g~).Show that dim12(A) = cardA. 3.2 Let 9 be a normed linear space for which the conclusion of the parallelogram law is valid. Show that an inner product can be defined on 9 for which the associated norm is the given norm.
3.3
Show that the completion of an inner product space is a Hilbert space.
3.4 Show that C([O,11) is not a Hilbert space, that is, there is no inner product on C(C0, 11) for which the associated norm is the supremum norm.
3.5 Show that C([O,11) is not homeomorphically isomorphic to a Hilbert space.
*
3.6 Complete the proof began in Section 3.14 that the space 9 defined there is not complete.
3.7 Give an example of a finite dimensional space containing a closed convex set which contains more than one point of smallest norm.*
3.8 Give an example of an infinite dimensional Banach space and a closed convex set having no point of smallest norm.* 3.9 Let cp be a bounded linear functional on the subspace 4 of the Hilbert space X . Show that there exists a unique extension of cp to X havingthe same norm.
3.10 Let X and X be Hilbert spaces and X @ X denote the algebraic direct sum. Show that
((hi, ki), ( h z , k z ) ) = (hi,h2) + (ki,kz) defines an inner product on X @ X ,that X @ X is complete with respect to this inner product, and that the subspaces X 0 (0)and { 0 } 0 X are closed and orthogonal in A@ 8 X . 3.11 Show that each vector of norm one is an extreme point of the unit ball of a Hilbert space.
3.12 Show that the w*-closure of the unit sphere in an infinite-dimensional Hilbert space is the entire unit ball.
Exercises
79
3.13 Show that every orthonormal subset of the Hilbert space &' is contained in an orthonormal basis for &'. 3.14
dim&
Show that if 4 is a closed subspace of the Hilbert space #, then < dim&'.
(Gram-Schmidt) Let {f,}."=l be a subset of the Hilbert space 2 whose closed linear span is A?. Set el =fl/llf l I( and assuming { e k } i = lto have been defined, set
3.15
where en+ is taken to be the zero vector if
c n
fn+l
=
k= 1
(fn+l,ek)ek.
Show that {en},", is an orthonormal basis for 3. 3.16 Show that L2([0,13) has an orthonormal basis {en},"=osuch that en is a polynomial of degree n.
Let Y be a dense linear subspace of the separable Hilbert space &'. Show that Y contains an orthonormal basis for 2. Consider the same question for nonseparable #.*
3.17
Give an example of two closed subspace 4 and Jlr of the Hilbert space &' for which the linear span
3.18
A + N ={f+g:f€M,g€N}
fails to be closed.* (Hint: Take 4 to be the graph of an appropriately chosen bounded linear transformation from X to .f and Jt' to be X@{O},where H =X @ X . ) Show that no Hilbert space has linear dimension KO.(Hint: Use the Baire category theorem.)
3.19
If &' is an infinite-dimensional Hilbert space, then d i m 2 coincides with the smallest cardinal of a dense subset of &'.
3.20
3.21 Let &' and X be Hilbert spaces and let 3 8 X denote the algebraic a
tensor product of &' and X considered as linear spaces over @. Show that
80
3 Geometry of Hilbert Space
defines an inner product on X @ X .Denote the completion of this inner a
product space by S O X XShow . that if { e J a P Aand { f s } s E B are orthonormal bases for X and X , respectively, then { e u @ f D } ( u , B ) E Aisx Ban orthonormal basis for X @ X . 3.22 Let ( X , Y , p ) be a measure space with p finite and cp be a bounded linear functional on L' ( X ) . Show that the restriction of cp to L2( X ) is a bounded linear functional on L 2 ( X ) and hence there exists g in L2(X) such that cp(f) = dp for f in Lz(X). Show further that g is in L"O(X)and hence obtain the characterization of L'(X)* as Lm(X).(Neither the result obtained in Chapter 1 nor the Radon-Nikodym theorem is to be used in this problem.)
lxfa
3.23 (uon Neumann) Let p and v be positive finite measures on (A',Y) such that v is absolutely continuous with respect to p. Show that f+ S,fdp is well defined and a bounded linear functional on L2(p + v). If cp is the function in L2(p+ v) satisfying Jxfcp d ( p v) = J , f d p , then (1 - q)/q is in L' (p) and
+
for E in 9. 3.24 Interpret the results of Exercises 1.30 and 1.31 under the assumption that S is a Hilbert space.