- Email: [email protected]
CHAPTER 3. Inner Product Spaces and Some of their Properties
CHAPTER 3
Inner Product Spaces and Some of their Properties 3.1 INNER PRODUCT In applied mathematics, we are used to considering quantities in R3 like 1x1 .lyl cos 0 where x, y are vectors and 0 is the angle between x and y. A generalization of this concept is the inner product. The inner product of two elements x, y of a real or complex linear space Y is denoted (x, y) and has the followingproperties : (i) (x, x) > 0 whenever x # 0 (ii) (X,Y> = (Y,) (iii) (ax, y) = a(x, y), a a scalar (iv) (x + y, z> = (x, z> + ( Y , z). Any space on which an inner product has been defined is called an inner product space. The inner product can immediately be used to define a norm by setting llyll = (y, Y ) ” ~ .The axioms for a norm can be seen to appear in the definition of the inner product except for the triangle inequality which is not difficult to prove.
3.2 ORTHOGONALITY two elements x , y ~Y are defined to be In an inner product space, orthogonal, (denoted x I y) if (x, y) = 0.Let Z be a subset of Y and y be an element of Y then y is said to be orthogonal to Z (denoted y I Z)if (y, z ) = 0, vz E z.
Let { y i } be a set of elements in an inner product space Y satisfying ( y , y j ) =0, i # j , i.e. {y,} is a set of mutually orthogonal elements. Define ei = yi/llyi)I,then each ei has unit norm and the set { e i }is called an orthonormal set. 26
3.2. ORTHOGONALITY
27
Recall that Y has finite dimension if a subset E of Y exists, called a basis for Y, such that every element of Y can be written uniquely as a finite linear combination of elements of E. Inner product spaces will often have infinite dimension and the Schauder basis is a suitable extension of the (purely algebraic) concept of basis defined for finite dimensional spaces. Let Y be a linear space and let E be a sequence of elements { e i }of elements of Y; E is called a Schauder basis for Y if every element y E Y can be uniquely represented by a sum y = Ciaiei,uiE K . Finally suppose that a sequence { e,} is a Schauder basis for Y while also {e,} is an orthonormal set, then { e , } is called an orthonormal basis for Z
3.3 HILBERT SPACE Let X be a linear space on which an inner product has been defined, with a norm derived from the inner product and which is complete in this norm. Then X is called a Hilbert space. More briefly X is a Banach space with an inner product from which its norm has been derived. A general comment which should not be interpreted too literally is that geometric intuition usually leads to correct conclusions in Hilbert space (although it usually fails in the more general setting of a Banach space). This makes generalization of familiar finite dimensional theorems easy to visualize, with the projection theorem which follows later being a good example.
Examples of Hilbert spaces (1) R" is a Hilbert space with inner product n
1 XiYi
(X,Y> =
i= 1
(2) lz is a Hilbert space with inner product
0,Y >
c m
=
XiYi
i= 1
s,
(3) L2 is a Hilbert space with inner product (x, Y > =
X(t)Y(t)dt
where the functions x,y are defined and integrable on a domain I.
3. INNER
28
PRODUCT SPACES AND SOME OF THEIR PROPERTIES
3.4 THE PARALLELOGRAM LAW Let X be a Hilbert space and x, y be two elements of X. Then
+ Y(I2 = (x + Y,X + Y> = (x, x> + ( Y , x> + (x, Y> + (Y, Y> = lIxIIZ ++ (x, Y> + IJY1I2 =llx1I2 + 2B((X,Y)) + llY1I2. Ilx
Similarly
Ilx-YlIZ= l l x 1 1 2 - ~ ( ( x , Y > ) =IlYIIZ so that IIX
where x gram.
+ Y1I2 + Ilx - YllZ = 211x112 + 211Y112
+ y is the long diagonal and x - y the short diagonal of a parallelo-
3.5 THEOREMS Theorem 3.1. (Existence of a unique minimizing element.) Let X be a Hilbert space and M be a closed subspace of X . Let x E X , x 4 M then there exists an element y E M and a real number d satisjying
IIx
- yll =
inf I(x - zll
=
d
zsM
Proof: Let {zi}be a sequence in M such that
Ilx - 'ill d Let z,,,, zn be any two elements of the sequence (zi} then by the parallelogram law +
II(x--n)+(x-z,)112+
$zn
II(X--n)-(x--m)112=211x-zn1/2+2Jlx-ZmIlZ
+ zm)E M since M is a subspace.
29
3.5. THEOREMS Thus,
- $zn - ZJIJ 2 d
JJX
and equation (3.5.1) can be rewritten 4d2
+ ) ( z m-
2)lx - zn112+ 21)x - zmI(2
ZJ<
+
zl(zm 1 - zn(I2<$1) x - zn(I2 + ( ) x- zrn1l2- d2
Let m, n -,00 then the right-hand side goes to zero. Hence {zi> is a Cauchy sequence in a closed subspace of a complete space, and hence lim { z i } = y E M 1-03
(Since the limit of the sequence must be unique.)
Theorem 3.2. (Unique decomposition property or projection theorem.) Let M be a closed subspace in a Hilbert space X , then every x E X can be uniquely decomposed as x = v + w, v E M , w E M I . Proof. If x E M then trivially u = x, w = 0. Thus, assume x q! M. Choose y E M such that
IIx
- yll
=
inf J ( x- z(I
=d
Z€M
(see previous theorem). Now let v = y and w = x - y . Let z be any non zero element in M. Let a be any scalar then d 2 < / ( ~ + a z l ( ~~l =~ ( ~ ~ + 2 a < w , z ) + a ~ 1 / z ( 1 ~
Now put ct = - (w,Z > / ) ( Z ( ) then ~ (w,z ) ? < 0. Thus w is orthogonal to z and we have proved that w E M I while since v = y , v E M . To show uniqueness assume that x =v
+ w = V’ + w‘;
v,v’EM,
w,w’EM‘-
Then v - v’
=
w’ - w
= q (say)
The element q = o - v’, hence q is in M (subspace property). Also q = w’- w,
30
3. INNER PRODUCT SPACES AND SOME OF THEIR PROPERTIES
hence q is in M I . Only q = 0 can satisfy these conditions, hence q = 0 and uniqueness is proved.
Corollary 3.2.1. There exists an element M’ # 0 in X which is orthogonal to M . Proof: Let x E X , x 4 M then from Theorem 3.2. we can write x = v + w, v E M, w E MI. Comment Referring back to Theorem 3.1, it is clear that ( x - y ) E M I while y E M . Hence ( x - y ) is orthogonal to y. This can be considered a corollary to Theorem 3.1.
Theorem 3.3. (Riesz representation theorem). Let X be a Hilbert space and take any bounded linear functional f : X -,K. Then there exists some element g E X such that f ( x ) = (x, g ) (i.e. the functional can be represented by an inner product). Further I( f 11 = (Ig(1.( I t is clear that ifg isjxed in the inner product ( x , g ) then the inner product is a linearfunctional X --* K . The theorem asserts the converse, that everyfunctional on X can be represented by an inner product.)
-
Proof: Iff maps every element of X into zero then we take g = 0. In the other cases g must be nonzero. Let N , be the null space off, i.e.
N , = (.If(.)
= 0} then
xE N,
i.e.
O N , Now N , is a subspace of X (check this) and N , is closed since, let {xi}-, x be a sequence in N then
,
f (x) = f (x) - f ( x i ) since f (xi)= 0 = f ( x - Xi)
Thus
If ( x ) ( = If ( x - xi)(< kllx - xiII
for some k E K
The right-hand side goes to zero as i -+ CQ, hencef (x) = 0, x E N , and N , is closed. Now let z be any element in X satisfying z 4 N, then by Theorem 3.2 we can write
z=u+~,
UEN,,
gENi
31
3.5. THEOREMS
Thus, there does exist a g satisfying g INf as required. g is unique since let g' satisfy (x,
s> = 0,g'>
then (x, g - g') = 0, Vx,
i.e. g
= gf
Finally, by definition
llfll = SUP If(x)l/llxII xex
x+o
Put x = y, then lf(x)l/llx )I takes on its supremum and becomes l ( y ~ Y ) l / l l Y l l = Ilyll
Thus
IIflI
=
SUP lf(x)l/llxIl = llyll xex
Let E be a Hilbert space with an orthonormal basis {ai}. Let T : E + E be a bounded linear operator. Then T can be represented by a matrix A with coefficients aij = ( a , T a ). Clearly, if the space E is dnite dimensional, the study of operators can be entirely in terms of matrices. In the general case the operator T can be considered to be represented by a sequenceof Fourier coefficients. Unbounded operators are encountered in the study of distributed parameter systems. However, if the operator T is unbounded, then there is no result equivalent to the above. If the operator is unbounded but closed, useful results can still be found. Such results are given and used in the chapter on distributed parameter systems.
3.6. EXERCISES (1)Show that (x, y ) in R" satisfies the given axioms for an inner product and show also that (x, y ) l l 2 satisfies the given axioms for a norm. (2) Let X be a Hilbert space and let x, y E X. Prove the Schwarz inequality cx, Y )
Hint: (x - ay, x
- ay)
II x I1 II Y li
3 0 for any a E K : put a = (x, y)/(y, y).
32
3. INNW PRODUCT SPACES AND SOME OF THEIR PROPERTIES
(3)Let X be a Hilbert space and let x, y E X such that (x, y) = 0. Prove that Ilx + y1I2 =
(known as the Pythagorean law).
1 1 ~ 1 1 2 + IIYlIZ
(4) Show that the norm in Hilbert space satisfies the parallelogram law.
(5) Let E be the set of functions in X = LZ[a,b] defined by
If
E = {f
continuous on [a,b ] }
Is E closed in x? (6) Let X be an infinite dimensional vector space. Let {e,}, i = 1,. . ., be an infinite set of linearly independent vectors in X. Does {ei} necessarily form a basis for x?
(7) Show that the function sin(nx) is orthogonal to each of the functions cos(nx), sin(mx), cos(mx), m # n, on the interval. (-a,a) (m, n being integers). Go on to show that the set of functions
{+
cos(nx) -sin (nx) , .Ja l l is an orthonormal set on [ - a,a].
n = 1 , . . .,
(8) Let X be a Hilbert space and IetfG X, let {ei} be an orthonormal set in X. Define i=l
Define Jn
=
(If-
gn1I2 =
<.Lf>- 2 < . gn> ~ +
show that J is minimized when
ai = ( 5 ei),
i = 1,. . ., n
The ai are the Fourier coefficients off:
Inner Product Spaces and Some of their Properties 3.1 INNER PRODUCT In applied mathematics, we are used to considering quantities in R3 like 1x1 .lyl cos 0 where x, y are vectors and 0 is the angle between x and y. A generalization of this concept is the inner product. The inner product of two elements x, y of a real or complex linear space Y is denoted (x, y) and has the followingproperties : (i) (x, x) > 0 whenever x # 0 (ii) (X,Y> = (Y,) (iii) (ax, y) = a(x, y), a a scalar (iv) (x + y, z> = (x, z> + ( Y , z). Any space on which an inner product has been defined is called an inner product space. The inner product can immediately be used to define a norm by setting llyll = (y, Y ) ” ~ .The axioms for a norm can be seen to appear in the definition of the inner product except for the triangle inequality which is not difficult to prove.
3.2 ORTHOGONALITY two elements x , y ~Y are defined to be In an inner product space, orthogonal, (denoted x I y) if (x, y) = 0.Let Z be a subset of Y and y be an element of Y then y is said to be orthogonal to Z (denoted y I Z)if (y, z ) = 0, vz E z.
Let { y i } be a set of elements in an inner product space Y satisfying ( y , y j ) =0, i # j , i.e. {y,} is a set of mutually orthogonal elements. Define ei = yi/llyi)I,then each ei has unit norm and the set { e i }is called an orthonormal set. 26
3.2. ORTHOGONALITY
27
Recall that Y has finite dimension if a subset E of Y exists, called a basis for Y, such that every element of Y can be written uniquely as a finite linear combination of elements of E. Inner product spaces will often have infinite dimension and the Schauder basis is a suitable extension of the (purely algebraic) concept of basis defined for finite dimensional spaces. Let Y be a linear space and let E be a sequence of elements { e i }of elements of Y; E is called a Schauder basis for Y if every element y E Y can be uniquely represented by a sum y = Ciaiei,uiE K . Finally suppose that a sequence { e,} is a Schauder basis for Y while also {e,} is an orthonormal set, then { e , } is called an orthonormal basis for Z
3.3 HILBERT SPACE Let X be a linear space on which an inner product has been defined, with a norm derived from the inner product and which is complete in this norm. Then X is called a Hilbert space. More briefly X is a Banach space with an inner product from which its norm has been derived. A general comment which should not be interpreted too literally is that geometric intuition usually leads to correct conclusions in Hilbert space (although it usually fails in the more general setting of a Banach space). This makes generalization of familiar finite dimensional theorems easy to visualize, with the projection theorem which follows later being a good example.
Examples of Hilbert spaces (1) R" is a Hilbert space with inner product n
1 XiYi
(X,Y> =
i= 1
(2) lz is a Hilbert space with inner product
0,Y >
c m
=
XiYi
i= 1
s,
(3) L2 is a Hilbert space with inner product (x, Y > =
X(t)Y(t)dt
where the functions x,y are defined and integrable on a domain I.
3. INNER
28
PRODUCT SPACES AND SOME OF THEIR PROPERTIES
3.4 THE PARALLELOGRAM LAW Let X be a Hilbert space and x, y be two elements of X. Then
+ Y(I2 = (x + Y,X + Y> = (x, x> + ( Y , x> + (x, Y> + (Y, Y> = lIxIIZ +
Similarly
Ilx-YlIZ= l l x 1 1 2 - ~ ( ( x , Y > ) =IlYIIZ so that IIX
where x gram.
+ Y1I2 + Ilx - YllZ = 211x112 + 211Y112
+ y is the long diagonal and x - y the short diagonal of a parallelo-
3.5 THEOREMS Theorem 3.1. (Existence of a unique minimizing element.) Let X be a Hilbert space and M be a closed subspace of X . Let x E X , x 4 M then there exists an element y E M and a real number d satisjying
IIx
- yll =
inf I(x - zll
=
d
zsM
Proof: Let {zi}be a sequence in M such that
Ilx - 'ill d Let z,,,, zn be any two elements of the sequence (zi} then by the parallelogram law +
II(x--n)+(x-z,)112+
$zn
II(X--n)-(x--m)112=211x-zn1/2+2Jlx-ZmIlZ
+ zm)E M since M is a subspace.
29
3.5. THEOREMS Thus,
- $zn - ZJIJ 2 d
JJX
and equation (3.5.1) can be rewritten 4d2
+ ) ( z m-
2)lx - zn112+ 21)x - zmI(2
ZJ<
+
zl(zm 1 - zn(I2<$1) x - zn(I2 + ( ) x- zrn1l2- d2
Let m, n -,00 then the right-hand side goes to zero. Hence {zi> is a Cauchy sequence in a closed subspace of a complete space, and hence lim { z i } = y E M 1-03
(Since the limit of the sequence must be unique.)
Theorem 3.2. (Unique decomposition property or projection theorem.) Let M be a closed subspace in a Hilbert space X , then every x E X can be uniquely decomposed as x = v + w, v E M , w E M I . Proof. If x E M then trivially u = x, w = 0. Thus, assume x q! M. Choose y E M such that
IIx
- yll
=
inf J ( x- z(I
=d
Z€M
(see previous theorem). Now let v = y and w = x - y . Let z be any non zero element in M. Let a be any scalar then d 2 < / ( ~ + a z l ( ~~l =~ ( ~ ~ + 2 a < w , z ) + a ~ 1 / z ( 1 ~
Now put ct = - (w,Z > / ) ( Z ( ) then ~ (w,z ) ? < 0. Thus w is orthogonal to z and we have proved that w E M I while since v = y , v E M . To show uniqueness assume that x =v
+ w = V’ + w‘;
v,v’EM,
w,w’EM‘-
Then v - v’
=
w’ - w
= q (say)
The element q = o - v’, hence q is in M (subspace property). Also q = w’- w,
30
3. INNER PRODUCT SPACES AND SOME OF THEIR PROPERTIES
hence q is in M I . Only q = 0 can satisfy these conditions, hence q = 0 and uniqueness is proved.
Corollary 3.2.1. There exists an element M’ # 0 in X which is orthogonal to M . Proof: Let x E X , x 4 M then from Theorem 3.2. we can write x = v + w, v E M, w E MI. Comment Referring back to Theorem 3.1, it is clear that ( x - y ) E M I while y E M . Hence ( x - y ) is orthogonal to y. This can be considered a corollary to Theorem 3.1.
Theorem 3.3. (Riesz representation theorem). Let X be a Hilbert space and take any bounded linear functional f : X -,K. Then there exists some element g E X such that f ( x ) = (x, g ) (i.e. the functional can be represented by an inner product). Further I( f 11 = (Ig(1.( I t is clear that ifg isjxed in the inner product ( x , g ) then the inner product is a linearfunctional X --* K . The theorem asserts the converse, that everyfunctional on X can be represented by an inner product.)
-
Proof: Iff maps every element of X into zero then we take g = 0. In the other cases g must be nonzero. Let N , be the null space off, i.e.
N , = (.If(.)
= 0} then
xE N,
i.e.
O N , Now N , is a subspace of X (check this) and N , is closed since, let {xi}-, x be a sequence in N then
,
f (x) = f (x) - f ( x i ) since f (xi)= 0 = f ( x - Xi)
Thus
If ( x ) ( = If ( x - xi)(< kllx - xiII
for some k E K
The right-hand side goes to zero as i -+ CQ, hencef (x) = 0, x E N , and N , is closed. Now let z be any element in X satisfying z 4 N, then by Theorem 3.2 we can write
z=u+~,
UEN,,
gENi
31
3.5. THEOREMS
Thus, there does exist a g satisfying g INf as required. g is unique since let g' satisfy (x,
s> = 0,g'>
then (x, g - g') = 0, Vx,
i.e. g
= gf
Finally, by definition
llfll = SUP If(x)l/llxII xex
x+o
Put x = y, then lf(x)l/llx )I takes on its supremum and becomes l ( y ~ Y ) l / l l Y l l = Ilyll
Thus
IIflI
=
SUP lf(x)l/llxIl = llyll xex
Let E be a Hilbert space with an orthonormal basis {ai}. Let T : E + E be a bounded linear operator. Then T can be represented by a matrix A with coefficients aij = ( a , T a ). Clearly, if the space E is dnite dimensional, the study of operators can be entirely in terms of matrices. In the general case the operator T can be considered to be represented by a sequenceof Fourier coefficients. Unbounded operators are encountered in the study of distributed parameter systems. However, if the operator T is unbounded, then there is no result equivalent to the above. If the operator is unbounded but closed, useful results can still be found. Such results are given and used in the chapter on distributed parameter systems.
3.6. EXERCISES (1)Show that (x, y ) in R" satisfies the given axioms for an inner product and show also that (x, y ) l l 2 satisfies the given axioms for a norm. (2) Let X be a Hilbert space and let x, y E X. Prove the Schwarz inequality cx, Y )
Hint: (x - ay, x
- ay)
II x I1 II Y li
3 0 for any a E K : put a = (x, y)/(y, y).
32
3. INNW PRODUCT SPACES AND SOME OF THEIR PROPERTIES
(3)Let X be a Hilbert space and let x, y E X such that (x, y) = 0. Prove that Ilx + y1I2 =
(known as the Pythagorean law).
1 1 ~ 1 1 2 + IIYlIZ
(4) Show that the norm in Hilbert space satisfies the parallelogram law.
(5) Let E be the set of functions in X = LZ[a,b] defined by
If
E = {f
continuous on [a,b ] }
Is E closed in x? (6) Let X be an infinite dimensional vector space. Let {e,}, i = 1,. . ., be an infinite set of linearly independent vectors in X. Does {ei} necessarily form a basis for x?
(7) Show that the function sin(nx) is orthogonal to each of the functions cos(nx), sin(mx), cos(mx), m # n, on the interval. (-a,a) (m, n being integers). Go on to show that the set of functions
{+
cos(nx) -sin (nx) , .Ja l l is an orthonormal set on [ - a,a].
n = 1 , . . .,
(8) Let X be a Hilbert space and IetfG X, let {ei} be an orthonormal set in X. Define i=l
Define Jn
=
(If-
gn1I2 =
<.Lf>- 2 < . gn> ~ +
show that J is minimized when
ai = ( 5 ei),
i = 1,. . ., n
The ai are the Fourier coefficients off: