23.1 Basic Theory
As explained earlier, a normed linear space is a vector space X with a norm. This is a map
) which satisfies the following axioms.
≥ 0 and equals 0 if and only if x = 0
- For α a scalar and x a vector in X, =
Then, as discussed earlier, this is a metric space if d
. The field of scalars will be either
, usually ℂ
. Then as before, there is a definition of an inner product space. If
meaning that Cauchy sequences converge, then this is called a
Banach space. A whole lot can be said about
Banach spaces but not in this book. Here, a specialization will be considered which ties in well with the
earlier material on inner product spaces.
Definition 23.1.1 Let X be a vector space. An inner product is a mapping from X × X to
ℂ if X is complex and from X × X to ℝ if X is real, denoted by (x,y) which satisfies the
For a,b ∈ ℂ and x,y,z ∈ X,
Note that 23.2 and 23.3 imply (x,ay + bz) = a(x,y) + b(x,z). Such a vector space is called an inner product
The Cauchy Schwarz inequality is fundamental for the study of inner product spaces. The proof is
identical to that given earlier in Theorem 11.4.4 on Page 730.
Theorem 23.1.2 (Cauchy Schwarz) In any inner product space
Also, as earlier, the norm given by the inner product, really is a norm.
Proposition 23.1.3 For an inner product space,
1∕2 does specify a norm.
The following lemma is called the parallelogram identity. It was also discussed earlier.
Lemma 23.1.4 In an inner product space,
The proof, a straightforward application of the inner product axioms, is left to the reader.
Lemma 23.1.5 For x ∈ H, an inner product space,
Proof: By the Cauchy Schwarz inequality, if x≠0,
It is obvious that 23.4 holds in the case that x = 0.
Definition 23.1.6 A Hilbert space is an inner product space which is complete. Thus a Hilbert
space is a Banach space in which the norm comes from an inner product as described above. Often
people use the symbol
to denote the norm rather than
In Hilbert space, one can define a projection map onto closed convex nonempty sets.
Definition 23.1.7 A set, K, is convex if whenever λ ∈ [0,1] and x,y ∈ K,λx + (1 − λ)y ∈ K.
This was done in the problems beginning with Problem 8 on Page 777.
Theorem 23.1.8 Let K be a closed convex nonempty subset of a Hilbert space, H, and let x ∈ H.
Then there exists a unique point Px ∈ K such that ||Px − x||≤||y − x|| for all y ∈ K.
Corollary 23.1.9 Let K be a closed, convex, nonempty subset of a Hilbert space, H, and let x ∈ H. Then
for z ∈ K,z = Px if and only if
for all y ∈ K.
Here we present an easier version which will be sufficient for what is needed. First is a simple lemma
which is interesting for its own sake.
Lemma 23.1.10 Suppose Re
for all y ∈ M a subspace of H, an inner product space.
Proof: Consider the following:
and so Im
= 0 as well as
it follows that
Theorem 23.1.11 Let H be a Hilbert space and let M be a closed subspace. Then if w ∈ H, there exists a
unique Pw ∈ H such that
for all y ∈ M. Thus Pw is the point of M closest to w and it is unique. Then z ∈ M equals Pw if and only
Also the map w → Pw is a linear map which satisfies
Proof: Let λ ≡ inf
be a minimizing sequence. Say
Then, by the parallelogram identity,
and clearly the right side converges to 0 if m,n →∞
. Thus this is a Cauchy sequence and so it converges to
some z ∈ M
is closed. Then
and so Pw ≡ z is a closest point. There can be only one closest point because if zi works, then by the
parallelogram identity again,
and so if these are different, then
is closer to
contradicting the choice of zi
Now for the characterization: For z ∈ M, letting y ∈ M consider z + t
t ∈ ℝ
Then for z to be the closest point to w, one needs the above to be minimized when t = 0. Taking a
derivative, this requires that
for any y ∈ M. But this is the same as saying that Re
= 0 for all
y ∈ M
. By Lemma 23.1.10
= 0 for all
y ∈ M
= 0 if
= 0 for all
y ∈ M,
minimum when t
= 0 for any y.
But a generic point of M
is of the form z
As to P being linear, for y ∈ M arbitrary,
By uniqueness, P
which yields the desired estimate. ■
Note that the operator norm of P equals 1.
Now pick w ∈ M with
= 1. Then
Definition 23.1.12 If A : X → Y is linear where X,Y are two normed linear spaces, then A is said to be
if and only if
In case that Y = F the field of scalars, equal to either ℝ or ℂ, ℒ
is known as the dual space, written
here as X′. Actually it is more often written as X∗ but I prefer the former notation because the latter is
sometimes used to denote a purely algebraic dual space, meaning only that its elements are linear
maps into F with no requirement of continuity. Doubtless there are drawbacks to my notation
Thus P the projection map in the above is in ℒ
There is a general easy result about ℒ
which follows. It says that these linear maps are
Theorem 23.1.13 Let X and Y be two normed linear spaces and let L : X → Y be linear
(L(ax + by) = aL(x) + bL(y) for a,b scalars and x,y ∈ X). The following are equivalent
a.) L is continuous at 0
b.) L is continuous
c.) There exists K > 0 such that
Y ≤ K
X for all x ∈ X (L is bounded).
Proof: a.)⇒b.) Let xn → x. It is necessary to show that Lxn → Lx. But (xn − x) → 0 and so from
continuity at 0, it follows
so Lxn → Lx. This shows a.) implies b.).
b.)⇒c.) Since L is continuous, L is continuous at 0. Hence ||Lx||Y < 1 whenever ||x||X ≤ δ for some δ.
Therefore, suppressing the subscript on the ||||,
c.)⇒a.) follows from the inequality given in c.). ■
The following theorem is called the Riesz representation theorem for the dual of a Hilbert space. If
z ∈ H then define an element f ∈ H′ by the rule
. It follows from the Cauchy Schwarz
inequality and the properties of the inner product that f ∈ H′
. The Riesz representation theorem says that
all elements of H′
are of this form.
Theorem 23.1.14 Let H be a Hilbert space and let f ∈ H′. Then there exists a unique z ∈ H such
for all x ∈ H.
Proof: Letting y,w ∈ H the assumption that f is linear implies
which shows that yf(w) − f(y)w ∈ f−1
, which is a closed subspace of
is continuous. If
is the zero map and z
= 0 is the unique element of H
which satisfies 23.8
w ≡ u − Pu≠
0. Thus (y,w
) = 0 for all y ∈ f−1
. In particular, let
) − f
where x ∈ H
is arbitrary. Therefore,
Thus, solving for f
and using the properties of the inner product,
Let z = f(w)w∕||w||2. This proves the existence of z. If f
2, for all x ∈ H
for all x ∈ H
= 0 which implies, upon taking
= z1 − z2
If R : H → H′ is defined by Rx
the Riesz representation theorem above states this map is
onto. This map is called the
Riesz map. It is routine to show R
is conjugate linear and
so it is conjugate linear meaning it goes across plus signs and you factor out conjugates.