Chapter 12

Inner Product Spaces, Least SquaresIn this chapter is a more complete discussion of important theorems for inner productspaces. These results are presented for inner product spaces, the typical example beingCn or Rn. The extra generality is used because most of the ideas have a straight forwardgeneralization to something called a Hilbert space which is just a complete inner productspace. First is a major result about projections.

12.1 Orthogonal ProjectionsRecall that any finite dimensional normed linear space is complete. The following defini-tion includes the case where the norm comes from an inner product.

Definition 12.1.1 Let (H,(·, ·)) be a complete inner product space. This means the normcomes from an inner product as described on Page 263, |v| ≡ (v,v)1/2. Such a space iscalled a Hilbert space

As shown earlier, if H is finite dimensional, then it is a Hilbert space automatically.The following is the definition of a convex set. This is a set with the property that the linesegment between any two points in the set is in the set.

Definition 12.1.2 A nonempty subset K of a vector space is said to be convex if wheneverx,y ∈ K and t ∈ [0,1] , it follows that tx+(1− t)y ∈ K.

Theorem 12.1.3 Let K be a closed and convex nonempty subset of a Hilbert space and lety ∈ H. Also let

λ ≡ inf{|x− y| : x ∈ K}Then if {xn} ⊆ K is a sequence such that limn→∞ |xn− y| = λ , then it follows that {xn} isa Cauchy sequence and limn→∞ xn = x ∈ K with |x− y| = λ . Also if |x− y| = λ = |x̂− y| ,then x̂ = x.

Proof: Recall the parallelogram identity valid in any innner product space:

|x+ y|2 + |x− y|2 = 2 |x|2 +2 |y|2

First consider the claim about uniqueness. Letting x, x̂ be as given,∣∣∣∣x+ x̂2− y∣∣∣∣2 + ∣∣∣∣x− x̂

2

∣∣∣∣2 =

∣∣∣∣x− y2

+x̂− y

2

∣∣∣∣2 + ∣∣∣∣x− x̂2

∣∣∣∣2= 2

∣∣∣∣x− y2

∣∣∣∣2 +2∣∣∣∣ x̂− y

2

∣∣∣∣2 = λ2

Since x+x̂2 ∈ K due to convexity, this is a contradiction unless x = x̂ since it shows that x+x̂

2is closer to y than λ .

Now consider the minimizing sequence. From the same computation just given,∣∣∣∣xn + xm

2− y∣∣∣∣2 + ∣∣∣∣xn− xm

2

∣∣∣∣2 = 2∣∣∣∣xn− y

2

∣∣∣∣2 +2∣∣∣∣xm− y

2

∣∣∣∣2=

12|xn− y|2 + 1

2|xm− y|2

307