310 CHAPTER 12. INNER PRODUCT SPACES, LEAST SQUARES

Proof: Let u1≡ v1/ |v1| . Thus for k = 1, span(u1) = span(v1) and {u1} is an orthonor-mal set. Now suppose for some k < n, u1, · · · , uk have been chosen such that (u j,ul) = δ jland span(v1, · · · ,vk) = span(u1, · · · ,uk). Then define

uk+1 ≡vk+1−∑

kj=1 (vk+1,u j)u j∣∣∣vk+1−∑kj=1 (vk+1,u j)u j

∣∣∣ , (12.2)

where the denominator is not equal to zero because the v j form a basis, and so

vk+1 /∈ span(v1, · · · ,vk) = span(u1, · · · ,uk)

Thus by induction,

uk+1 ∈ span(u1, · · · ,uk,vk+1) = span(v1, · · · ,vk,vk+1) .

Also, vk+1 ∈ span(u1, · · · ,uk,uk+1) which is seen easily by solving 10.13 for vk+1, and itfollows

span(v1, · · · ,vk,vk+1) = span(u1, · · · ,uk,uk+1) .

If l ≤ k,

(uk+1,ul) =C

((vk+1,ul)−

k

∑j=1

(vk+1,u j)(u j,ul)

)=

C

((vk+1,ul)−

k

∑j=1

(vk+1,u j)δ l j

)=C ((vk+1,ul)− (vk+1,ul)) = 0.

The vectors,{

u j}n

j=1 , generated in this way are therefore orthonormal because each vectorhas unit length. ■

Theorem 12.1.7 Let K be a nonempty closed subspace of H a Hilbert space. Let y ∈ H.Then x = Py, the closest point in K to y if and only if

(y− x,w) = 0

for all w ∈ K. If K is a finite dimensional subspace of H then by Lemma 12.1.6 it has anorthonormal basis {u1, · · · ,un} . Then Py = ∑

nk=1 (y,uk)uk. In particular, if y ∈ K, then

y = Py = ∑nk=1 (y,uk)uk

Proof: From Theorem 12.1.4, x = Py,x ∈ K if and only if for all z ∈ K,

Re(y− x,z− x)≤ 0

However, if w ∈ K, let z = x+w and this shows that x = Py if and only if for all w ∈ K,

Re(y− x,w)≤ 0

From Proposition 12.1.5 above, this happens if and only if (y− x,w) = 0.It only remains to verify the orthogonality condition for the vector claimed to be the

closest point.(y−

n

∑k=1

(y,uk)uk,u j

)= (y,u j)−

n

∑k=1

(y,uk)(uk,u j) = (y,u j)− (y,u j) = 0