528 CHAPTER 19. HILBERT SPACES

19.3 Approximations In Hilbert SpaceThe Gram Schmidt process applies in any Hilbert space.

Theorem 19.3.1 Let {x1, · · · ,xn} be a basis for M a subspace of H a Hilbert space. Thenthere exists an orthonormal basis for M, {u1, · · · ,un} which has the property that for eachk ≤ n, span(x1, · · · ,xk) = span(u1, · · · ,uk) . Also if {x1, · · · ,xn} ⊆ H, then

span(x1, · · · ,xn)

is a closed subspace.

Proof: Let {x1, · · · ,xn} be a basis for M. Let u1 ≡ x1/ |x1| . Thus for k = 1, span(u1) =span(x1) and {u1} is an orthonormal set. Now suppose for some k < n, u1, · · · , uk havebeen chosen such that (u j ·ul) = δ jl and span(x1, · · · ,xk) = span(u1, · · · ,uk). Then define

uk+1 ≡xk+1−∑

kj=1 (xk+1 ·u j)u j∣∣∣xk+1−∑kj=1 (xk+1 ·u j)u j

∣∣∣ , (19.3.16)

where the denominator is not equal to zero because the x j form a basis and so

xk+1 /∈ span(x1, · · · ,xk) = span(u1, · · · ,uk)

Thus by induction,

uk+1 ∈ span(u1, · · · ,uk,xk+1) = span(x1, · · · ,xk,xk+1) .

Also, xk+1 ∈ span(u1, · · · ,uk,uk+1) which is seen easily by solving 19.3.16 for xk+1 and itfollows

span(x1, · · · ,xk,xk+1) = span(u1, · · · ,uk,uk+1) .

If l ≤ k,

(uk+1 ·ul) = C

((xk+1 ·ul)−

k

∑j=1

(xk+1 ·u j)(u j ·ul)

)

= C

((xk+1 ·ul)−

k

∑j=1

(xk+1 ·u j)δ l j

)= C ((xk+1 ·ul)− (xk+1 ·ul)) = 0.

The vectors,{

u j}n

j=1 , generated in this way are therefore an orthonormal basis becauseeach vector has unit length.

Consider the second claim about finite dimensional subspaces. Without loss of gener-ality, assume {x1, · · · ,xn} is linearly independent. If it is not, delete vectors until a linearlyindependent set is obtained. Then by the first part, span(x1, · · · ,xn)= span(u1, · · · ,un)≡Mwhere the ui are an orthonormal set of vectors. Suppose {yk} ⊆ M and yk → y ∈ H. Isy ∈M? Let

yk ≡n

∑j=1

ckju j