17.3. APPROXIMATION AND LEAST SQUARES 411

Theorem 17.3.2 Let V be an inner product space and let U be an n dimensional subspaceof V . Then if y ∈V is given, there exists a unique x ∈U such that

|y−x| ≤ |y−w|

for all w ∈U and in addition, there is a formula for x in terms of any orthonormal basisfor U,{u1, · · · ,un},

x =n

∑k=1⟨y,uk⟩uk

Proof: By Lemma 17.3.1 there is at most one minimizer and it is characterized by thecondition

⟨y−x,w⟩= 0

for all w ∈U . Let {uk}nk=1 be an orthonormal basis for U . By the Gram Schmidt process,

Lemma 17.2.2, there exists such an orthonormal basis. Now it only remains to verify that⟨y−

n

∑k=1⟨y,uk⟩uk,w

⟩= 0

for all w. Since {uk}nk=1 is a basis, it suffices to verify that⟨

y−n

∑k=1⟨y,uk⟩uk,ul

⟩= 0, all l = 1,2, · · · ,n

However, from the properties of the inner product,⟨y−

n

∑k=1⟨y,uk⟩uk,ul

⟩= ⟨y,ul⟩−

n

∑k=1⟨y,uk⟩⟨uk,ul⟩

= ⟨y,ul⟩−n

∑k=1⟨y,uk⟩δ kl = ⟨y,ul⟩−⟨y,ul⟩= 0. ■

Note it follows that for any orthonormal basis {uk}nk=1 , the same unique vector x is

obtained asn

∑k=1⟨y,uk⟩uk

and it is the unique minimizer of w7→|y−w|. This is stated in the following corollary forthe sake of emphasis.

Corollary 17.3.3 Let V be an inner product space and let U be an n dimensional subspaceof V. Then for y given in V, and {uk}n

k=1 ,{vk}nk=1 two orthonormal bases for U,

n

∑k=1⟨y,uk⟩uk =

n

∑k=1⟨y,vk⟩vk

The scalars ⟨y,uk⟩ are called the Fourier coefficients.