Kenneth Kuttler

206 CHAPTER 11. MATRICES AND THE INNER PRODUCT

Proof:We only need to show that this satisfies the orthogonality condition 11.7. Butthis is fairly obvious because, from properties of the inner product and y given above,(

y,u j)=

∑k=1

(b,uk)uk,u j

∑k=1

(b,uk)(uk,u j) =(b,u j

)Thus

(b−y,u j

)= 0. Since this holds for every basis vector, it holds for every z ∈V also.

(z,b−y) =

∑j=1

(z,u j

)u j,b−y

∑j=1

(z,u j

)(u j,b−y) = 0

Therefore, the orthogonality condition holds for y given by the above formula and so yequals the above sum in 11.10. ■

The sum in 11.10 is the Fourier series approximation to b. The scalars (b,uk) are theFourier coefficients. Note that all this works any time you have a norm which comes froman inner product, something which satisfies the same axioms as the dot product. That is,|x|= (x,x) where (·, ·) satisfies the inner product axioms:

1. ( f ,g) = (g, f )

2. (a f +bg,h) = a( f ,h)+b(g,h)

3. ( f , f )≥ 0 and equals 0 only if f = 0

The conjugate is placed on the (g, f ) to include the case of a complex inner product.Just ignore it in the case where the scalars are real numbers.

Now we generalize these ideas more.

Theorem 11.6.16 Let V be a finite dimensional subspace of an inner product space X,something with an inner product. (X is a nonempty set which satisfies the vector spaceaxioms. In addition it has an inner product satisfying the inner product axioms.) If b ∈ X ,there exists a unique y ∈V such that |b− y| ≤ |b− z| for all z ∈V . This point is character-ized by (b− y,z) = 0 for all z ∈V .

Proof: Letting t ∈ R,

|b− (y+ tz)|2 = |b− y|2−2t (b− y,z)+ t2 |z|2

If y is closest to b then taking the derivative and setting t = 0, we must have (b− y,z) = 0.Conversely, if this equals zero, let t = 1 and you have

|b− (y+ z)|2 = |b− y|2 + |z|2

and so y solves the minimization property. It only remains to show the existence of such ysatisfying (b− y,z)= 0. However, using the Gram Schmidt process, there is an orthonormalbasis {u1, · · · ,un} whose span is V . Then all that remains is to verify that ∑

ni=1 (b,ui)ui

satisfies the orthogonality condition. Indeed,(b−

∑i=1

(b,ui)ui,u j

)= (b,u j)−

∑i=1

(b,ui)δ i j = 0

Since the ui are a basis, it follows that (b− y,z) = 0 for all z ∈V . ■Note that any time the norm comes from an inner product, something which satisfies

the properties of the dot product, all of this holds. You don’t need to be considering vectorsin Rn. It was only the axioms of the inner product which were used.

206 CHAPTER 11. MATRICES AND THE INNER PRODUCTProof:We only need to show that this satisfies the orthogonality condition 11.7. Butthis is fairly obvious because, from properties of the inner product and y given above,(y,w,) = (i su )mcu) = y (b, ux) (Ux, Uj) = (b, w;)k=1 k=1Thus (b —Yy,u i) = 0. Since this holds for every basis vector, it holds for every z € V also.m m(z,b-y) = (i (z,u,;) Hoo) = ye (z,u,;) (uj,b—y) =0j=l j=lTherefore, the orthogonality condition holds for y given by the above formula and so yequals the above sum in 11.10.The sum in 11.10 is the Fourier series approximation to b. The scalars (b, u;) are theFourier coefficients. Note that all this works any time you have a norm which comes froman inner product, something which satisfies the same axioms as the dot product. That is,|a| = (a,x) where (.,-) satisfies the inner product axioms:1. (f,8) =(s,f)2. (af +bg,h) =a(f,h) +b(g,h)3. (f, f) => 0 and equals 0 only if f =0The conjugate is placed on the (g, f) to include the case of a complex inner product.Just ignore it in the case where the scalars are real numbers.Now we generalize these ideas more.Theorem 11.6.16 Let V be a finite dimensional subspace of an inner product space X,something with an inner product. (X is a nonempty set which satisfies the vector spaceaxioms. In addition it has an inner product satisfying the inner product axioms.) If b € X,there exists a unique y € V such that |b—y| < |b—z| for all z € V. This point is character-ized by (b—y,z) =O for allz€V.Proof: Letting t € R,|b —(y+1t2)|° = |b—y|? —2¢ (b—y,z) +0 |z|?If y is closest to b then taking the derivative and setting t = 0, we must have (b—y,z) = 0.Conversely, if this equals zero, let t = 1 and you have|b—(y+2z)|? = |b-yl? + |z/and so y solves the minimization property. It only remains to show the existence of such ysatisfying (b — y,z) =0. However, using the Gram Schmidt process, there is an orthonormalbasis {u1,-+-,Un} whose span is V. Then all that remains is to verify that Y?_, (b, uj) ujsatisfies the orthogonality condition. Indeed,(i y: (oa) = (b,uj) —i=lMs:(b, uj) Oi; =0i=Since the u; are a basis, it follows that (b—y,z) =0 for allzcV.Note that any time the norm comes from an inner product, something which satisfiesthe properties of the dot product, all of this holds. You don’t need to be considering vectorsin R”. It was only the axioms of the inner product which were used.