4.6. EXERCISES 123

18. ↑Let X be as above, a real inner product space, and let V ≡ span(v1, ...,vn) . Let u∈Xand z ∈ V . Show that |u− z| = inf{|u− v| : v ∈V} if and only if (u− z,vi) = 0 forall vi. Note that the vi might not be linearly independent. Also show that |u− z|2 =|u|2− (z,u) .

19. ↑ Let G be the matrix of Problem 17 where {v1, ...,vn} is linearly independent andV ≡ span(v1, ...,vn) ⊆ X , an inner product space. Let x ≡ ∑i xivi,y ≡ ∑i yivi be twovectors of V. Show that (x,y) = ∑i, j xiGi jx j. Show that z ≡ ∑i zivi,z is closest tou ∈ X if and only if for all i = 1, ...,n,(u,vi) = ∑ j Gi jz j. This gives a system oflinear equations which must be satisfied by the zi in order that z just given is the bestapproximation to u. Next show that there exists such a solution thanks to Problem17 which says that the matrix G is invertible, and if G−1 has i jth component Gi j, onefinds that ∑ j Gi j (u,v j) = zi.

20. ↑ In the situation of the above problems, suppose A is an m× n matrix. Use Prob-lem 18 to show that for y ∈ Rm, there always exists a solution x to the system ofequations ATy = AT Ax. Explain how this is in a sense the best you can do to solvey= Ax even though this last system of equations might not have a solution. Here AT

is the transpose of the matrix A. The equations ATy = AT Ax are called the normalequations for the least squares problem. Hint: Verify that

(ATy,x

)= (y,Ax). Let

the subspace V be A(Rn), the vectors spanning it being {Ae1, ...,Aen}. From theabove problem, there exists Ax in V which is closest to y. Now use the character-ization of this vector (y−Ax,Az) = 0 for all z ∈ Rn,Az being a generic vector inA(Rn).

21. ↑As an example of an inner product space, consider C ([0,1]) with the inner product∫ 10 f (x)g(x)dx where this is the ordinary integral from calculus. Abusing notation,

let {xp1 , ...,xpn} with − 12 < p1 < · · ·< pn be functions, (vectors) in C ([0,1]) . Verify

that these vectors are linearly independent. Hint: You might want to use the Cauchyidentity, Theorem 1.9.28.

22. ↑As above, if {v1, ...,vn} is linearly independent, the Grammian is G = G(v1, ...,vn),Gi j ≡ (vi,v j) , then if u /∈ span(v1, ...,vn) ≡ V you could consider G(v1, ...,vn,u) .Then if d ≡ min{|u− v| : v ∈ span(v1, ...,vn)} , show that d2 = detG(v1,...,vn,u)

detG(v1,...,vn). Jus-

tify the following steps. Letting z be the closest point of V to u, from the above,(u−∑

ni=1 zivi,vp

)= 0 for each vp and so

(u,vp) =n

∑i=1

(vp,vi)zi (∗)

Also, since (u− z,v) = 0 for all v ∈V, |u|2 = |u− z+ z|2 = |u− z|2 + |z|2 so

|u|2 =

∣∣∣∣∣u− n

∑i=1

zivi

∣∣∣∣∣2

+

∣∣∣∣∣ n

∑i=1

zivi

∣∣∣∣∣2

= d2 +

∣∣∣∣∣ n

∑i=1

zivi

∣∣∣∣∣2

= d2 +∑j

=(u,v j)︷ ︸︸ ︷∑

i(v j,vi)ziz j = d2 +∑

j(u,v j)z j