310 CHAPTER 13. MATRICES AND THE INNER PRODUCT
•
• • AFnAu
y
Ax
Theorem 13.3.3 Let y ∈ Fm and let A be an m×n matrix. Then there exists x ∈ Fn mini-mizing the function x 7→ |y−Ax|2 . Furthermore, x minimizes this function if and only if
((y−Ax) ,Au) = 0
for all u ∈ Fn.
Proof: First consider the characterization of the minimizer. Let u ∈ Fn. Let |θ |= 1,
θ̄ (y−Ax,Au) = |(y−Ax,Au)|
Now consider the function of t ∈ R
p(t)≡ |y−(Ax+tθAu)|2 = ((y−Ax)− tθAu,(y−Ax)− tθAu)
= |y−Ax|2 + t2 |Au|2−2t Re(y−Ax,θAu)≥ |y−Ax|2
= |y−Ax|2 + t2 |Au|2−2t Re θ̄ (y−Ax,Au)
= |y−Ax|2 + t2 |Au|2−2t |(y−Ax,Au)|
Then if |y−Ax| is as small as possible, this will occur when t = 0 and so p′ (0) = 0. Butthis says
|(y−Ax,Au)|= 0
You could also use Lemma 13.3.2 to see this is 0. Since u was arbitrary, this proves onedirection.
Conversely, if this quantity equals 0,
|y−(Ax+Au)|2 = |y−Ax|2 + |Ax−Au|2 +2Re(y−Ax,Au)= |y−Ax|2 + |Ax−Au|2
and so the minimum occurs at any point z such that Ax = Az.Does there exist an x which minimizes this function? From what was just shown,
it suffices to show that there exists x such that ((y−Ax) ,Au) for all u. By the GrammSchmidt process there exists an orthonormal basis {Axk} for A(Fn). Then for a given y,
(y−
r
∑k=1
(y,Axk)Axk,Ax j
)= (y,Ax j)−
r
∑k=1
(y,Axk)
δ k j︷ ︸︸ ︷(Axk,Ax j) = 0.
In particular, (y−A
(r
∑k=1
(y,Axk)xk
),w
)= 0