13.5. THE SINGULAR VALUE DECOMPOSITION 317

Is F (∑rk=1 bkxk) = F (x)?(

F

(r

∑k=1

bkxk

)−F (x) ,F

(r

∑k=1

bkxk

)−F (x)

)

=

((F∗F)

(r

∑k=1

bkxk−x

),

(r

∑k=1

bkxk−x

))

=

(U2

(r

∑k=1

bkxk−x

),

(r

∑k=1

bkxk−x

))

=

(U

(r

∑k=1

bkxk−x

),U

(r

∑k=1

bkxk−x

))

=

(r

∑k=1

bkUxk−Ux,r

∑k=1

bkUxk−Ux

)= 0

Therefore, F (∑rk=1 bkxk)=F (x) and this shows RUx=Fx. From 13.12 and Lemma 13.3.7

R preserves distances. Therefore, by Lemma 13.4.4 R∗R = I. ■

13.5 The Singular Value DecompositionIn this section, A will be an m×n matrix. To begin with, here is a simple lemma.

Lemma 13.5.1 Let A be an m×n matrix. Then A∗A is self adjoint and all its eigenvaluesare nonnegative.

Proof: It is obvious that A∗A is self adjoint. Suppose A∗Ax = λx. Then λ |x|2 =(λx,x) = (A∗Ax,x) = (Ax,Ax)≥ 0. ■

Definition 13.5.2 Let A be an m×n matrix. The singular values of A are the square rootsof the positive eigenvalues of A∗A.

With this definition and lemma here is the main theorem on the singular value decom-position.

Theorem 13.5.3 Let A be an m×n matrix. Then there exist unitary matrices, U and V ofthe appropriate size such that

U∗AV =

(σ 00 0

)where σ is of the form

σ =

σ1 0

. . .

0 σ k

for the σ i the singular values of A.