13.6. APPROXIMATION IN THE FROBENIUS NORM∗ 321

Lemma 13.6.4 Let A be an m×n complex matrix with singular matrix

Σ =

(σ 00 0

)

with σ as defined above. Then||Σ||2F = ||A||2F (13.13)

and the following hold for the Frobenius norm. If U,V are unitary and of the right size,

||UA||F = ||A||F , ||UAV ||F = ||A||F . (13.14)

Proof: From the definition and letting U,V be unitary and of the right size,

||UA||2F ≡ trace(UAA∗U∗) = trace(AA∗) = ||A||2F

Also,||AV ||2F ≡ trace(AVV ∗A∗) = trace(AA∗) = ||A||2F .

It follows||UAV ||2F = ||AV ||2F = ||A||2F .

Now consider 13.13. From what was just shown,

||A||2F = ||UΣV ∗||2F = ||Σ||2F . ■

Of course, this shows that||A||2F = ∑

iσ

2i ,

the sum of the squares of the singular values of A.Why is the singular value decomposition important? It implies

A =U

(σ 00 0

)V ∗

where σ is the diagonal matrix having the singular values down the diagonal. Now some-times A is a huge matrix, 1000×2000 or something like that. This happens in applicationsto situations where the entries of A describe a picture. What also happens is that most ofthe singular values are very small. What if you deleted those which were very small, sayfor all i≥ l and got a new matrix,

A′ ≡U

(σ ′ 00 0

)V ∗?

Then the entries of A′ would end up being close to the entries of A but there is much lessinformation to keep track of. This turns out to be very useful. More precisely, letting

σ =

σ1 0

. . .

0 σ r

 , U∗AV =

(σ 00 0

),