322 CHAPTER 13. MATRICES AND THE INNER PRODUCT
∣∣∣∣A−A′∣∣∣∣2
F =
∣∣∣∣∣∣∣∣∣∣U(
σ −σ ′ 00 0
)V ∗∣∣∣∣∣∣∣∣∣∣2
F
=r
∑k=l+1
σ2k
Thus A is approximated by A′ where A′ has rank l < r. In fact, it is also true that out ofall matrices of rank l, this A′ is the one which is closest to A in the Frobenius norm.
Thus A is approximated by A′ where A′ has rank l < r. In fact, it is also true that out ofall matrices of rank l, this A′ is the one which is closest to A in the Frobenius norm. Hereis roughly why this is so. First consider approximating 3 0 0 0
0 2 0 00 0 1 0
as well as possible with a rank 2 matrix. It seems clear that the one which will work best is 3 0 0 0
0 2 0 00 0 0 0
More generally if σ is a r× r diagonal matrix in which the positive diagonal entries aredecreasing from upper left to lower right, then the best rank l approximation to(
σ 00 0
)
would be (σ ′ 00 0
)where σ ′ is the upper left l× l corner of σ as in the above example.
Now suppose A is an m×n matrix. Let U,V be unitary and of the right size such that
U∗AV =
(σ r×r 0
0 0
)
Then suppose B approximates A as well as possible in the Frobenius norm. Then you wouldwant
∥A−B∥= ∥U∗AV −U∗BV∥=
∥∥∥∥∥(
σ r×r 00 0
)−U∗BV
∥∥∥∥∥to be as small as possible. Therefore, from the above discussion, you should have
U∗BV =
(σ ′ 00 0
),B =U
(σ ′ 00 0
)V ∗
whereas
A =U
(σ r×r 0
0 0
)V ∗