12.9 The Singular Value Decomposition
In this section, A will be an m × n matrix. To begin with, here is a simple lemma.
Lemma 12.9.1 Let A be an m×n matrix. Then A∗A is self adjoint and all its eigenvalues
Proof: It is obvious that A∗A is self adjoint. Suppose A∗Ax = λx. Then
Definition 12.9.2 Let A be an m×n matrix. The singular values of A are the square roots
of the positive eigenvalues of A∗A.
With this definition and lemma here is the main theorem on the singular value decomposition.
In all that follows, I will write the following partitioned matrix
where σ denotes an r × r diagonal matrix of the form
and the bottom row of zero matrices in the partitioned matrix, as well as the right columns of zero
matrices are each of the right size so that the resulting matrix is m × n. Either could vanish
completely. However, I will write it in the above form. It is easy to make the necessary
adjustments in the other two cases.
Theorem 12.9.3 Let A be an m×n matrix. Then there exist unitary matrices, U and V of the
appropriate size such that
where σ is of the form
for the σi the singular values of A, arranged in order of decreasing size.
Proof: By the above lemma and Theorem 12.3.2 there exists an orthonormal basis,
such that A∗Avi
where σi2 >
0 for i
zero if i > k.
Let the eigenvalues σi2
be arranged in decreasing order. It is desired to
and so if U =
one needs to have for j ≤ k, σjuj
Then for i,j ≤ k,
Now extend to an orthonormal basis of Fm,
If i > k,
so Avi = 0. Then for σ given as above in the statement of the theorem, it follows that
The singular value decomposition has as an immediate corollary the following interesting
Corollary 12.9.4 Let A be an m×n matrix. Then the rank of A and A∗equals the number
of singular values.
Proof: Since V and U are unitary, they are each one to one and onto and so it follows
Also since U,V are unitary,