14.13 The Singular Value Decomposition
In this section, A will be an m × n matrix. To begin with, here is a simple lemma observed
Lemma 14.13.1 Let A be an m × n matrix. Then A∗A is self adjoint and all its eigenvalues are
Proof: It is obvious that A∗A is self adjoint. Suppose A∗Ax = λx. Then
Definition 14.13.2 Let A be an m × n matrix. The singular values of A are the square roots of
the positive eigenvalues of A∗A.
With this definition and lemma here is the main theorem on the singular value decomposition. In all
that follows, I will write the following partitioned matrix
where σ denotes an r × r diagonal matrix of the form
and the bottom row of zero matrices in the partitioned matrix, as well as the right columns of zero matrices
are each of the right size so that the resulting matrix is m × n. Either could vanish completely. However,
I will write it in the above form. It is easy to make the necessary adjustments in the other two
Theorem 14.13.3 Let A be an m × n matrix. Then there exist unitary matrices, U and V of the
appropriate size such that
where σ is of the form
for the σi the singular values of A, arranged in order of decreasing size.
Proof: By the above lemma and Theorem 14.11.4 there exists an orthonormal basis,
such that A∗Avi
where σi2 >
0 for i
and equals zero if i > k.
be arranged in decreasing order. It is desired to have
and so if U =
one needs to have for j ≤ k, σjuj
Then for i,j ≤ k,
Now extend to an orthonormal basis of Fm,
If i > k,
so Avi = 0. Then for σ given as above in the statement of the theorem, it follows that
The singular value decomposition has as an immediate corollary the following interesting
Corollary 14.13.4 Let A be an m × n matrix. Then the rank of both A and A∗equals the number
of singular values.
Proof: Since V and U are unitary, they are each one to one and onto and so it follows
Also since U,V are unitary,