7.7 The Singular Value Decomposition∗
In this section, A will be an m × n matrix. To begin with, here is a simple lemma.
Lemma 7.7.1 Let A be an m × n matrix. Then A∗A is Hermitian and all its eigenvalues are
Proof: It is obvious that A∗A is Hermitian because
. Suppose A∗Ax
Definition 7.7.2 Let A be an m × n matrix. The singular values of A are the square roots of the
positive eigenvalues of A∗A.
With this definition and lemma here is the main theorem on the singular value decomposition.
Theorem 7.7.3 Let A be an m×n matrix. Then there exist unitary matrices, U and V of the appropriate
size such that
where σ is of the form
for the σi the singular values of A.
Proof: By the above lemma and Theorem 7.4.6 there exists an orthonormal basis,
where σi2 >
0 for i
and equals zero if
i > k.
Thus for i > k, Avi
For i = 1,
define ui ∈ Fm
Thus Avi = σiui. Now
is an orthonormal set of vectors in Fm.
to an orthonormal basis for all of Fm,
while V ≡
is the matrix which has the ui
as columns and V
is defined as the matrix
which has the vi
as columns. Then
where σ is given in the statement of the theorem. ■
The singular value decomposition has as an immediate corollary the following interesting
Corollary 7.7.4 Let A be an m×n matrix. Then the rank of A and A∗equals the number of singular
Proof: Since V and U are unitary, it follows that
Also since U,V
This is based on the simple observation that for A an m × n matrix, the dimension of Im
same as the dimension of
are invertible matrices of the right size. Indeed,
being invertible maps Fn
The dimension of Im
must be the same because
is one to one. Thus if a basis for Im
columns of A,
then a basis for UA