12.12 The Moore Penrose Inverse
The particular solution of the least squares problem given in 12.18 is important enough that it
motivates the following definition.
Definition 12.12.1 Let A be an m×n matrix. Then the Moore Penrose inverse of A, denoted by
A+ is defined as
Thus A+y is a solution to the minimization problem to find x which minimizes
fact, one can say more about this. In the following picture My
denotes the set of least squares
such that A∗Ax
is as given in the picture.
Proposition 12.12.2 A+y is the solution to the problem of minimizing
for all x which
has smallest norm. Thus
and if x1 satisfies
for all x, then
Proof: Consider x satisfying 12.17, equivalently A∗Ax =A∗y,
which has smallest norm. This is equivalent to making
as small as possible because
unitary and so it preserves norms. For z
a vector, denote by
the vector in Fk
of the first k
entries of z.
Then if x
is a solution to 12.17
Thus the first k
entries of V ∗x
are determined. In order
as small as possible, the remaining
n − k
entries should equal zero.
Lemma 12.12.3 The matrix A+ satisfies the following conditions.
Proof: This is routine. Recall
so you just plug in and verify it works. ■
A much more interesting observation is that A+ is characterized as being the unique matrix
which satisfies 12.19. This is the content of the following Theorem. The conditions are sometimes
called the Penrose conditions.
Theorem 12.12.4 Let A be an m×n matrix. Then a matrix A0, is the Moore Penrose inverse
of A if and only if A0 satisfies
Proof: From the above lemma, the Moore Penrose inverse satisfies 12.20. Suppose then that
A0 satisfies 12.20. It is necessary to verify that A0 = A+. Recall that from the singular value
decomposition, there exist unitary matrices, U and V such that
where P is r × r, the same size as the diagonal matrix composed of the singular values on the
Next use the first equation of 12.20 to write
Then multiplying both sides on the left by U∗ and on the right by V,
Therefore, P = σ−1. From the requirement that AA0 is Hermitian,
must be Hermitian. Therefore, it is necessary that
is Hermitian. Then
and so Q = 0.
is Hermitian. Therefore, also
is Hermitian. Thus R = 0 because
which requires Rσ = 0. Now multiply on right by σ−1 to find that R = 0.
Use 12.21 and the second equation of 12.20 to write
This yields from the above in which is was shown that R,Q are both 0
= 0 also and so
The theorem is significant because there is no mention of eigenvalues or eigenvectors in the
characterization of the Moore Penrose inverse given in 12.20. It also shows immediately
that the Moore Penrose inverse is a generalization of the usual inverse. See Problem