318 CHAPTER 12. SELF ADJOINT OPERATORS
12.12 The Moore Penrose Inverse
The particular solution of the least squares problem given in 12.18 is important enough thatit motivates the following definition.
Definition 12.12.1 Let A be an m × n matrix. Then the Moore Penrose inverse of A,denoted by A+ is defined as
A+ ≡ V
(σ−1 0
0 0
)U∗.
Here
U∗AV =
(σ 0
0 0
)as above.
Thus A+y is a solution to the minimization problem to find x which minimizes |Ax− y| .In fact, one can say more about this. In the following picture My denotes the set of leastsquares solutions x such that A∗Ax = A∗y.
Myx
A+(y)
ker(A∗A)
Then A+ (y) is as given in the picture.
Proposition 12.12.2 A+y is the solution to the problem of minimizing |Ax− y| for all xwhich has smallest norm. Thus∣∣AA+y − y
∣∣ ≤ |Ax− y| for all x
and if x1 satisfies |Ax1 − y| ≤ |Ax− y| for all x, then |A+y| ≤ |x1| .
Proof: Consider x satisfying 12.17, equivalently A∗Ax =A∗y,(σ2 0
0 0
)V ∗x =
(σ 0
0 0
)U∗y
which has smallest norm. This is equivalent to making |V ∗x| as small as possible becauseV ∗ is unitary and so it preserves norms. For z a vector, denote by (z)k the vector in Fk
which consists of the first k entries of z. Then if x is a solution to 12.17(σ2 (V ∗x)k
0
)=
(σ (U∗y)k
0
)