12.9. THE SINGULAR VALUE DECOMPOSITION 313

Now let B = S1/2Σ−1S1/2. Then B is positive and self adjoint also and so there ex-ists U unitary such that B = U∗DU where D is the diagonal matrix having the positivescalars λ1, · · · , λp down the main diagonal. Solving for Σ−1 in terms of B, this yieldsS−1/2BS−1/2 = Σ−1 and so

ln(det(Σ−1

))= ln

(det(S−1/2

)det (B) det

(S−1/2

))= ln

(det(S−1

))+ ln (det (B))

which yieldsC (S) + n ln (det (B))− trace (B)

as the thing to maximize. Of course this yields

C (S) + n ln

(p∏

i=1

λi

)−

p∑i=1

λi

= C (S) + n

p∑i=1

ln (λi)−p∑

i=1

λi

as the quantity to be maximized. To do this, take ∂/∂λk and set equal to 0. This yieldsλk = n. Therefore, from the above, B = U∗nIU = nI. Also from the above,

B−1 =1

nI = S−1/2ΣS−1/2

and so

Σ =1

nS =

1

n

n∑i=1

(xi −m) (xi −m)∗

This has shown that the maximum likelihood estimates are

m = x̄ ≡ 1

n

n∑i=1

xi, Σ =1

n

n∑i=1

(xi −m) (xi −m)∗.

12.9 The Singular Value Decomposition

In this section, A will be an m× n matrix. To begin with, here is a simple lemma.

Lemma 12.9.1 Let A be an m×n matrix. Then A∗A is self adjoint and all its eigenvaluesare nonnegative.

Proof: It is obvious that A∗A is self adjoint. Suppose A∗Ax = λx. Then λ |x|2 =(λx,x) = (A∗Ax,x) = (Ax,Ax) ≥ 0. ■

Definition 12.9.2 Let A be an m×n matrix. The singular values of A are the square rootsof the positive eigenvalues of A∗A.

With this definition and lemma here is the main theorem on the singular value decom-position. In all that follows, I will write the following partitioned matrix(

σ 0

0 0

)where σ denotes an r × r diagonal matrix of the form

σ1 0. . .

0 σk

