172 CHAPTER 6. SPECTRAL THEORY

Then this implies f (x+ v) =

f (x) +

n∑j=1

∂f

∂xj(x) vk+

1

2vTH (x)v+

1

2

(vT (H (x+tv)−H (x))v

). (6.20)

Using the above formula, here is the second derivative test.

Theorem 6.7.7 In the above situation, suppose fxj(x) = 0 for each xj . Then if H (x) has

all positive eigenvalues, x is a local minimum for f . If H (x) has all negative eigenvalues,then x is a local maximum. If H (x) has a positive eigenvalue, then there exists a directionin which f has a local minimum at x, while if H (x) has a negative eigenvalue, there existsa direction in which H (x) has a local maximum at x.

Proof: Since fxj (x) = 0 for each xj , formula 6.20 implies

f (x+ v) = f (x)+1

2vTH (x)v+

1

2

(vT (H (x+tv)−H (x))v

)where H (x) is a symmetric matrix. Thus, by Corollary 6.4.12 H (x) has all real eigenvalues.Suppose first that H (x) has all positive eigenvalues and that all are larger than δ2 > 0.Then H (x) has an orthonormal basis of eigenvectors, {vi}ni=1 and if u is an arbitrary vector,u =

∑nj=1 ujvj where uj = u · vj . Thus

uTH (x)u =

(n∑

k=1

ukvTk

)H (x)

 n∑j=1

ujvj

 =

n∑j=1

u2jλj ≥ δ2n∑

j=1

u2j = δ2 |u|2 .

From 6.20 and the continuity of H, if v is small enough,

f (x+ v) ≥ f (x) +1

2δ2 |v|2 − 1

4δ2 |v|2 = f (x) +

δ2

4|v|2 .

This shows the first claim of the theorem. The second claim follows from similar reasoning.Suppose H (x) has a positive eigenvalue λ2. Then let v be an eigenvector for this eigenvalue.From 6.20,

f (x+tv) = f (x)+1

2t2vTH (x)v+

1

2t2(vT (H (x+tv)−H (x))v

)which implies

f (x+tv) = f (x)+1

2t2λ2 |v|2 +1

2t2(vT (H (x+tv)−H (x))v

)≥ f (x)+

1

4t2λ2 |v|2

whenever t is small enough. Thus in the direction v the function has a local minimum atx. The assertion about the local maximum in some direction follows similarly. ■

This theorem is an analogue of the second derivative test for higher dimensions. As inone dimension, when there is a zero eigenvalue, it may be impossible to determine from theHessian matrix what the local qualitative behavior of the function is. For example, consider

f1 (x, y) = x4 + y2, f2 (x, y) = −x4 + y2.

Then Dfi (0, 0) = 0 and for both functions, the Hessian matrix evaluated at (0, 0) equals(0 0

0 2

)but the behavior of the two functions is very different near the origin. The second has asaddle point while the first has a minimum there.