8.7. SECOND DERIVATIVE TEST 219

where ei are the usual basis vectors. Lettin v= ∑ni=1 viei, the second derivative term in

8.22 reduces to

12 ∑

i, jD2 f (x+tv)(ei)(e j)viv j =

12 ∑

i, jHi j (x+tv)viv j

where

Hi j (x+tv) = D2 f (x+tv)(ei)(e j) =∂ 2 f (x+tv)

∂x j∂xi.

Definition 8.7.1 The matrix whose i jth entry is ∂ 2 f (x)∂x j∂xi

is called the Hessian matrix,denoted asH (x).

From Theorem 7.10.1, this is a symmetric real matrix, thus self adjoint. By the conti-nuity of the second partial derivative,

f (x+v) = f (x)+D f (x)v+12vT H (x)v+

12(vT (H (x+tv)−H (x))v

). (8.23)

where the last two terms involve ordinary matrix multiplication and

vT =(

v1 · · · vn)

for vi the components of v relative to the standard basis.

Definition 8.7.2 Let f : D→ R where D is a subset of some normed vector space.Then f has a local minimum at x ∈ D if there exists δ > 0 such that for all y ∈ B(x,δ )

f (y)≥ f (x) .

f has a local maximum at x ∈ D if there exists δ > 0 such that for all y ∈ B(x,δ )

f (y)≤ f (x) .

Theorem 8.7.3 If f : U → R where U is an open subset of Rn and f is C2, supposeD f (x) = 0. Then if H (x) has all positive eigenvalues, x is a local minimum. If theHessian matrix H (x) has all negative eigenvalues, then x is a local maximum. If H (x)has a positive eigenvalue, then there exists a direction in which f has a local minimum atx, while if H (x) has a negative eigenvalue, there exists a direction in which H (x) has alocal maximum at x.

Proof: Since D f (x)= 0, formula 8.23 holds and by continuity of the second derivative,H (x) is a symmetric matrix. Thus H (x) has all real eigenvalues. Suppose first that H (x)

has all positive eigenvalues and that all are larger than δ2 > 0. Then by Theorem 1.4.1,

H (x) has an orthonormal basis of eigenvectors, {vi}ni=1 and if u is an arbitrary vector,

such that u= ∑nj=1 u jv j where u j = u ·v j, then

uT H (x) u=n

∑j=1

u jvTj H (x)

n

∑j=1

u jv j