7.5. SECOND DERIVATIVE TEST 171

where ei are the usual basis vectors. Lettin v = ∑ni=1 viei, the second derivative term in

7.20 reduces to

12 ∑

i, jD2 f (x+tv)(ei)(e j)viv j =

12 ∑

i, jHi j (x+tv)viv j

where

Hi j (x+tv) = D2 f (x+tv)(ei)(e j) =∂ 2 f (x+tv)

∂x j∂xi.

Definition 7.5.1 The matrix whose i jth entry is ∂ 2 f (x)∂x j∂xi

is called the Hessian matrix,denoted as H (x).

From Theorem 6.10.1, this is a symmetric real matrix, thus self adjoint. By the conti-nuity of the second partial derivative,

f (x+v) = f (x)+D f (x)v+12vT H (x)v+

12(vT (H (x+tv)−H (x))v

). (7.21)

where the last two terms involve ordinary matrix multiplication and

vT =(

v1 · · · vn)

for vi the components of v relative to the standard basis.

Definition 7.5.2 Let f : D→ R where D is a subset of some normed vector space.Then f has a local minimum at x ∈ D if there exists δ > 0 such that for all y ∈ B(x,δ ),f (y)≥ f (x) . Also f has a local maximum at x ∈ D if there exists δ > 0 such that for ally ∈ B(x,δ ) , f (y)≤ f (x).

Theorem 7.5.3 If f : U → R where U is an open subset of Rn and f is C2, supposeD f (x) = 0. Then if H (x) has all positive eigenvalues, x is a local minimum. If theHessian matrix H (x) has all negative eigenvalues, then x is a local maximum. If H (x)has a positive eigenvalue, then there exists a direction in which f has a local minimum atx, while if H (x) has a negative eigenvalue, there exists a direction in which H (x) has alocal maximum at x.

Proof: Since D f (x)= 0, formula 7.21 holds and by continuity of the second derivative,H (x) is a symmetric matrix. Thus H (x) has all real eigenvalues. Suppose first that H (x)

has all positive eigenvalues and that all are larger than δ2 > 0. Then by Theorem 1.4.1,

H (x) has an orthonormal basis of eigenvectors, {vi}ni=1 and if u is an arbitrary vector,

such that u= ∑nj=1 u jv j where u j = u ·v j, then

uT H (x) u=n

∑j=1

u jvTj H (x)

n

∑j=1

u jv j =n

∑j=1

u2jλ j ≥ δ

2n

∑j=1

u2j = δ

2 |u|2 .

From 7.21 and the continuity of H, if v is small enough,

f (x+v)≥ f (x)+12

δ2 |v|2− 1

4δ

2 |v|2 = f (x)+δ

2

4|v|2 .