346 CHAPTER 18. OPTIMIZATION

Therefore, F (c) = 0 by the way K was chosen and also F (x) = 0. Then F ′ (t) =

−

(f ′ (t)−

(∑

nk=1

f (k)(t)k! k (x− t)k−1−∑

nk=1

f (k+1)(t)k! (x− t)k

+K (n+1)(x− t)n

))

= −

(f ′ (t)−

(∑

n−1k=0

f (k+1)(t)k! (x− t)k−∑

nk=1

f (k+1)(t)k! (x− t)k

+K (n+1)(x− t)n

))= −

(f ′ (t)−

(f ′ (t)− f (n+1) (t)(x− t)n +K (n+1)(x− t)n

))= − f ′ (t)+ f ′ (t)− f (n+1) (t)(x− t)n +K (n+1)(x− t)n

= − f (n+1) (t)1n!

(x− t)n +K (n+1)(x− t)n

By the mean value theorem or Rolle’s theorem, there exists ξ between x and c such thatF ′ (ξ ) = 0. Therefore,

− f (n+1) (ξ )1n!

(x−ξ )n +K (n+1)(x−ξ )n = 0

and so

K (n+1) = f (n+1) (ξ )1n!

K =f (n+1) (ξ )

(n+1)!■

The term f (n+1)(ξ )(n+1)! (x− c)n+1 , is called the remainder and this particular form of the

remainder is called the Lagrange form of the remainder.

Definition 18.7.2 The matrix(

∂ 2 f∂xi∂x j

(x))

is called the Hessian matrix, denoted by H (x).

Now recall the Taylor formula with the Lagrange form of the remainder.

Theorem 18.7.3 Let h : (−δ ,1+δ )→ R have m+ 1 derivatives. Then there exists t ∈(0,1) such that

h(1) = h(0)+m

∑k=1

h(k) (0)k!

+h(m+1) (t)(m+1)!

.

Now let f : U → R where U is an open subset of Rn. Suppose f ∈C2 (U). Let x ∈Uand let r > 0 be such that

B(x,r)⊆U.

Then for ||v||< r considerf (x+tv)− f (x)≡ h(t)

for t ∈ [0,1]. Then from Taylor’s theorem for the case where m = 2 and the chain rule,using the repeated index summation convention and the chain rule,

h′ (t) =∂ f∂xi

(x+ tv)vi, h′′ (t) =∂ 2 f

∂x j∂xi(x+ tv)viv j.