714 CHAPTER 22. THE DERIVATIVE
The matrix of Df(x) is the matrix having the ith column equal to Df(x)ei and so it isonly necessary to compute this. Let t be a small real number such that both
f(x+ tei)− f(x)−Df(x)(tei)
t=
o(t)t
Therefore,f(x+ tei)− f(x)
t= Df(x)(ei)+
o(t)t
The limit exists on the right and so it exists on the left also. Thus
∂ f(x)∂xi
≡ limt→0
f(x+ tei)− f(x)t
= Df(x)(ei)
and so the matrix of the derivative is just the matrix which has the ith column equal to theith partial derivative of f. Note that this shows that whenever f is differentiable, it followsthat the partial derivatives all exist. It does not go the other way however as discussed later.
Theorem 22.5.1 Let f : U ⊆ Fn → Fm and suppose f is differentiable at x. Then all thepartial derivatives ∂ fi(x)
∂x jexist and if Jf(x) is the matrix of the linear transformation, Df(x)
with respect to the standard basis vectors, then the i jth entry is given by ∂ fi∂x j
(x) also denoted
as fi, j or fi,x j . It is the matrix whose ith column is
∂ f(x)∂xi
≡ limt→0
f(x+ tei)− f(x)t
.
Of course there is a generalization of this idea called the directional derivative.
Definition 22.5.2 In general, the symbol
Dvf(x)
is defined by
limt→0
f(x+ tv)− f(x)t
where t ∈ F. In case |v| = 1 and the norm is the standard Euclidean norm, this is calledthe directional derivative. More generally, with no restriction on the size of v and in anylinear space, it is called the Gateaux derivative. f is said to be Gateaux differentiable at xif there exists Dvf(x) such that
limt→0
f(x+ tv)− f(x)t
= Dvf(x)
where v→Dvf(x) is linear. Thus we say it is Gateaux differentiable if the Gateaux deriva-tive exists for each v and v→ Dvf(x) is linear. 1
1ReneĢ Gateaux was one of the many young French men killed in world war I. This derivative is named afterhim, but it developed naturally from ideas used in the calculus of variations which were due to Euler and Lagrangeback in the 1700’s.