Topics In Analysis

5.12 The Frechet Derivative

Let U be an open set in Fⁿ, and let f : U → F^m be a function.

Definition 5.12.1 A function g is o

lim g(v)= 0 (5.12.14) |v|\to0 |v|

(5.12.14)

A function f : U → F^m is differentiable at x ∈ U if there exists a linear transformation L ∈ℒ

such that

f (x+ v) = f (x)+ Lv + o(v)

This linear transformation L is the definition of Df

. This derivative is often called the Frechet derivative. .

Usually no harm is occasioned by thinking of this linear transformation as its matrix taken with respect to the usual basis vectors.

The definition 5.12.14 means that the error,

f (x +v )- f (x)- Lv

converges to 0 faster than

. Thus the above definition is equivalent to saying

|f (x-+-v)--f (x)--Lv-| |lvim|\to0 |v| = 0 (5.12.15)

(5.12.15)

or equivalently,

lim |f (y)--f (x)--Df-(x)(y--x)|= 0. (5.12.16) y\tox |y- x|

(5.12.16)

Now it is clear this is just a generalization of the notion of the derivative of a function of one variable because in this more specialized situation,

lim |f-(x-+-v)--f (x)--f'(x)v|= 0, |v|\to0 |v|

due to the definition which says

f'(x) = lim f-(x+-v)--f (x). v\to0 v

For functions of n variables, you can’t define the derivative as the limit of a difference quotient like you can for a function of one variable because you can’t divide by a vector. That is why there is a need for a more general definition.

The term o

is notation that is descriptive of the behavior in 5.12.14 and it is only this behavior that is of interest. Thus, if t and k are constants,

o(v) = o(v)+ o (v ),o (tv) = o(v),ko(v) = o(v)

and other similar observations hold. The sloppiness built in to this notation is useful because it ignores details which are not important. It may help to think of o

as an adjective describing what is left over after approximating f

by f

+ Df

Theorem 5.12.2 The derivative is well defined.

Proof: First note that for a fixed vector, v, o

= o

. Now suppose both L₁ and L₂ work in the above definition. Then let v be any vector and let t be a real scalar which is chosen small enough that tv + x ∈ U. Then

f (x+ tv) = f (x )+ L1tv + o(tv),f (x+ tv) = f (x) +L2tv + o(tv ).

Therefore, subtracting these two yields

= o

. Therefore, dividing by t yields

. Now let t → 0 to conclude that

= 0. Since this is true for all v, it follows L₂ = L₁. This proves the theorem.

Lemma 5.12.3 Let f be differentiable at x. Then f is continuous at x and in fact, there exists K > 0 such that whenever

is small enough,

|f (x+ v )- f (x)| \leq K |v|

Proof: From the definition of the derivative, f

− f

= Df

v + o

. Let

be small enough that

< 1 so that

≤

. Then for such v,

|f (x+ v) - f (x )| \leq |Df (x) v|+ |v| \leq (|Df (x)|+ 1)|v |

This proves the lemma with K =

+ 1.

Theorem 5.12.4 (The chain rule) Let U and V be open sets, U ⊆ Fⁿ and V ⊆ F^m. Suppose f : U → V is differentiable at x ∈ U and suppose g : V → F^q is differentiable at f

∈ V . Then g ∘ f is differentiable at x and

D (g \circf)(x) = Dg (f (x))Df (x ).

Proof: This follows from a computation. Let B

⊆ U and let r also be small enough that for

≤ r, it follows that f

∈ V . Such an r exists because f is continuous at x. For

< r, the definition of differentiability of g and f implies

g (f (x + v))- g(f (x)) =

Dg (f (x))(f (x +v )- f (x))+ o(f (x + v)- f (x)) = Dg (f (x))[Df (x )v+ o(v)]+ o(f (x+ v) - f (x )) = Dg (f (x))Df (x)v + o(v)+ o(f (x+ v) - f (x )). (5.12.17)

It remains to show o

= o

By Lemma 5.12.3, with K given there, letting ε > 0, it follows that for

small enough,

|o(f (x+ v)- f (x))| \leq (ε∕K)|f (x+ v) - f (x )| \leq (ε∕K )K |v | = ε|v |.

Since ε > 0 is arbitrary, this shows o

= o

because whenever

is small enough,

|o(f (x + v)- f (x))| ------------------\leq ε. |v|

By 5.12.17, this shows

g (f (x+ v))- g (f (x)) = Dg (f (x))Df (x)v+ o (v )

which proves the theorem.

The derivative is a linear transformation. What is the matrix of this linear transformation taken with respect to the usual basis vectors? Let e_i denote the vector of Fⁿ which has a one in the i^th entry and zeroes elsewhere. Then the matrix of the linear transformation is the matrix whose i^th column is Df

e_i. What is this? Let t ∈ ℝ such that

is sufficiently small.

f (x + tei)- f (x) = Df (x)tei + o(tei) = Df (x)te + o(t) . i

Then dividing by t and taking a limit,

Df (x)ei = lim f (x+-tei)-- f-(x-)\equiv-\partialf (x ). t\to0 t \partialxi

Thus the matrix of Df

with respect to the usual basis vectors is the matrix of the form

( ) f1,x1 (x) f1,x2 (x) \cdot\cdot\cdot f1,xn (x) |( ... ... ... |) . f (x) f (x) \cdot\cdot\cdot f (x) m,x1 m,x2 m,xn

As mentioned before, there is no harm in referring to this matrix as Df

but it may also be referred to as Jf

This is summarized in the following theorem.

Theorem 5.12.5 Let f : Fⁿ → F^m and suppose f is differentiable at x. Then all the partial derivatives

exist and if Jf

is the matrix of the linear transformation with respect to the standard basis vectors, then the ij^th entry is given by f_i,j or

What if all the partial derivatives of f exist? Does it follow that f is differentiable? Consider the following function.

{ xy f (x,y) = x2+y2 if (x,y) ⁄= (0,0) . 0 if (x,y) = (0,0)

Then from the definition of partial derivatives,

lim f (h,0)--f-(0,0) = lim 0---0= 0 h\to0 h h\to0 h

and

f (0,h)- f (0,0) 0 - 0 lim -------------- = lim -----= 0 h\to0 h h\to0 h

However f is not even continuous at

which may be seen by considering the behavior of the function along the line y = x and along the line x = 0. By Lemma 5.12.3 this implies f is not differentiable. Therefore, it is necessary to consider the correct definition of the derivative given above if you want to get a notion which generalizes the concept of the derivative of a function of one variable in such a way as to preserve continuity whenever the function is differentiable.

class=”left” align=”middle”(Fⁿ, F^m)5.13. C¹ FUNCTIONS