2.6 Norms On Linear Maps
To begin with, the notion of a linear map is just a function which is linear. Such a function, denoted by L,
and mapping ℝn to ℝm is linear means
In other words, it distributes across additions and allows one to factor out scalars. Hopefully this is familiar
from linear algebra. If not, there is a short review of linear algebra in the appendix.
Definition 2.6.1 We use the symbol ℒ
to denote the space of linear transformations, also called
linear operators, which map ℝn to ℝm. For L ∈ℒ
one can always consider it as an m×n matrix
A as follows. Let
where in the above Lei is the ith column. Define the sum and scalar multiple of linear transformations in
the natural manner. That is, for L,M linear transformations and α,β scalars,
Observation 2.6.2 With the above definition of sums and scalar multiples of linear transformations, the
result of such a linear combination of linear transformations is itself linear. Indeed, for x,y vectors and a,b
Also, a linear combination of linear transformations corresponds to the linear combination
of the corresponding matrices in which addition is defined in the usual manner as addition
of corresponding entries. To see this, note that if A is the matrix of L and B the matrix of
by the usual rules of matrix multiplication. Thus the ith column of
is the linear combination of
the ith columns of A and B according to usual rules of matrix multiplication.
Proposition 2.6.3 For L ∈ℒ
, the matrix defined above satisfies
and if any m × n matrix A does satisfy Ax = Lx, then A is given in the above definition.
Proof: Ax = Lx for all x if and only if for x =∑
if and only if for every x ∈ ℝn,
which happens if and only if A =
Definition 2.6.4 The norm of a linear transformation of A ∈ℒ
is defined as
is referred to as the operator norm of the linear transformation A.
It is an easy exercise to verify that
is a norm on
and it is always the case
Furthermore, you should verify that you can replace ≤ 1 with = 1 in the definition. Thus
It is necessary to verify that this norm is actually well defined.
Lemma 2.6.5 The operator norm is well defined. Let A ∈ℒ
Proof: We can use the matrix of the linear transformation with matrix multiplication interchangeably
with the linear transformation. This follows from the above considerations. Suppose limk→∞vk = v in ℝn.
Does it follow that Avk → Av? This is indeed the case with the usual Euclidean norm and therefore, it is
also true with respect to any other norm by the equivalence of norms (Theorem 2.5.4). To see
Thus A is continuous. Then also v →
is a continuous function by the triangle inequality.
Now let D be the closed ball of radius 1 in V . By Theorem 2.5.3, this set D is compact and
Then we have the following theorem.
Theorem 2.6.6 Let ℝn and ℝm be finite dimensional normed linear spaces of dimension n and m
respectively and denote by
the norm on either ℝn or ℝm. Then if A is any linear function mapping ℝn
to ℝm, then A ∈ℒ
is a complete normed linear space of dimension nm
Also if A ∈ℒ
and B ∈ℒ
where ℝn, ℝm, ℝp are normed linear spaces,
Proof: It is necessary to show the norm defined on linear transformations really is a norm. Again the
triangle inequality is the only property which is not obvious. It remains to show this and verify
This last follows from the above Lemma 2.6.5
. Thus the norm is at least well defined. It remains to verify
Next consider the assertion about the dimension of ℒ
This is fairly obvious because a basis
for the space of m × n
matrices is clearly the matrices Eij
which has a 1 in the ijth
position and a 0
everywhere else. By Theorem 2.5.4
is complete. If
Consider the last claim.
What does it mean to say that Ak → A in terms of this operator norm? In words, this happens if and
only if the ijth entry of Ak converges to the ijth entry of A for each ij.
Proposition 2.6.7 limk→∞
if and only if for every i,j
Proof: If A is an m×n matrix, then Aij = eiTAej. Suppose now that
0. Then in terms
of the usual Euclidean norm and using the Cauchy Schwarz inequality,
If the operator norm is taken with respect to
some other norm than the Euclidean norm, then the
right side of the above after ≤
Thus convergence in operator norm implies pointwise convergence of the entries of Ak to the corresponding
entries of A.
Next suppose the entries of Ak converge to the corresponding entries of A. If
and to save
notation, let Bk
= Ak − A
By equivalence of norms,
and so if
the quantity on the right converging to 0. ■