To begin with, the notion of a linear map is just a function which is linear. Such a function,
denoted by L, and mapping ℝ^{n} to ℝ^{m} is linear means
(∑m ) ∑m
L xivi = xiLvi
i=1 i=1
In other words, it distributes across additions and allows one to factor out scalars. Hopefully
this is familiar from linear algebra. If not, there is a short review of linear algebra in the
appendix.
Definition B.3.1We use the symbol ℒ
(ℝn,ℝm )
to denote the space of lineartransformations which map ℝ^{n}to ℝ^{m}. For L ∈ℒ
(ℝn, ℝm )
one can always consider it as anm × n matrix A as follows. Let
( )
A = Le1 Le2 ⋅⋅⋅ Len
where in the above Le_{i}is the i^{th}column. Define the sum and scalar multiple of lineartransformations in the natural manner. That is, for L,M linear transformations and α,βscalars,
(αL + βM )(x) ≡ αL (x)+ βM (x )
Observation B.3.2With the above definition of sums and scalar multiples of lineartransformations, theresult of such a linear combination of linear transformations is itselflinear. Indeed, forx,yvectors and a,b scalars,
Also, a linear combination of linear transformations corresponds to the linear combination ofthe corresponding matrices in which addition is defined in the usual manner as addition ofcorresponding entries. To see this, note that if A is the matrix of L and B the matrix ofM,
(αL + βM )ei ≡ (αA + βB )ei = αAei + βBei
by the usual rules of matrix multiplication. Thus the i^{th}column of
(αA + βB )
is thelinear combination of the i^{th}columns of A and B according to usual rules of matrixmultiplication.
Proposition B.3.3For L ∈ℒ
(ℝn,ℝm )
, the matrix defined above satisfies
n
Ax = Lx, x ∈ ℝ
and if any m × n matrix A does satisfy Ax = Lx, then A is given in the abovedefinition.
Proof:Ax = Lx for all x if and only if for x =∑_{i=1}^{n}x_{i}e_{i}
Definition B.3.4The norm of a linear transformation of A ∈ℒ
n m
(ℝ ,ℝ )
is definedas
||A || ≡ sup{||Ax ||m : ||x||n ≤ 1} < ∞.
ℝ ℝ
Then
||A||
is referred to as the operator norm of the linear transformation A.
It is an easy exercise to verify that
||⋅||
is a norm on ℒ
(ℝn, ℝm )
and it is always the case
that
||Ax ||ℝm ≤ ||A||||x||ℝn .
Furthermore, you should verify that you can replace ≤ 1 with = 1 in the definition.
Thus
||A || ≡ sup{||Ax||ℝm : ||x||ℝn = 1}.
It is necessary to verify that this norm is actually well defined.
Lemma B.3.5The operator norm is well defined. Let A ∈ℒ
(ℝn, ℝm )
.
Proof: We can use the matrix of the linear transformation with matrix multiplication
interchangeably with the linear transformation. This follows from the above considerations.
Suppose lim_{k→∞}v^{k} = v in ℝ^{n}. Does it follow that Av^{k}→ Av? This is indeed the case with
the usual Euclidean norm and therefore, it is also true with respect to any other norm by the
equivalence of norms (Theorem B.2.33). To see this,
( m )1∕2 ( m ||n ||2) 1∕2
||Avk − Av || ≡ ∑ ||(Avk ) − (Av )||2 ≤ | ∑ ||∑ A (vk− v )|| |
i=1 i i ( i=1 ||j=1 ij j j|| )
( ) ( )
m ( n )2 1∕2 m ( n ) 2 1∕2
≤ |(∑ (∑ |Aij|||vk− vj||) |) ≤ ||vk − v|||( ∑ (∑ |Aij|) |)
i=1 j=1 j i=1 j=1
Thus A is continuous. Then also v →
||Av ||
_{ℝm} is a continuous function by the triangle
inequality. Indeed,
|||Av ||ℝm − ||Au||ℝm | ≤ ||Av − Au||ℝm
Now let D be the closed ball of radius 1 in V . By Theorem B.2.32, this set D is compact and
so
max {||Av || : ||v|| ≤ 1} ≡ ||A|| < ∞.■
ℝm ℝn
Then we have the following theorem.
Theorem B.3.6Let ℝ^{n}and ℝ^{m}be finite dimensional normed linear spaces ofdimension n and m respectively and denote by
||⋅||
the norm on either ℝ^{n}or ℝ^{m}. Then if Ais any linear function mapping ℝ^{n}to ℝ^{m}, then A ∈ℒ
(ℝn,ℝm )
and
(ℒ (ℝn,ℝm ),||⋅||)
is acomplete normed linear space of dimension nm with
||Ax || ≤ ||A||||x||.
Also if A ∈ℒ
(ℝn,ℝm )
and B ∈ℒ
(ℝm, ℝp)
where ℝ^{n}, ℝ^{m}, ℝ^{p}are normed linearspaces,
∥BA∥ ≤ ∥B∥ ∥A∥
Proof:It is necessary to show the norm defined on linear transformations really is a
norm. Again the triangle inequality is the only property which is not obvious. It remains to
show this and verify
||A||
< ∞. This last follows from the above Lemma B.3.5. Thus the
norm is at least well defined. It remains to verify its properties.
Next consider the assertion about the dimension of ℒ
(ℝn,ℝm )
. This is fairly obvious
because a basis for the space of m × n matrices is clearly the matrices E_{ij} which has a 1 in
the ij^{th} position and a 0 everywhere else. By Theorem B.2.33
What does it mean to say that A^{k}→ A in terms of this operator norm? In words, this
happens if and only if the ij^{th} entry of A^{k} converges to the ij^{th} entry of A for each
ij.
Proposition B.3.7lim_{k→∞}
||||Ak − A||||
= 0 if and only if for every i,j
lim ||Ak − A || = 0
k→ ∞ ij ij
Proof:If A is an m×n matrix, then A_{ij} = e_{i}^{T}Ae_{j}. Suppose now that
|||| k ||||
A − A
→ 0.
Then in terms of the usual Euclidean norm and using the Cauchy Schwarz inequality,
| k | |T ( k ) |
|Aij − Aij| = |ei A − A ej| =
|( ( ) )| |( ) | || ||
| ei, Ak − A ej | ≤ |ei|| Ak − A ej| ≤ ||Ak − A|| (2.10)
(2.10)
If the operator norm is taken with respect to
||⋅||
, some other norm than the Euclidean norm,
then the right side of the above after ≤
||( k ) || ||||( k ) |||| |||| k ||||
A − A ej ≤ Δ A − A ej ≤ Δ A − A ||ej||
Thus convergence in operator norm implies pointwise convergence of the entries of A^{k} to the
corresponding entries of A.
Next suppose the entries of A^{k} converge to the corresponding entries of A. If
||v||
≤ 1, and
to save notation, let B^{k} = A^{k}− A. Then
||( k ) || ||( ∑ k ∑ k ∑ )T||
A − A v = | jB1jvj jB 2jvj ⋅⋅⋅ jBmjvj |