To begin with, the notion of a linear map is just a function which is linear. Such a function, denoted by L,
and mapping ℝ^{n} to ℝ^{m} is linear means
( )
∑m ∑m
L xivi = xiLvi
i=1 i=1
In other words, it distributes across additions and allows one to factor out scalars. Hopefully this is familiar
from linear algebra. If not, there is a short review of linear algebra in the appendix.
Definition 2.6.1We use the symbol ℒ
(ℝn,ℝm )
to denote the space of linear transformations, also calledlinear operators, which map ℝ^{n}to ℝ^{m}. For L ∈ℒ
(ℝn,ℝm )
one can always consider it as an m×n matrixA as follows. Let
( )
A = Le1 Le2 ⋅⋅⋅ Len
where in the above Le_{i}is the i^{th}column. Define the sum and scalar multipleof linear transformations inthe natural manner. That is, for L,M linear transformations and α,β scalars,
(αL + βM )(x) ≡ αL(x)+ βM (x)
Observation 2.6.2With the above definition of sums and scalar multiples of linear transformations, theresult of such a linear combination of linear transformations is itself linear. Indeed, forx,yvectors and a,bscalars,
Also, a linear combination of linear transformations corresponds to the linear combinationof the corresponding matrices in which addition is defined in the usual manner as additionof corresponding entries. To see this, note that if A is the matrix of L and B the matrix ofM,
(αL+ βM )ei ≡ (αA + βB)ei = αAei + βBei
by the usual rules of matrix multiplication. Thus the i^{th}column of
(αA + βB )
is the linear combination ofthe i^{th}columns of A and B according to usual rules of matrix multiplication.
Proposition 2.6.3For L ∈ℒ
(ℝn,ℝm )
, the matrix defined above satisfies
n
Ax = Lx, x ∈ ℝ
and if any m × n matrix A does satisfy Ax = Lx, then A is given in the above definition.
Proof:Ax = Lx for all x if and only if for x =∑_{i=1}^{n}x_{i}e_{i}
Definition 2.6.4The norm of a linear transformation of A ∈ℒ
(ℝn,ℝm )
is defined as
∥A∥ ≡ sup {∥Ax∥ℝm : ∥x∥ℝn ≤ 1} < ∞.
Then
∥A ∥
is referred to as the operator norm of the linear transformation A.
It is an easy exercise to verify that
∥⋅∥
is a norm on ℒ
n m
(ℝ ,ℝ )
and it is always the case
that
∥Ax∥ m ≤ ∥A∥ ∥x∥ n .
ℝ ℝ
Furthermore, you should verify that you can replace ≤ 1 with = 1 in the definition. Thus
∥A∥ ≡ sup {∥Ax∥ : ∥x∥ = 1}.
ℝm ℝn
It is necessary to verify that this norm is actually well defined.
Lemma 2.6.5The operator normis well defined. Let A ∈ℒ
(ℝn,ℝm )
.
Proof: We can use the matrix of the linear transformation with matrix multiplication interchangeably
with the linear transformation. This follows from the above considerations. Suppose lim_{k→∞}v^{k} = v in ℝ^{n}.
Does it follow that Av^{k}→ Av? This is indeed the case with the usual Euclidean norm and therefore, it is
also true with respect to any other norm by the equivalence of norms (Theorem 2.5.4). To see
this,
( )1∕2 ( || ||2) 1∕2
|| k || m∑ ||( k) ||2 | ∑m ||∑n ( k )|||
Av − Av ≡ Av i − (Av )i ≤ ( || Aij vj − vj ||)
i=1 i=1 j=1
_{ℝm} is a continuous function by the triangle inequality.
Indeed,
∥|Av∥ℝm − ∥Au∥ℝm | ≤ ∥Av − Au∥ℝm
Now let D be the closed ball of radius 1 in V . By Theorem 2.5.3, this set D is compact and
so
max{∥Av ∥ℝm : ∥v∥ℝn ≤ 1} ≡ ∥A∥ < ∞.■
Then we have the following theorem.
Theorem 2.6.6Let ℝ^{n}and ℝ^{m}be finite dimensional normed linear spaces of dimension n and mrespectively and denote by
∥⋅∥
the norm on either ℝ^{n}or ℝ^{m}. Then if A is any linear function mapping ℝ^{n}to ℝ^{m}, then A ∈ℒ
(ℝn,ℝm )
and
(ℒ (ℝn, ℝm ),∥⋅∥)
is a complete normed linear space of dimension nmwith
∥Ax∥ ≤ ∥A∥ ∥x ∥.
Also if A ∈ℒ
(ℝn,ℝm )
and B ∈ℒ
(ℝm, ℝp)
where ℝ^{n}, ℝ^{m}, ℝ^{p}are normed linear spaces,
∥BA ∥ ≤ ∥B ∥∥A∥
Proof:It is necessary to show the norm defined on linear transformations really is a norm. Again the
triangle inequality is the only property which is not obvious. It remains to show this and verify
∥A ∥
< ∞.
This last follows from the above Lemma 2.6.5. Thus the norm is at least well defined. It remains to verify
its properties.
Next consider the assertion about the dimension of ℒ
(ℝn,ℝm )
. This is fairly obvious because a basis
for the space of m × n matrices is clearly the matrices E_{ij} which has a 1 in the ij^{th} position and a 0
everywhere else. By Theorem 2.5.4
What does it mean to say that A^{k}→ A in terms of this operator norm? In words, this happens if and
only if the ij^{th} entry of A^{k} converges to the ij^{th} entry of A for each ij.
Proposition 2.6.7lim_{k→∞}
∥ k ∥
∥A − A∥
= 0 if and only if for every i,j
|| k ||
lkim→∞ Aij − Aij = 0
Proof:If A is an m×n matrix, then A_{ij} = e_{i}^{T}Ae_{j}. Suppose now that
∥ k ∥
∥A − A ∥
→ 0. Then in terms
of the usual Euclidean norm and using the Cauchy Schwarz inequality,
|| k || ||T ( k ) ||
Aij − Aij = ei A − A ej =
|( ( ) )| |( ) | ∥ ∥
| ei,Ak − A ej| ≤ |ei|| Ak − A ej| ≤ ∥Ak − A∥ (2.17)
(2.17)
If the operator norm is taken with respect to
∥⋅∥
, some other norm than the Euclidean norm, then the
right side of the above after ≤
|( k ) | ∥( k ) ∥ ∥ k ∥
|A − A ej| ≤ Δ ∥ A − A ej∥ ≤ Δ ∥A − A∥∥ej∥
Thus convergence in operator norm implies pointwise convergence of the entries of A^{k} to the corresponding
entries of A.
Next suppose the entries of A^{k} converge to the corresponding entries of A. If
∥v∥
≤ 1, and to save
notation, let B^{k} = A^{k}− A. Then
| |
||(Ak − A) v|| = ||( ∑ Bk v ∑ Bk v ⋅⋅⋅ ∑ B v )T||
| j 1j j j 2j j j mj j |