In this chapter, X and Y are finite dimensional vector spaces which have a norm. The following is
a definition.
Definition 13.0.1A linear space X is a normedlinear space if there is a norm defined on X,
||⋅||
satisfying
||x|| ≥ 0, ||x|| = 0 if and only if x = 0,
||x + y|| ≤ ||x ||+ ||y||,
||cx || = |c|||x ||
whenever c is a scalar. A set, U ⊆ X, a normed linear space is open if for every p ∈ U, there existsδ > 0 such that
B (p,δ) ≡ {x : ||x − p|| < δ} ⊆ U.
Thus, a set is open if every point of the set is an interior point. Also, lim_{n→∞}x_{n}= xmeans
lim_{n→∞}
∥xn − x∥
= 0. This is written sometimes as x_{n}→ x.
Note first that
∥x∥ = ∥x− y + y∥ ≤ ∥x− y∥ +∥y ∥
so
∥x ∥− ∥y∥ ≤ ∥x − y∥.
Similarly
∥y∥− ∥x∥ ≤ ∥x− y∥
and so
|∥x ∥− ∥y∥| ≤ ∥x− y ∥. (13.1)
(13.1)
To begin with recall the Cauchy Schwarz inequality which is stated here for convenience in
terms of the inner product space, ℂ^{n}.
Theorem 13.0.2The following inequality holds for a_{i}and b_{i}∈ ℂ.
| n | ( n )1 ∕2 ( n )1 ∕2
||∑ -|| ∑ 2 ∑ 2
|| aibi|| ≤ |ai| |bi| . (13.2)
i=1 i=1 i=1
(13.2)
Let X be a finite dimensional normed linear space with norm
||⋅||
where the field of
scalars is denoted by F and is understood to be either ℝ or ℂ. Let {v_{1},
⋅⋅⋅
,v_{n}} be a
basis for X. If x ∈ X, denote by x_{i} the i^{th} component of x with respect to this basis.
Thus
∑n
x = xivi.
i=1
Definition 13.0.3For x ∈ X and {v_{1},
⋅⋅⋅
,v_{n}} a basis, define a new norm by
(∑n )1∕2
|x | ≡ |xi|2 .
i=1
where
n
x = ∑ x v .
i=1 i i
Similarly, for y ∈ Y with basis {w_{1},
⋅⋅⋅
,w_{m}}, and y_{i}its components with respect to thisbasis,
(∑m )1∕2
|y| ≡ |yi|2
i=1
For A ∈ℒ
(X,Y )
, the space of linear mappings from X to Y,
||A|| ≡ sup{|Ax | : |x| ≤ 1}. (13.3)
(13.3)
The first thing to show is that the two norms,
||⋅||
and
|⋅|
, are equivalent. This means the
conclusion of the following theorem holds.
Theorem 13.0.4Let
(X,||⋅||)
be a finite dimensional normedlinear space and let
|⋅|
bedescribed above relative to a given basis,
{v1,⋅⋅⋅,vn}
. Then
|⋅|
is a norm and there existconstants δ,Δ > 0 independent of x such that
δ ||x||≤ |x|≤Δ ||x||. (13.4)
(13.4)
Proof:All of the above properties of a norm are obvious except the second, the triangle
inequality. To establish this inequality, use the Cauchy Schwarz inequality to write
Thus, letting x = ∑_{i=1}^{n}x_{i}v_{i}, it follows from the equivalence of the two norms shown above
that
lim ||xk − x|| = lim ∥∥xk − x∥∥ = 0.■
k→∞ k→ ∞
Corollary 13.0.8Suppose X is a finitedimensional linear space with the field of scalars eitherℂ or ℝ and
||⋅||
and
|||⋅|||
are two norms on X. Then there exist positive constants, δ and Δ,independent of x ∈ X such that
δ|||x||| ≤ ||x || ≤ Δ |||x|||.
Thus any two norms are equivalent.
This is very important because it shows that all questions of convergence can be considered
relative to any norm with the same outcome.
Proof:Let {v_{1},
⋅⋅⋅
,v_{n}} be a basis for X and let
|⋅|
be the norm taken with respect to this
basis which was described earlier. Then by Theorem 13.0.4, there are positive constants
δ_{1},Δ_{1},δ_{2},Δ_{2}, all independent of x ∈X such that
Definition 13.0.9Let X and Y be normed linear spaces with norms
||⋅||
_{X}and
||⋅||
_{Y }respectively. Then ℒ
(X, Y)
denotes the space of linear transformations, calledbounded lineartransformations, mapping X to Y which have the property that
||A|| ≡ sup{||Ax ||Y : ||x||X ≤ 1} < ∞.
Then
||A||
is referred to as the operator norm of the bounded linear transformationA.
It is an easy exercise to verify that
||⋅||
is a norm on ℒ
(X,Y )
and it is always the case
that
||Ax||Y ≤ ||A ||||x||X .
Furthermore, you should verify that you can replace ≤ 1 with = 1 in the definition.
Thus
||A|| ≡ sup{||Ax ||Y : ||x||X = 1}.
Theorem 13.0.10Let X and Y be finite dimensional normed linear spaces of dimension n andm respectively and denote by
||⋅||
the norm on either X or Y . Then if A is any linear functionmapping X to Y, then A ∈ℒ
(X, Y)
and
(ℒ(X,Y ),||⋅||)
is a complete normed linear space ofdimension nm with
||Ax || ≤ ||A ||||x||.
Also if A ∈ℒ
(X,Y )
and B ∈ℒ
(Y,Z)
where X,Y,Z are normed linear spaces,
∥BA ∥ ≤ ∥B∥∥A ∥
Proof:It is necessary to show the norm defined on linear transformations really is a norm.
Again the first and third properties listed above for norms are obvious. It remains to show the
second and verify
||A||
< ∞. Letting
{v1,⋅⋅⋅,vn}
be a basis and
|⋅|
defined with respect to this
basis as above, there exist constants δ,Δ > 0 such that
Note by Corollary 13.0.8 you can define a norm any way desired on any finite dimensional
linear space which has the field of scalars ℝ or ℂ and any other way of defining a norm on this
space yields an equivalent norm. Thus, it doesn’t much matter as far as notions of convergence are
concerned which norm is used for a finite dimensional space. In particular in the space of m × n
matrices, you can use the operator norm defined above, or some other way of giving this space a
norm. A popular choice for a norm is the Frobenius norm discussed earlier but reviewed
here.
Definition 13.0.11Make the space of m × n matrices into a inner product space bydefining
∗
(A,B ) ≡ trace(AB ).
Another way of describing a norm for an n × n matrix is as follows.
Definition 13.0.12Let A be an m×n matrix. Define the spectral normof A, written as
||A ||
_{2}to be
{ }
max λ1∕2 : λ is an eigenvalue of A ∗A .
That is, the largest singular value of A. (Note the eigenvalues of A^{∗}A are all positive because ifA^{∗}Ax = λx,then
λ |x |2 = λ (x,x) = (A∗Ax, x) = (Ax,Ax) ≥ 0.)
Actually, this is nothing new. It turns out that
||⋅||
_{2} is nothing more than the operator norm
for A taken with respect to the usual Euclidean norm,
( )1∕2
∑n 2
|x| = |xk | .
k=1
Proposition 13.0.13The following holds.
||A||2 = sup {|Ax | : |x| = 1} ≡ ||A ||.
Proof:Note that A^{∗}A is Hermitian and so by Corollary 12.3.4,
{ ∗ 1∕2 } { 1∕2 }
||A||2 = max (A Ax, x) : |x| = 1 = max (Ax,Ax ) : |x| = 1
= max {|Ax| : |x| = 1} = ||A ||.■
Here is another proof of this proposition. Recall there are unitary matrices of the right size
U,V such that A = U
( )
σ 0
0 0
V^{∗} where the matrix on the inside is as described in
the section on the singular value decomposition. Then since unitary matrices preserve
norms,
_{2} will mean either the operator norm of A taken with respect to the usual
Euclidean norm or the largest singular value of A, whichever is most convenient.
An interesting application of the notion of equivalent norms on ℝ^{n} is the process of giving a
norm on a finite Cartesian product of normed linear spaces.
Definition 13.0.14Let X_{i}, i = 1,
⋅⋅⋅
,n be normed linear spaces with norms,
||⋅||
_{i}.For
∏n
x ≡ (x1,⋅⋅⋅,xn) ∈ Xi
i=1
define θ : ∏_{i=1}^{n}X_{i}→ ℝ^{n}by
θ(x) ≡ (||x || ,⋅⋅⋅,||x ||)
11 n n
Then if
||⋅||
is any norm on ℝ^{n}, define a norm on∏_{i=1}^{n}X_{i}, also denoted by
||⋅||
by
||x|| ≡ ||θx||.
The following theorem follows immediately from Corollary 13.0.8.
Theorem 13.0.15Let X_{i}and
||⋅||
_{i}be given in the above definition and consider the normson∏_{i=1}^{n}X_{i}described there in terms of norms on ℝ^{n}. Then any two of these norms on∏_{i=1}^{n}X_{i}obtained in this way are equivalent.
For example, define
n
||x||≡ ∑ |x |,
1 i=1 i
||x||∞ ≡ max {|xi|,i = 1,⋅⋅⋅,n} ,
or
( n )1∕2
||x|| = ∑ |x |2
2 i=1 i
and all three are equivalent norms on ∏_{i=1}^{n}X_{i}.