Kenneth Kuttler

14.3. THE OPERATOR NORM∗ 339

and so|∥x∥−∥y∥| ≤ ∥x−y∥

This section will involve some analysis. If you want to talk about norms, this is in-evitable. It will need some of the theorems of calculus which are usually neglected. Inparticular, it needs the following result which is a case of the Heine Borel theorem. To seethis proved, see any good calculus book, not most of the ones which are used in beginningcourses on calculus.

Theorem 14.3.1 Let S denote the points x ∈ Fn such that |x| = 1. Then if {xk}∞

k=1 is anysequence of points of S, there exists a subsequence which converges to a point of S.

Definition 14.3.2 Let A be an m× n matrix. Let ∥·∥k denote a norm on Ck. Then theoperator norm is defined as follows.

∥A∥ ≡max{∥Ax∥m : ∥x∥n ≤ 1}

Lemma 14.3.3 The operator norm is well defined and is in fact a norm on the vector spaceof m×n matrices.

Proof: It has already been observed that the m×n matrices form a vector space startingon Page 83. Why is ∥A∥< ∞?

claim: There exists c > 0 such that whenever ∥x∥ ≤ 1, it follows that |x| ≤ c.Proof of the claim: If not, then there exists {xk} such that ∥xk∥ ≤ 1 but |xk| > k for

k = 1,2, · · · . Then |xk/ |xk|| = 1 and so by the Heine Borel theorem from calculus, thereexists a further subsequence, still denoted by k such that∣∣∣∣ xk

|xk|−y∣∣∣∣→ 0, |y|= 1.

Lettingxk

|xk|=

∑i=1

aki ei, y =

∑i=1

aiei,

It follows that ak→ a in Fn. Hence∥∥∥∥ xk

|xk|−y∥∥∥∥≤ n

∑i=1

∣∣∣aki −ai

∣∣∣∥ei∥

which converges to 0. However, ∥∥∥∥ xk

|xk|

∥∥∥∥≤ 1k

and so, by continuity of ∥·∥ mentioned above,

∥y∥= limk→∞

∥∥∥∥ xk

|xk|

∥∥∥∥= 0

Therefore, y = 0 but also |y|= 1, a contradiction. This proves the claim.Now consider why ∥A∥< ∞. Let c be as just described in the claim.

sup{∥Ax∥m : ∥x∥n ≤ 1} ≤ sup{∥Ax∥m : |x| ≤ c}

14.3. THE OPERATOR NORM* 339and soIIlxll — llylll < IIx—yllThis section will involve some analysis. If you want to talk about norms, this is in-evitable. It will need some of the theorems of calculus which are usually neglected. Inparticular, it needs the following result which is a case of the Heine Borel theorem. To seethis proved, see any good calculus book, not most of the ones which are used in beginningcourses on calculus.Theorem 14.3.1 Let S denote the points x € F" such that |x| = 1. Then if {xx };_, is anysequence of points of S, there exists a subsequence which converges to a point of S.Definition 14.3.2 Let A be an m xn matrix. Let ||-|\, denote a norm on C*. Then theoperator norm is defined as follows.||A|| = max {||Ax||,, = [Ixlln <1Lemma 14.3.3 The operator norm is well defined and is in fact a norm on the vector spaceof m Xn matrices.Proof: It has already been observed that the m x n matrices form a vector space startingon Page 83. Why is ||A|| < 00?claim: There exists c > 0 such that whenever ||x|| < 1, it follows that |x| < c.Proof of the claim: If not, then there exists {x;} such that ||x,|| <1 but |x,| > k fork = 1,2,---. Then |x;/|x;|| = 1 and so by the Heine Borel theorem from calculus, thereexists a further subsequence, still denoted by k such thatXk-y +0, ly|=1.xx|LettingXj n k n— =Yaje, y=) ae,xe] i=lIt follows that a — a in F”. Hencexk [sdi=1[x;| fmaj —a;\ \\ei|which converges to 0. However,xe | 1xx| |] &and so, by continuity of ||-|| mentioned above,, Xki= Jn |Therefore, y = 0 but also |y| = 1, a contradiction. This proves the claim.Now consider why ||A|| < co. Let c be as just described in the claim.sup {||AX||, : {IXlln < 1} < sup {||Axll,, + Ix] Sc}