be a linear transformation where X is a finite dimensional vector space and
consider the problem Ax = b where it is assumed there is a unique solution to this problem.
How does the solution change if A is changed a little bit and if b is changed a little bit? This
is clearly an interesting question because you often do not know A and b exactly. If a small
change in these quantities results in a large change in the solution, x, then it seems clear this
would be undesirable. In what follows
||⋅||
when applied to a linear transformation will always
refer to the operator norm. Recall the following property of the operator norm in Theorem
11.6.3.
Lemma 15.1.1Let A,B ∈ℒ
(X,X )
where X is a normed vector space as above. Then for
, controls how sensitive the relative change in the solution of
Ax = b is to small changes in A and b. This number is called the condition number. It is bad when this
number is large because a small relative change in b, for example could yield a large relative change in
x.
Recall that for A an n × n matrix,
||A||
2 = σ1 where σ1 is the largest singular value. The largest
singular value of A−1 is therefore, 1/σn where σn is the smallest singular value of A. Therefore, the
condition number is controlled by σ1∕σn, the ratio of the largest to the smallest singular value of A
provided the norm is the usual Euclidean norm.