First is some notation which may be useful since it will be used in the following presentation.
Definition 14.11.1Let X,Y be inner product space and let u ∈ Y,v ∈ X. Then define u⊗v ∈ℒ
(X,Y )
asfollows.
u⊗ v (w ) ≡ (w,v)u
where
(w,v)
is the inner product in X. Then this is clearly linear. That it is continuous follows right awayfrom
|(w,v)u| ≤ |u|Y |w |X |v|X
and so
sup |u ⊗ v(w)| ≤ |u| |v|
|w|X ≤1 Y Y X
Sometimes this is called the tensor product, although much more can be said about the tensorproduct.
Note how this is similar to the rank one transformations used to consider the dimension of the space
ℒ
(V,W )
in Theorem 5.1.4. This is also a rank one transformation but here there is no restriction on the
dimension of the vector spaces although, as usual, the interest is in finite dimensional spaces. In case you
have
{v1,⋅⋅⋅,vn}
an orthonormal basis for V and
{u1,⋅⋅⋅,um}
an orthonormal basis for Y, (or even just a
basis.) the linear transformations u_{i}⊗v_{j} are the same as those rank one transformations used before in the
above theorem and are a basis for ℒ
(V,W )
. Thus for A = ∑_{i,j}a_{ij}u_{i}⊗ v_{j}, the matrix of A
with respect to the two bases has its ij^{th} entry equal to a_{ij}. This is stated as the following
proposition.
Proposition 14.11.2Suppose
{v1,⋅⋅⋅,vn}
is an orthonormal basis for V and
{u1,⋅⋅⋅,um}
is abasis for W. Then if A ∈ℒ
(V,W )
is given by A = ∑_{i,j}a_{ij}u_{i}⊗v_{j}, then the matrix of A with respectto these two bases is an m × n matrix whose ij^{th}entry is a_{ij}.
In case A is a Hermitian matrix, and you have an orthonormal basis of eigenvectors and U is the
unitary matrix having these eigenvectors as columns, recall that the matrix of A with respect to this basis
is diagonal. Recall why this is.
( ) ( )
Au1 ⋅⋅⋅ Aun = u1 ⋅⋅⋅ un D
where D is the diagonal matrix having the eigenvalues down the diagonal. Thus D = U^{∗}AU and
Au_{i} = λ_{i}u_{i}. It follows that as a linear transformation,
∑
A = λiui ⊗ ui
i
because both give the same answer when acting on elements of the orthonormal basis. This also says that
the matrix of A with respect to the given orthonormal basis is just the diagonal matrix having the
eigenvalues down the main diagonal.
The following theorem is about the eigenvectors and eigenvalues of a self adjoint operator. Such
operators may also be called Hermitian as in the case of matrices. The proof given generalizes to the
situation of a compact self adjoint operator on a Hilbert space and leads to many very useful results. It is
also a very elementary proof because it does not use the fundamental theorem of algebra and it
contains a way, very important in applications, of finding the eigenvalues. This proof depends
more directly on the methods of analysis than the preceding material. Recall the following
notation.
Definition 14.11.3Let X be an inner product space and let S ⊆ X. Then
S ⊥ ≡ {x ∈ X : (x,s) = 0 for all s ∈ S}.
Note that even if S is not a subspace, S^{⊥} is.
Theorem 14.11.4Let A ∈ℒ
(X,X )
be self adjoint (Hermitian)where X is a finite dimensionalinner product space of dimension n. Thus A = A^{∗}. Then there exists an orthonormal basis ofeigenvectors,
{vj}
_{j=1}^{n}.
Proof: Consider
(Ax,x)
. This quantity is always a real number because
------ ∗
(Ax,x) = (x,Ax) = (x,A x) = (Ax,x)
thanks to the assumption that A is self adjoint. Now define
λ1 ≡ inf{(Ax,x) : |x| = 1,x ∈ X1 ≡ X} .
Claim:λ_{1} is finite and there exists v_{1}∈ X with
|v1|
= 1 such that
(Av1,v1)
= λ_{1}.
Proof of claim:The set of vectors
{x : |x| = 1}
is a closed and bounded subset of the
finite dimensional space X. Therefore, it is compact and so the vector v_{1} exists by Theorem
11.5.3.
I claim that λ_{1} is an eigenvalue and v_{1} is an eigenvector. Letting w ∈ X_{1}≡ X, the function of the real
variable, t, given by
(A (v + tw),v + tw) (Av ,v) + 2tRe (Av ,w) +t2(Aw, w)
f (t) ≡ ----1-------12------= ---1--12---------1-----2---2----
|v1 + tw| |v1| + 2tRe (v1,w) + t|w|
achieves its minimum when t = 0. Therefore, the derivative of this function evaluated at t = 0 must equal
zero. Using the quotient rule, this implies, since
= 0 for all w ∈ X. This implies Av_{1} = λ_{1}v_{1} by Proposition 13.1.5.
Continuing with the proof of the theorem, let X_{2}≡
{v1}
^{⊥}. This is a closed subspace of X and
A : X_{2}→ X_{2} because for x ∈ X_{2},
(Ax,v1) = (x,Av1) = λ1 (x,v1) = 0.
Let
λ2 ≡ inf{(Ax, x) : |x| = 1,x ∈ X2}
As before, there exists v_{2}∈ X_{2} such that Av_{2} = λ_{2}v_{2}, λ_{1}≤ λ_{2}. Now let X_{3}≡
{v,v }
1 2
^{⊥} and continue in
this way. As long as k < n, it will be the case that
{v ,⋅⋅⋅,v }
1 k
^{⊥}≠
{0}
. This is because for k < n these
vectors cannot be a spanning set and so there exists some w
∕∈
span
(v ,⋅⋅⋅,v )
1 k
. Then letting z be the
closest point to w from span
(v,⋅⋅⋅,v)
1 k
, it follows that w − z ∈
{v ,⋅⋅⋅,v }
1 k
^{⊥}. Thus there is an
decreasing sequence of eigenvalues
{λ }
k
_{k=1}^{n} and a corresponding sequence of eigenvectors,
{v ,⋅⋅⋅,v }
1 n
with this being an orthonormal set. ■
Contained in the proof of this theorem is the following important corollary.
Corollary 14.11.5Let A ∈ℒ
(X, X )
be self adjoint where X is a finite dimensional inner product space.Then all the eigenvalues are real and for λ_{1}≤ λ_{2}≤
⋅⋅⋅
≤ λ_{n}the eigenvalues of A, there exists anorthonormal set of vectors
{u1,⋅⋅⋅,un }
for which
Auk = λkuk.
Furthermore,
λk ≡ inf{(Ax, x) : |x| = 1,x ∈ Xk }
where
⊥
Xk ≡ {u1,⋅⋅⋅,uk−1} ,X1 ≡ X.
Corollary 14.11.6Let A ∈ℒ
(X, X )
be self adjoint (Hermitian)where X is a finite dimensional innerproduct space. Then the largest eigenvalue of A is given by
max {(Ax,x ) : |x | = 1} (14.20)
(14.20)
and the minimum eigenvalue of A is given by
min {(Ax,x) : |x| = 1}. (14.21)
(14.21)
Proof:The proof of this is just like the proof of Theorem 14.11.4. Simply replace inf with sup and
obtain a decreasing list of eigenvalues. This establishes 14.20. The claim 14.21 follows from Theorem
14.11.4. ■
Another important observation is found in the following corollary.
Corollary 14.11.7Let A ∈ ℒ
(X,X )
where A is self adjoint. Then A = ∑_{i}λ_{i}v_{i}⊗ v_{i}whereAv_{i} = λ_{i}v_{i}and
{vi}
_{i=1}^{n}is an orthonormal basis.
Proof :If v_{k} is one of the orthonormal basis vectors, Av_{k} = λ_{k}v_{k}. Also,
∑ ∑ ∑
λivi ⊗ vi(vk) = λivi(vk,vi) = λiδikvi = λkvk.
i i i
Since the two linear transformations agree on a basis, it follows they must coincide. ■
By Proposition 14.11.2 this says the matrix of A with respect to this basis
{vi}
_{i=1}^{n} is the diagonal
matrix having the eigenvalues λ_{1},
⋅⋅⋅
,λ_{n} down the main diagonal.
The result of Courant and Fischer which follows resembles Corollary 14.11.5 but is more useful because
it does not depend on a knowledge of the eigenvectors.
Theorem 14.11.8Let A ∈ℒ
(X,X )
be self adjoint where X is a finite dimensional inner product space.Then for λ_{1}≤ λ_{2}≤
⋅⋅⋅
≤ λ_{n}the eigenvalues of A, there exist orthonormal vectors