11.1. REGULAR MARKOV MATRICES 287

Therefore, the entries of (I +N)n cannot all be bounded. From block multiplication,

P−1AnP =

(I +N)n

(Jr2 (λ 2))n

. . .

(Jrm (λ m))n

and this is a contradiction because entries are bounded on the left and unbounded on theright.

Since N = 0, the above equation implies limn→∞ An exists and equals

P

I

0. . .

0

P−1 ■

Are there examples which will cause the eigenvalue condition of this theorem to hold?The following lemma gives such a condition. It turns out that if ai j > 0, not just ≥ 0, thenthe eigenvalue condition of the above theorem is valid.

Lemma 11.1.4 Suppose A = (ai j) is a stochastic matrix. Then λ = 1 is an eigenvalue. Ifai j > 0 for all i, j, then if µ is an eigenvalue of A, either |µ|< 1 or µ = 1.

Proof: First consider the claim that 1 is an eigenvalue. By definition,

∑i

1ai j = 1

and so ATv = v where v =(

1 · · · 1)T

. Since A,AT have the same eigenvalues, thisshows 1 is an eigenvalue of A. Suppose then that µ is an eigenvalue. Is |µ|< 1 or µ = 1?Let v be an eigenvector for AT and let |vi| be the largest of the

∣∣v j∣∣ .

µvi = ∑j

a jiv j

and now multiply both sides by µvi to obtain

|µ|2 |vi|2 = ∑j

a jiv jµvi = ∑j

a ji Re(v jµvi)

≤ ∑j

a ji |vi|2 |µ|= |µ| |vi|2

Therefore, |µ| ≤ 1. If |µ|= 1, then equality must hold in the above, and so v jviµ mustbe real and nonnegative for each j. In particular, this holds for j = i which shows µ is realand nonnegative. Thus, in this case, µ = 1 because µ̄ = µ is nonnegative and equal to 1.The only other case is where |µ|< 1. ■

The next lemma is sort of a conservation result. It says the sign and sum of entries of avector are preserved when multiplying by a Markov matrix.