270 CHAPTER 10. MARKOV PROCESSES

It follows that

C2 =pb

pb − qb, C1 =

qb

qb − pb

Thus Pj =

qb

qb − pb+

pb

pb − qb

(q

p

)j

=qb

qb − pb− pb−jqj

qb − pb=qj(qb−j − pb−j

)qb − pb

To find the solution in the case of a fair game, one could take the limp→1/2 of the abovesolution. Taking this limit, you get

Pj =b− j

b.

You could also verify directly in the case where p = q = 1/2 in 10.3 and 10.4 that Pj = 1and Pj = j are two solutions to the difference equation and proceeding as before.

10.4 Exercises

1. Suppose the migration matrix for three locations is .5 0 .3

.3 .8 0

.2 .2 .7

 .

Find a comparison for the populations in the three locations after a long time.

2. Show that if∑

i aij = 1, then if A = (aij) , then the sum of the entries of Av equalsthe sum of the entries of v. Thus it does not matter whether aij ≥ 0 for this to be so.

3. If A satisfies the conditions of the above problem, can it be concluded that limn→∞An

exists?

4. Give an example of a non regular Markov matrix which has an eigenvalue equal to−1.

5. Show that when a Markov matrix is non defective, all of the above theory can be provedvery easily. In particular, prove the theorem about the existence of limn→∞An if theeigenvalues are either 1 or have absolute value less than 1.

6. Find a formula for An where

A =

52 − 1

2 0 −1

5 0 0 −472 − 1

212 − 5

272 − 1

2 0 −2

Does limn→∞An exist? Note that all the rows sum to 1. Hint: This matrix is similarto a diagonal matrix. The eigenvalues are 1,−1, 12 ,

12 .

7. Find a formula for An where

A =

2 − 1

212 −1

4 0 1 −452 − 1

2 1 −2

3 − 12

12 −2

