10.3. ABSORBING STATES 267

As an example of the above, consider the case where n = 3 and the migration matrix isof the form  .6 0 .1

.2 .8 0

.2 .2 .9

 .

Now  .6 0 .1

.2 .8 0

.2 .2 .9

2

=

 . 38 .0 2 . 15

. 28 . 64 .0 2

. 34 . 34 . 83

and so the Markov matrix is regular. Therefore,

(Akv

)iwill converge to the ith component

of a steady state. It follows the steady state can be obtained from solving the system

. 6x+ . 1z = x

. 2x+ . 8y = y

. 2x+ . 2y + . 9z = z

along with the stipulation that the sum of x, y, and z must equal the constant value presentat the beginning of the process. The solution to this system is

{y = x, z = 4x, x = x} .

If the total population at the beginning is 150,000, then you solve the following system

y = x, z = 4x, x+ y + z = 150000

whose solution is easily seen to be {x = 25 000, z = 100 000, y = 25 000} . Thus, after a longtime there would be about four times as many people in the third location as in either ofthe other two.

10.3 Absorbing States

There is a different kind of Markov process containing so called absorbing states which resultin transition matrices which are not regular. However, Theorem 10.1.3 may still apply. Onesuch example is the Gambler’s ruin problem. There is a total amount of money denoted byb. The Gambler starts with an amount j > 0 and gambles till he either loses everything orgains everything. He does this by playing a game in which he wins with probability p andloses with probability q. When he wins, the amount of money he has increases by 1 andwhen he loses, the amount of money he has decreases by 1. Thus the states are the integersfrom 0 to b. Let pij denote the probability that the gambler has i at the end of a gamegiven that he had j at the beginning. Let pnij denote the probability that the gambler has iafter n games given that he had j initially. Thus

pn+1ij =

∑k

pikpnkj ,