Kenneth Kuttler

266 CHAPTER 10. MARKOV PROCESSES

Theorem 10.1.6 Suppose A is a Markov matrix in which aij > 0 for all i, j and supposew is a vector. Then for each i,

limk→∞

(Akw

)i= vi

where Av = v. In words, Akw always converges to a steady state. In addition to this, ifthe vector w satisfies wi ≥ 0 for all i and

∑i wi = c, then the vector v will also satisfy the

conditions, vi ≥ 0,∑

i vi = c.

Proof: By Lemma 10.1.4, since each aij > 0, the eigenvalues are either 1 or have absolutevalue less than 1. Therefore, the claimed limit exists by Theorem 10.1.3. The assertion thatthe components are nonnegative and sum to c follows from Lemma 10.1.5. That Av = vfollows from

v = limn→∞

Anw = limn→∞

An+1w = A limn→∞

Anw = Av. ■

It is not hard to generalize the conclusion of this theorem to regular Markov processes.

Corollary 10.1.7 Suppose A is a regular Markov matrix, one for which the entries of Ak

are all positive for some k, and suppose w is a vector. Then for each i,

limn→∞

(Anw)i = vi

where Av = v. In words, Anw always converges to a steady state. In addition to this, ifthe vector w satisfies wi ≥ 0 for all i and

∑i wi = c, Then the vector v will also satisfy the

conditions vi ≥ 0,∑

i vi = c.

Proof: Let the entries of Ak be all positive for some k. Now suppose that aij ≥ 0 forall i, j and A = (aij) is a Markov matrix. Then if B = (bij) is a Markov matrix with bij > 0for all ij, it follows that BA is a Markov matrix which has strictly positive entries. This isbecause the ijth entry of BA is ∑

bikakj > 0,

Thus, from Lemma 10.1.4, Ak has an eigenvalue equal to 1 for all k sufficiently large, andall the other eigenvalues have absolute value strictly less than 1. The same must be true ofA. If v ̸= 0 and Av = λv and |λ| = 1, then Akv = λkv and so, by Lemma 10.1.4, λm = 1if m ≥ k. Thus

1 = λk+1 = λkλ = λ

By Theorem 10.1.3, limn→∞Anw exists. The rest follows as in Theorem 10.1.6. ■

10.2 Migration Matrices

Definition 10.2.1 Let n locations be denoted by the numbers 1, 2, · · · , n. Also suppose it isthe case that each year aij denotes the proportion of residents in location j which move tolocation i. Also suppose no one escapes or emigrates from without these n locations. This lastassumption requires

∑i aij = 1. Thus (aij) is a Markov matrix referred to as a migration

matrix.

If v =(x1, · · · , xn)T where xi is the population of location i at a given instant, you obtainthe population of location i one year later by computing

∑j aijxj = (Av)i . Therefore, the

population of location i after k years is(Akv

)i. Furthermore, Corollary 10.1.7 can be used

to predict in the case where A is regular what the long time population will be for the givenlocations.

266 CHAPTER 10. MARKOV PROCESSESTheorem 10.1.6 Suppose A is a Markov matrix in which aj; > 0 for all i,j and supposew is a vector. Then for each i,lim (A‘w).k—-00 0where Av =v. In words, A*w always converges to a steady state. In addition to this, ifthe vector w satisfies w; > 0 for alli and )°, w; =, then the vector v will also satisfy theconditions, vj > 0, >); vi = ¢.Proof: By Lemma 10.1.4, since each a;; > 0, the eigenvalues are either 1 or have absolutevalue less than 1. Therefore, the claimed limit exists by Theorem 10.1.3. The assertion thatthe components are nonnegative and sum to c follows from Lemma 10.1.5. That Av = vfollows fromv= lim A®w= lim A”*+!w=A lim A"w = Av.noo noo nooIt is not hard to generalize the conclusion of this theorem to regular Markov processes.Corollary 10.1.7 Suppose A is a regular Markov matrix, one for which the entries of A®are all positive for some k, and suppose w is a vector. Then for each i,Jim (A"w), =u;where Av =v. In words, A”w always converges to a steady state. In addition to this, ifthe vector w satisfies w; > 0 for alli and \>, w; =c, Then the vector v will also satisfy theconditions v; > 0, 55; vi = ¢.Proof: Let the entries of A’ be all positive for some k. Now suppose that aij > O forall i,j and A = (a;;) is a Markov matrix. Then if B = (b;;) is a Markov matrix with b;; > 0for all ij, it follows that BA is a Markov matrix which has strictly positive entries. This isbecause the ij*” entry of BA isS- bik Gh; > 0,kThus, from Lemma 10.1.4, A* has an eigenvalue equal to 1 for all k sufficiently large, andall the other eigenvalues have absolute value strictly less than 1. The same must be true ofA. If v 40 and Av = dw and |\| = 1, then A*v = A*v and so, by Lemma 10.1.4, \ = 1ifm >k. Thus1=\Mtt= YK =XBy Theorem 10.1.3, limp 5... Aw exists. The rest follows as in Theorem 10.1.6.10.2 Migration MatricesDefinition 10.2.1 Let n locations be denoted by the numbers 1,2, --- ,n. Also suppose it isthe case that each year aj; denotes the proportion of residents in location j which move tolocation i. Also suppose no one escapes or emigrates from without these n locations. This lastassumption requires >), ai; = 1. Thus (a;j) is a Markov matrix referred to as a migrationmatrix.Ifv =(a,--- Zn) where 2; is the population of location 7 at a given instant, you obtainthe population of location i one year later by computing )> j ijtj = (Av), . Therefore, thepopulation of location i after k years is (A*v), . Furthermore, Corollary 10.1.7 can be usedto predict in the case where A is regular what the long time population will be for the givenlocations.