26.8. STRONG LAW OF LARGE NUMBERS 733

≤ 2CKn

+ε

2n−K

n<

2CKn

+ε

2

Therefore, whenever n is large enough,∣∣s− 1

n ∑nk=1 sk

∣∣< ε.

Now consider the second claim. Let sk = ∑kj=1

a jj and s = limk→∞ sk Then by the first

part,

s = limn→∞

1n

n

∑k=1

sk = limn→∞

1n

n

∑k=1

k

∑j=1

a j

j

= limn→∞

1n

n

∑j=1

a j

j

n

∑k= j

1 = limn→∞

1n

n

∑j=1

a j

j(n− j)

= limn→∞

(n

∑j=1

a j

j− 1

n

n

∑j=1

a j

)= s− lim

n→∞

1n

n

∑j=1

a j ■

Now here is the strong law of large numbers.

Theorem 26.8.5 Suppose {Xk} are independent random variables, and also sup-pose that E (|Xk|)< ∞ for each k and E (Xk) =mk. Suppose also

∞

∑j=1

1j2 E

(∣∣X j−m j∣∣2)< ∞. (26.9)

Then limn→∞1n ∑

nj=1 (X j−m j) = 0.

Proof: Consider the sum ∑∞j=1

X j−m jj . This sum converges a.e. because of 26.9 and

Theorem 26.8.3 applied to the random vectors{

X j−m jj

}. Therefore, from Lemma 26.8.4

it follows that for a.e. ω, limn→∞1n ∑

nj=1 (X j (ω)−m j) = 0 ■

The next corollary is often called the strong law of large numbers. It follows immedi-ately from the above theorem.

Corollary 26.8.6 Suppose{X j}∞

j=1 are independent having mean m and variance

equal to σ2 ≡∫

Ω

∣∣X j−m∣∣2 dP < ∞. Then for a.e. ω ∈Ω,

limn→∞

1n

n

∑j=1X j (ω) =m