59.7. KOLMOGOROV’S INEQUALITY 1881

Now consider the second claim. Let

sk =k

∑j=1

a j

j

and s = limk→∞ sk Then by the first part,

s = limn→∞

1n

n

∑k=1

sk = limn→∞

1n

n

∑k=1

k

∑j=1

a j

j

= limn→∞

1n

n

∑j=1

a j

j

n

∑k= j

1 = limn→∞

1n

n

∑j=1

a j

j(n− j)

= limn→∞

(n

∑j=1

a j

j− 1

n

n

∑j=1

a j

)= s− lim

n→∞

1n

n

∑j=1

a j

Now here is the strong law of large numbers.

Theorem 59.7.5 Suppose {Xk} are independent random variables and E (|Xk|) < ∞ foreach k and E (Xk) = mk. Suppose also

∞

∑j=1

1j2 E

(∣∣X j−m j∣∣2)< ∞. (59.7.12)

Then

limn→∞

1n

n

∑j=1

(X j−m j) = 0

Proof: Consider the sum∞

∑j=1

X j−m j

j.

This sum converges a.e. because of 59.7.12 and Theorem 59.7.3 applied to the randomvectors

{X j−m j

j

}. Therefore, from Lemma 59.7.4 it follows that for a.e. ω,

limn→∞

1n

n

∑j=1

(X j (ω)−m j) = 0

The next corollary is often called the strong law of large numbers. It follows immedi-ately from the above theorem.

Corollary 59.7.6 Suppose{

X j}∞

j=1 are independent having mean m and variance equalto

σ2 ≡

∫Ω

∣∣X j−m∣∣2 dP < ∞.

Then for a.e. ω ∈Ω

limn→∞

1n

n

∑j=1

X j (ω) = m