29.6. STRONG LAW OF LARGE NUMBERS 805

Lemma 29.6.2 Let {Xk} be a sequence of independent identically distributed randomvariables such that E (|Xk|)< ∞. Then letting Sn = ∑

nk=1 Xk, it follows that for k ≤ n

E (Xk|σ (Sn,Sn+1, · · ·)) = E (Xk|σ (Sn)) =Sn

n.

Proof: It was shown in Lemma 29.6.1 the first equality holds. It remains to show thesecond. Letting A = S−1

n (B) where B is Borel, it follows there exists B′ ⊆ Rn a Borel setsuch that

S−1n (B) = (X1, · · · ,Xn)

−1 (B′) .Then ∫

AE (Xk|σ (Sn))dP =

∫S−1

n (B)XkdP

=∫(X1,··· ,Xn)

−1(B′)XkdP =

∫(X1,··· ,Xn)

−1(B′)xkdλ (X1,··· ,Xn)

=∫· · ·∫

X(X1,··· ,Xn)

−1(B′) (x)xkdλ X1dλ X2 · · ·dλ Xn

=∫· · ·∫

X(X1,··· ,Xn)

−1(B′) (x)xldλ X1dλ X2 · · ·dλ Xn

=∫

AE (Xl |σ (Sn))dP

and so since A ∈ σ (Sn) is arbitrary,

E (Xl |σ (Sn)) = E (Xk|σ (Sn))

for each k, l ≤ n. Therefore,

Sn = E (Sn|σ (Sn)) =n

∑j=1

E (X j|σ (Sn)) = nE (Xk|σ (Sn)) a.e.

and so

E (Xk|σ (Sn)) =Sn

na.e.

as claimed. ■With this preparation, here is the strong law of large numbers for identically distributed

random variables.

Theorem 29.6.3 Let {Xk} be a sequence of independent identically distributed ran-dom variables such that E (|Xk|) < ∞ for all k. Since these are identicaly distributed,E (|Xk|) does not depend on k and so the process is bounded. Letting m = E (Xk) ,

limn→∞

1n

n

∑k=1

Xk (ω) = m a.e.

and convergence also takes place in L1 (Ω).