26.8. STRONG LAW OF LARGE NUMBERS 731

Thus the Ak partition A and ω ∈ Ak means∣∣∣∑k

j=1X j

∣∣∣ ≥ ε but this did not happen for∣∣∣∑rj=1X j

∣∣∣ for any r < k. Note also that Ak ∈ σ (X1, · · · ,Xk) . Then from algebra,∣∣∣∣∣ n

∑j=1X j

∣∣∣∣∣2

=

(k

∑i=1X i +

n

∑j=k+1

X j,k

∑i=1X i +

n

∑j=k+1

X j

)

=

∣∣∣∣∣ k

∑j=1X j

∣∣∣∣∣2

+ ∑i≤k, j>k

(X i,X j)+ ∑i≤k, j>k

(X j,X i)+ ∑i>k, j>k

(X j,X i)

Written more succinctly,∣∣∣∑n

j=1X j

∣∣∣2 = ∣∣∣∑kj=1X j

∣∣∣2 +∑ j>k or i>k (X i,X j). Now multiplyboth sides by XAk and integrate. Suppose i ≤ k for one of the terms in the second sum.Then by Lemma 26.4.4 and Ak ∈ σ (X1, · · · ,Xk), the two random vectors XAkX i,X j areindependent, ∫

Ω

XAk (X i,X j)dP =

(∫Ω

XAkX idP,∫

Ω

X jdP)= 0

the last equality holding because by assumption E (X j) = 0. Therefore, it can be assumedboth i, j are larger than k and

∫Ω

XAk

∣∣∣∣∣ n

∑j=1X j

∣∣∣∣∣2

dP =∫

Ω

XAk

∣∣∣∣∣ k

∑j=1X j

∣∣∣∣∣2

dP+ ∑j>k,i>k

∫Ω

XAk (X i,X j)dP (26.8)

The last term on the right is interesting. Suppose i > j. The integral inside the sum is of theform

∫Ω

(X i,XAkX j

)dP. The second factor in the inner product is in

σ (X1, · · · ,Xk,X j)

and X i is not included in the list of random vectors. Thus by Lemma 26.4.4, the tworandom vectorsX i,XAkX j are independent and so the last term in 26.8 reduces to(∫

Ω

X idP,∫

Ω

XAkX jdP)=

(0,∫

Ω

XAkX jdP)= 0.

A similar result holds if j > i. Thus the mixed terms in the last term of 26.8 are all equal to0. Hence 26.8 reduces to∫

Ω

XAk

∣∣∣∣∣ n

∑j=1X j

∣∣∣∣∣2

dP =∫

Ω

XAk

∣∣∣∣∣ k

∑j=1X j

∣∣∣∣∣2

dP+∑i>k

∫Ω

XAk |X i|2 dP

and so∫

ΩXAk

∣∣∣∑nj=1X j

∣∣∣2 dP ≥∫

ΩXAk

∣∣∣∑kj=1X j

∣∣∣2 dP ≥ ε2P(Ak) .Now, summing theseyields

ε2P(A)≤

∫Ω

XA

∣∣∣∣∣ n

∑j=1X j

∣∣∣∣∣2

dP≤∫

Ω

∣∣∣∣∣ n

∑j=1X j

∣∣∣∣∣2

dP = ∑i, j

∫Ω

(X i,X j)dP

By independence of the random vectors the mixed terms of the above sum equal zero andso it reduces to ∑

ni=1∫

Ω|X i|2 dP ■

This theorem implies the following amazing result.