730 CHAPTER 26. INDEPENDENCE

26.8 Strong Law of Large NumbersKolmogorov’s inequality is a very interesting inequality which depends on independenceof a set of random vectors. The random vectors have values in Rn or more generally somereal separable Hilbert space.

Lemma 26.8.1 If Y,X are independent random variables having values in a real sep-arable Hilbert space, H with E

(|X|2

),E(|Y |2

)< ∞, then

∫Ω

(X,Y )dP =

(∫Ω

XdP,∫

Ω

Y dP).

Proof: Let {ek} be a complete orthonormal basis. Thus from Theorem 22.4.2,∫Ω

(X,Y )dP =∫

Ω

∞

∑k=1

(X,ek)(Y,ek)dP

Now

∫Ω

∞

∑k=1|(X,ek)(Y,ek)|dP≤

∫Ω

(∑k|(X,ek)|

2

)1/2(∑k|(Y,ek)|

2

)1/2

dP

=∫

Ω

|X| |Y |dP≤(∫

Ω

|X|2 dP)1/2(∫

Ω

|Y |2 dP)1/2

< ∞

and so by Fubini’s theorem and independence ofX,Y ,∫Ω

(X,Y )dP =∫

Ω

∞

∑k=1

(X,ek)(Y,ek)dP =∞

∑k=1

∫Ω

(X,ek)(Y,ek)dP

=∞

∑k=1

∫Ω

(X,ek)dP∫

Ω

(Y,ek)dP =∞

∑k=1

(∫Ω

XdP,ek

)(∫Ω

Y dP,ek

)dP

=

(∫Ω

XdP,∫

Ω

Y dP)

■

Now here is Kolmogorov’s inequality.

Theorem 26.8.2 Suppose {Xk}nk=1 are independent with E (|Xk|)< ∞, E (Xk) =

0. Then for any ε > 0,

P

([max

1≤k≤n

∣∣∣∣∣ k

∑j=1X j

∣∣∣∣∣≥ ε

])≤ 1

ε2

n

∑j=1

E(|Xk|2

).

Proof: Let A=[max1≤k≤n

∣∣∣∑kj=1X j

∣∣∣≥ ε

]. Now let A1≡ [|X1| ≥ ε] and if A1, · · · ,Am

have been chosen,

Am+1 ≡

[∣∣∣∣∣m+1

∑j=1X j

∣∣∣∣∣≥ ε

]∩

m⋂r=1

[∣∣∣∣∣ r

∑j=1X j

∣∣∣∣∣< ε

]