The concept of independence is probably the main idea which separates probability
from analysis and causes some of us to struggle to understand what is going
on.
Definition 58.3.1Let
(Ω,ℱ,P )
be a probability space. The sets in ℱ are called events.A set of events,
{Ai}
_{i∈I}is called independentif whenever
{Aik}
_{k=1}^{m}is a finitesubset
m
P (∩m Ai ) = ∏ P (Ai ).
k=1 k k=1 k
Each of these events defines a rather simple σ algebra,
( )
Ai,ACi ,∅,Ω
denoted by ℱ_{i}.
Now the following lemma is interesting because it motivates a more general notion of
independent σ algebras.
Lemma 58.3.2Suppose B_{i}∈ℱ_{i}for i ∈ I. Then for any m ∈ ℕ
m m∏
P (∩ k=1Bik) = P (Bik).
k=1
Proof: The proof is by induction on the number l of the B_{ik} which are not equal to
A_{ik}. First suppose l = 0. Then the above assertion is true by assumption. Suppose it is so
for some l and there are l + 1 sets not equal to A_{ik}. If any equals ∅ there is
nothing to show. Both sides equal 0. If any equals Ω, there is also nothing to
show. You can ignore that set in both sides and then you have by induction the
two sides are equal because you have no more than l sets different than A_{ik}.
The only remaining case is where some B_{ik} = A_{ik}^{C}. Say B_{im+1} = A_{im+1}^{C} for
simplicity.
( )
P (∩mk+=11 Bik) = P ACi ∩ ∩mk=1Bik
m+1
= P (∩m Bi) − P (Ai ∩∩m Bi )
k=1 k m+1 k=1 k
Then by induction,
∏m ( ) m∏ ∏m ( ( ))
= P (Bik)− P Aim+1 P (Bik) = P (Bik) 1− P Aim+1
k=1 k=1 k=1
( ) m∏ m+∏1
= P ACim+1 P (Bik) = P (Bik)
k=1 k=1
thus proving it for l + 1. ■
This motivates a more general notion of independence in terms of σ algebras.
Definition 58.3.3If
{ℱi}
_{i∈I}is any set of σ algebrascontained in ℱ, they are said tobe independent if whenever A_{ik}∈ℱ_{ik}for k = 1,2,
⋅⋅⋅
,m, then
m m∏
P (∩ k=1Aik) = P (Aik).
k=1
A set of random variables
{Xi}
_{i∈I}is independent if the σ algebras
{σ (Xi )}
_{i∈I}areindependent σ algebras. Here σ
(X )
denotes the smallest σ algebra such that X ismeasurable. Thus σ
(X)
=
{ −1 }
X (U) : U is a Borel set
. More generally, σ
(Xi : i ∈ I)
isthe smallest σ algebra such that each X_{i}is measurable.
Note that by Lemma 58.3.2 you can consider independent events in terms of
independent σ algebras. That is, a set of independent events can always be
considered as events taken from a set of independent σ algebras. This is a more
general notion because here the σ algebras might have infinitely many sets in
them.
Lemma 58.3.4Suppose the set of random variables,
{Xi }
_{i∈I}is independent.Also suppose I_{1}⊆ I and j
∈∕
I_{1}. Then the σ algebras σ
(Xi : i ∈ I1)
, σ
(Xj )
areindependent σ algebras.
Proof: Let B ∈ σ
(Xj )
. I want to show that for any A ∈ σ
(Xi : i ∈ I1)
, it
follows that P
(A ∩ B)
= P
(A )
P
(B )
. Let K consist of finite intersections of
sets of the form X_{k}^{−1}
(Bk)
where B_{k} is a Borel set and k ∈ I_{1}. Thus K is a π
system and σ
(K)
= σ
(Xi : i ∈ I1)
. Now if you have one of these sets of the form
A = ∩_{k=1}^{m}X_{k}^{−1}
(Bk )
where without loss of generality, it can be assumed the k are
distinct since X_{k}^{−1}
(Bk)
∩ X_{k}^{−1}
(B ′)
k
= X_{k}^{−1}
(Bk ∩B ′)
k
, then
m
P (A ∩ B) = P (∩m X −1(B )∩ B ) = P (B )∏ P (X− 1(B ))
k=1 k k k=1 k k
( m −1 )
= P (B)P ∩k=1X k (Bk) .
Thus K is contained in
G ≡ {A ∈ σ(Xi : i ∈ I1) : P (A∩ B ) = P (A)P (B)}.
Now G is closed with respect to complements and countable disjoint unions. Here is why:
If each A_{i}∈G and the A_{i} are disjoint,
P ((∪ ∞i=1Ai) ∩B ) = P (∪ ∞i=1(Ai ∩B ))
∑ ∑
= P (Ai ∩ B) = P (Ai)P (B)
i ∑ i
= P (B ) P (Ai) = P (B )P (∪∞i=1Ai)
i
If A ∈G,
( C )
P A ∩ B + P (A ∩B ) = P (B)
and so
( )
P AC ∩B = P (B)− P (A ∩B )
= P (B)− P (A)P (B)
= P (B)(1− P (A)) = P (B)P (AC ).
Therefore, from the lemma on π systems, Lemma 10.12.3 on Page 923, it follows
G⊇ σ
(K )
= σ
(Xi : i ∈ I1)
. ■
Lemma 58.3.5If
{Xk }
_{k=1}^{r}are independent random variables having values in Z aseparable metric space, and if g_{k}is a Borel measurable function, then
{gk(Xk)}
_{k=1}^{r}isalso independent. Furthermore, if the random variables have values in ℝ, and they are allbounded, then
( ∏r ) ∏r
E Xi = E(Xi).
i=1 i=1
More generally, the above formula holds if it is only known that each X_{i}∈ L^{1}
Now consider the last claim. Replace each X_{i} with X_{i}^{n} where this is just a truncation
of the form
(
{ Xi if |Xi| ≤ n
Xni ≡ n if Xi > n
( − n if Xi < n
Then by the first part
( )
∏r n ∏r n
E X i = E (X i )
i=1 i=1
Now
∏r n
| i=1X i |
≤
∏r
| i=1Xi |
∈ L^{1} and so by the dominated convergence theorem, you
can pass to the limit in both sides to get the desired result. ■
Maybe this would be a good place to put a really interesting result known as the
Doob Dynkin lemma. This amazing result is illustrated with the following diagram in
which X =
You start with X and can write it as the composition g ∘ X provided X is σ
(X )
measurable.
Lemma 58.3.6Let
(Ω, ℱ)
be a measure space and let X_{i} : Ω → E_{i}where E_{i}isa separable Banach space. Suppose also that X : Ω → F where F is a separableBanach space. Then X is σ
(X1,⋅⋅⋅,Xm )
measurable if and only if there exists aBorel measurable function g : ∏_{i=1}^{m}E_{i}→ F such that X = g