2.5. DOUBLE SERIES 65
and so the two sums are different. Moreover, you can see that by assigning different valuesof a,b, and c, you can get an example for any two different numbers desired.
It turns out that if ai j ≥ 0 for all i, j, then you can always interchange the order ofsummation. This is shown next and is based on the following lemma. First, some notationshould be discussed.
Definition 2.5.2 Let f (a,b)∈ [−∞,∞] for a∈A and b∈B where A,B are sets whichmeans that f (a,b) is either a number, ∞, or −∞. The symbol, +∞ is interpreted as a pointout at the end of the number line which is larger than every real number. Of course there isno such number. That is why it is called ∞. The symbol, −∞ is interpreted similarly. Thensupa∈A f (a,b) means sup(Sb) where Sb ≡ { f (a,b) : a ∈ A} .
Unlike limits, you can take the sup in different orders.
Lemma 2.5.3 Let f (a,b) ∈ [−∞,∞] for a ∈ A and b ∈ B where A,B are sets. Then
supa∈A
supb∈B
f (a,b) = supb∈B
supa∈A
f (a,b) .
Proof: Note that for all a,b, f (a,b) ≤ supb∈B supa∈A f (a,b) and therefore, for all a,supb∈B f (a,b)≤ supb∈B supa∈A f (a,b). Therefore,
supa∈A
supb∈B
f (a,b)≤ supb∈B
supa∈A
f (a,b) .
Repeat the same argument interchanging a and b, to get the conclusion of the lemma. ■
Theorem 2.5.4 Let ai j ≥ 0. Then ∑∞i=1 ∑
∞j=1 ai j = ∑
∞j=1 ∑
∞i=1 ai j.
Proof: First note there is no trouble in defining these sums because the ai j are allnonnegative. If a sum diverges, it only diverges to ∞ and so ∞ is the value of the sum. Nextnote that
∞
∑j=r
∞
∑i=r
ai j ≥ supn
∞
∑j=r
n
∑i=r
ai j
because for all j,∑∞i=r ai j ≥ ∑
ni=r ai j.Therefore,
∞
∑j=r
∞
∑i=r
ai j ≥ supn
∞
∑j=r
n
∑i=r
ai j = supn
limm→∞
m
∑j=r
n
∑i=r
ai j
= supn
limm→∞
n
∑i=r
m
∑j=r
ai j = supn
n
∑i=r
limm→∞
m
∑j=r
ai j
= supn
n
∑i=r
∞
∑j=r
ai j = limn→∞
n
∑i=r
∞
∑j=r
ai j =∞
∑i=r
∞
∑j=r
ai j
Interchanging the i and j in the above argument proves the theorem. ■