Sometimes it is required to consider double series which are of the form

In other words, first sum on j yielding something which depends on k and then sum these. The major consideration for these double series is the question of when

In other words, when does it make no difference which subscript is summed over first? In the case of finite sums there is no issue here. You can always write

because addition is commutative. However, there are limits involved with infinite sums and the interchange in order of summation involves taking limits in a different order. Therefore, it is not always true that it is permissible to interchange the two sums. A general rule of thumb is this: If something involves changing the order in which two limits are taken, you may not do it without agonizing over the question. In general, limits foul up algebra and also introduce things which are counter intuitive. Here is an example. This example is a little technical. It is placed here just to prove conclusively there is a question which needs to be considered.
Example 1.5.1 Consider the following picture which depicts some of the ordered pairs
The numbers next to the point are the values of a_{mn}. You see a_{nn} = 0 for all n, a_{21} = a,a_{12} = b,a_{mn} = c for
Then ∑ _{m=1}^{∞}a_{mn} = a if n = 1, ∑ _{m=1}^{∞}a_{mn} = b−c if n = 2 and if n > 2,∑ _{m=1}^{∞}a_{mn} = 0. Therefore,

Next observe that ∑ _{n=1}^{∞}a_{mn} = b if m = 1,∑ _{n=1}^{∞}a_{mn} = a + c if m = 2, and ∑ _{n=1}^{∞}a_{mn} = 0 if m > 2. Therefore,

and so the two sums are different. Moreover, you can see that by assigning different values of a,b, and c, you can get an example for any two different numbers desired.
It turns out that if a_{ij} ≥ 0 for all i,j, then you can always interchange the order of summation. This is shown next and is based on the following lemma. First, some notation should be discussed.
Definition 1.5.2 Let f
Unlike limits, you can take the sup in different orders.
Proof: Note that for all a,b, f

Repeat the same argument interchanging a and b, to get the conclusion of the lemma. ■
Proof: First note there is no trouble in defining these sums because the a_{ij} are all nonnegative. If a sum diverges, it only diverges to ∞ and so ∞ is the value of the sum. Next note that

because for all j,

Therefore,
