29.3. DISCRETE STOCHASTIC PROCESSES 787

Lemma 29.3.2 Let φ be a convex and increasing function and suppose

{(Xn,Fn)}

is a sub-martingale. Then if E (|φ (Xn)|)< ∞, it follows

{(φ (Xn) , Fn)}

is also a sub-martingale.

Proof: It is given that E (Xn+1,Fn)≥ Xn and so

φ (Xn)≤ φ (E (Xn+1|Fn))≤ E (φ (Xn+1) |Fn)

by Jensen’s inequality. ■Certainly one of the most amazing things about sub-martingales is the convergence

theorem. Recall the earlier sub-martingale convergence theorem. I am going to present thesame thing here in this more general setting. Then later, I will present it again in the caseof continuous sub-martingales. This which follows is almost identical to the earlier proof.

So why did I even bother with the earlier development? It is because I wanted to makea smooth transition from the idea we usually have of conditional probability where it isprobability of something given the value of something else to this more general and muchmore abstract notion involving σ algebras.

An upcrossing occurs when a sequence goes from a up to b. Thus it crosses the interval,[a,b] in the up direction, hence the name upcrossing. More precisely,

Definition 29.3.3 Let {xi}Ii=1 be any sequence of real numbers, I ≤ ∞. Define an

increasing sequence of integers {mk} as follows. m1 is the first integer ≥ 1 such thatxm1 ≤ a, m2 is the first integer larger than m1 such that xm2 ≥ b, m3 is the first integerlarger than m2 such that xm3 ≤ a, etc. Then each sequence,

{xm2k−1 , · · · ,xm2k

}, is called an

upcrossing of [a,b].

Here is a picture of an upcrossing.

b

a

Proposition 29.3.4 Let {Xi}ni=1 be a finite sequence of real random variables defined

on Ω where (Ω,F ,P) is a probability space. Let U[a,b] (ω) denote the number of upcross-ings of Xi (ω) of the interval [a,b]. Then U[a,b] is a random variable.

Proof: Let X0 (ω) ≡ a + 1, let Y0 (ω) ≡ 0, and let Yk (ω) remain 0 for k = 0, · · · , luntil Xl (ω) ≤ a. When this happens (if ever), Yl+1 (ω) ≡ 1. Then let Yi (ω) remain 1 fori = l + 1, · · · ,r until Xr (ω) ≥ b when Yr+1 (ω) ≡ 0. Let Yk (ω) remain 0 for k ≥ r + 1until Xk (ω)≤ a when Yk (ω)≡ 1 and continue in this way. Thus the upcrossings of Xi (ω)