# Ergodic theorem

We know that,

If $p_{jj}$ is the transition probability of staying in state j at n-th step and j at (n-1)-th step then the state j is said to be recurrent if,

$\sum_{n=0}^\infty p_{jj}^n = \infty$

and transient if,

$\sum_{n=0}^\infty p_{jj}^n \lt \infty$.

My problem of understanding is that if j is a recurrent state then the probability of returning to state j from state j in one step is $p_{jj}^1$ ,in two step is $p_{jj}^2$ and .......so on. So the probability of ever returning to the state j is the sum of the transition probabilities of returning to state j from j where n=1,2,......$\infty$.These probabilities are the probabilities of mutually events. So ,here how the probability of occurring an event is infinity?

I know the proof of the theorem but i don't understand the intuition behind the theorem and why the sum of probability is equal to infinity,if the state is recurrent?.

Can someone explain the theorem intuitively?

• Recurrence (as you probably know) means you return to the state infinitely often while transient means that at some point you will never return to that particular state. The summation result must have something to do with these definitions. Does the notation for p_jj mean the probability of return to j within n steps? If so p_jj will be 1 infinitely often for recurrent chains and only eventually 0 beyond some fixed n in thr transient case, maybe?. – Michael Chernick Jan 5 '17 at 19:02
• $p_jj$ means the probability of returning to state j from state j in one step. – Rhafi Sheikh Jan 5 '17 at 19:06
• That kills my theory. Is the n then there to mean that p_jj is raised to the nth power? With the definition you give p_jj would always be less than 1 in all both cases unless the chain can be caught in a single state (only possible for transient chains). – Michael Chernick Jan 5 '17 at 19:15
• @Michael The definitions differ from the characterizations given here, which is why there's a valid question. A transient state is one to which there is a non-zero chance of never returning once it has been visited. A recurrent state is anything else. See en.wikipedia.org/wiki/Markov_chain#Transience_and_recurrence. The theorem states that the expected number of visits to a recurrent state (once it is reached the first time) is infinite. The theorem's converse is more revealing: if there's any chance of not returning, then there's no chance of returning infinitely often. – whuber Jan 5 '17 at 19:17
• @whuber Where does ergodicity come in? Also as I recall recurrent is divided into two categories. One is positive recurrent and the other is null recurrent. – Michael Chernick Jan 5 '17 at 19:21

If the state $j$ is recurrent for the Markov chain $(X_t)$, $$\sum_{n=0}^\infty p_{jj}^n = \infty$$is equivalent to [in the sense that the lhs is the same] $$\sum_{n=0}^\infty \mathbb{P} (X_n=j|X_0=j) = \infty$$and to$$\sum_{n=0}^\infty \mathbb{E} [\mathbb{I}(X_n=j)|X_0=j]= \mathbb{E} \left[ \sum_{n=0}^\infty \mathbb{I}(X_n=j)\Big|X_0=j\right]=\infty$$which means the number of visits of state $j$ when starting from $j$ is on average infinite. (The state is Harris recurrent if the number of visits of state $j$ when starting from $j$ is almost surely infinite. The Markov chain is recurrent if there exists one recurrent state and if the chain is irreducible.)

The statement in the OP question

the probability of ever returning to the state j is the sum of the transition probabilities of returning to state j from j where n=1,2,......∞.These probabilities are the probabilities of mutually [exclusive] events.

is thus incorrect. Returns at times $n$ and $m$ are not exclusive events.

• In questioning the OP about p_jj the answer given was that p_jj was the probability of going from state j to state j in one step. That confused me I didn't think it was right. As you corrected that your answer is very clear . Well done! – Michael Chernick Jan 6 '17 at 6:15
• @MichaelChernick: well $p_{jj}$ is the probability to go from $j$ to $j$ in one step, while $p^n_{jj}$ is the probability to go from $j$ to $j$ in $n$ steps, with no constraint on earlier visits, $p^n_{jj}=\mathbb{P} (X_n=j|X_0=j)$. – Xi'an Jan 6 '17 at 6:18
• why the expected number of visiting j from j is equivalent to $\sum_{n=o}^\infty p_{jj}^n = \infty$ ? – Rhafi Sheikh Jan 6 '17 at 6:49
• As expressed in my answer,$$\sum_{n=0}^\infty p_{jj}^n = \sum_{n=0}^\infty \mathbb{P} (X_n=j|X_0=j) = \sum_{n=0}^\infty \mathbb{E} [\mathbb{I}(X_n=j)|X_0=j] = \mathbb{E} \left[ \sum_{n=0}^\infty \mathbb{I}(X_n=j)\Big|X_0=j\right]$$is the expected number of visits to $j$ when starting from $X_0=j$. – Xi'an Jan 6 '17 at 7:34
• @Xi'an At first i interpret $\sum_{n=o}^\infty p_{jj}^n = \infty$ as the probability of occurring the event returning to the state j from j in n time step so there is no way it will be greater than 1 but you pointed out that returns at times n and m are not exclusive events. So $\sum_{n=o}^\infty p_{jj}^n$ ,is not a probability rather it just a condition for recurrent state. – Rhafi Sheikh Jan 6 '17 at 15:18