# Equivalent definition of persistence of state in Markov Chain

Let $E_j$ be a particular state in a sequence of finite states that qualify to follow the Markov Chain Property. If $E_j$ is persistent then by definition, \begin{equation} f_{jj}=\sum_{n=1}^{\infty} f_{jj}^{(n)}=1 \end{equation} where $f_{jj}^{(n)}$ is the probability of occurrence of $E_{j}$ for the first time in $n$ steps, therefore $f_{jj}$ is the probability of recurrence of the event $E_{j}$.
From the above can we conclude that $E_j$ is reachable from initial state $E_j$ or mathematically, $\exists n>0$ such that $p_{jj}^{(n)} > 0$, where $p_{jj}^{(n)}$ is the occurrence of $E_{j}$ after $n$ steps starting from $E_{j}$ initially.

Will the converse of the above statement hold true in general?

• You did not define your notations, I presume that $f_{jj}^{(n)}$ is the probability to first return to $j$ in $n$ steps. In which case I would call the state $E_j$ recurrent. What are the roles of $f_{jj}$ and of $p_{jj}^{(n)}$? – Xi'an Oct 20 '15 at 7:00
• @Xi'an I've edited my question by defining the notations. – Akshay Bansal Oct 20 '15 at 8:27

I'm not entirely clear on your notation. You write "$p\left(n\right)_{jj}$ is the occurrence of $E_j$ after $n$ steps starting from $E_j$ initially." I'm going to assume you mean $p\left(n\right)_{jj}$ is the probability of being in state $E_j$ conditional on being there $n$ steps ago; note that this includes events where I transition back to $E_j$ in less than $n$ steps.

To be clear, let $X\left(t\right)$ be the state at time (or step) $t$. I am assuming your notation is

$\begin{equation*} p\left(n\right)_{jj} = \Pr\left[X\left(t+n\right) = E_j \;|\; X\left(t\right) = E_j\right] \end{equation*}$

$\begin{equation*} f\left(n\right)_{jj} = \Pr\left[X\left(t+n\right) = E_j, \,X\left(t+n-1\right) \neq E_j, \,\ldots, \, X\left(t+1\right) \neq E_j\;|\; X\left(t\right) = E_j\right] \end{equation*}$

This implies $f \leq p$.

You ask whether A implies B. I'm going to show that A and not-B can both be true (meaning the answer to your question is no). Consider a simple example with states $E_1$ and $E_2$. Conditional on state 1 you transition to 2 with probability 0.9 (and back to 1 otherwise), while 2 is an absorbing state.

In that case $f_{11}$ = 0.1, and state 1 is transient. But notice that $p\left(n\right)_{11} = 0.1^n > 0$ for all $n$. Hence the existence of an $n>0$ such that $p\left(n\right)_{jj}>0$ does not imply recurrence.