Let $E_j$ be a particular state in a sequence of finite states that qualify to follow the Markov Chain Property. If $E_j$ is persistent then by definition, \begin{equation} f_{jj}=\sum_{n=1}^{\infty} f_{jj}^{(n)}=1 \end{equation} where $f_{jj}^{(n)}$ is the probability of occurrence of $E_{j}$ for the first time in $n$ steps, therefore $f_{jj}$ is the probability of recurrence of the event $E_{j}$.
From the above can we conclude that $E_j$ is reachable from initial state $E_j$ or mathematically, $\exists n>0$ such that $p_{jj}^{(n)} > 0$, where $p_{jj}^{(n)}$ is the occurrence of $E_{j}$ after $n$ steps starting from $E_{j}$ initially.

Will the converse of the above statement hold true in general?

  • $\begingroup$ You did not define your notations, I presume that $f_{jj}^{(n)}$ is the probability to first return to $j$ in $n$ steps. In which case I would call the state $E_j$ recurrent. What are the roles of $f_{jj}$ and of $p_{jj}^{(n)}$? $\endgroup$ – Xi'an Oct 20 '15 at 7:00
  • $\begingroup$ @Xi'an I've edited my question by defining the notations. $\endgroup$ – Akshay Bansal Oct 20 '15 at 8:27

I'm not entirely clear on your notation. You write "$p\left(n\right)_{jj}$ is the occurrence of $E_j$ after $n$ steps starting from $E_j$ initially." I'm going to assume you mean $p\left(n\right)_{jj}$ is the probability of being in state $E_j$ conditional on being there $n$ steps ago; note that this includes events where I transition back to $E_j$ in less than $n$ steps.

To be clear, let $X\left(t\right)$ be the state at time (or step) $t$. I am assuming your notation is

$\begin{equation*} p\left(n\right)_{jj} = \Pr\left[X\left(t+n\right) = E_j \;|\; X\left(t\right) = E_j\right] \end{equation*}$

$\begin{equation*} f\left(n\right)_{jj} = \Pr\left[X\left(t+n\right) = E_j, \,X\left(t+n-1\right) \neq E_j, \,\ldots, \, X\left(t+1\right) \neq E_j\;|\; X\left(t\right) = E_j\right] \end{equation*}$

This implies $f \leq p$.

You ask whether A implies B. I'm going to show that A and not-B can both be true (meaning the answer to your question is no). Consider a simple example with states $E_1$ and $E_2$. Conditional on state 1 you transition to 2 with probability 0.9 (and back to 1 otherwise), while 2 is an absorbing state.

In that case $f_{11}$ = 0.1, and state 1 is transient. But notice that $p\left(n\right)_{11} = 0.1^n > 0$ for all $n$. Hence the existence of an $n>0$ such that $p\left(n\right)_{jj}>0$ does not imply recurrence.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.