# Determine recurrent states via First Return Probability

I have the following question. It's about transient and recurrent states in Markov Chains. I know when a state is one or the other, but there is one thing I can't figure out or understand. We have learned that the sum of the First Return Probabilites (FRP) must be = 1 for a state to be recurrent, otherwise it is transient (see Figure 1).

In my example below both states should be recurrent, but if I calculate the First Return Probability for state 1 (a time step) I get 0.5. The others are irrelevant because we already had our First Return. So the sum of the FRP = 0.5 so < 1. But with that it would not be recurrent anymore. What am I doing wrong?

Figure 1:

Example:

• Why $f_{11}^{(2)} = 0$? It is clearly greater than $0$. Commented Feb 22, 2023 at 21:31

You made a mistake on evaluating $$f_{11}^{(2)}$$, which should be \begin{align} P[X_2 = 1, X_1 = 2 | X_0 = 1] = p_{12}p_{21} = \frac{1}{2} \times 1 = \frac{1}{2}. \end{align}
For $$n \geq 3$$, it indeed holds that $$f_{11}^{(n)} = 0$$, as once the chain visits the state $$2$$, it for sure will return to the state $$1$$ in the next step. Therefore, \begin{align} f_{11} = \sum_{n = 1}^\infty f_{11}^{(n)} = f_{11}^{(1)} + f_{11}^{(2)} = \frac{1}{2} + \frac{1}{2} = 1, \end{align} showing that the state $$1$$ is recurrent.