Recurrence definition for a Markov chain We define a state i to be recurrent if $\sum\limits_{n=0}^\infty P(X_n=i,X_k \neq i$ for $1\leq k < n | X_0=i)$=1.
Why do we take infinite series over the probability? Why don't we define recurrent just as:
$P(X_n=i,X_k \neq i$ for $1\leq k < n | X_0=i)$=1
Doesn't this already mean, that if we start in s, we will with probability 1 (almost surely) return to s, after n steps?
Edit:
Also, i is said to be recurrent if:
$\sum\limits_{n=0}^\infty p_{ii}^{(n)}= \infty$.
How does this fit logically with the other definition? Why would the series of the transition probabilities be unbounded?
 A: $$P(X_n=i,X_k \neq i \text{ for } 1\leq k < n \mid X_0=i) = 1$$
This is the probability that the Markov chain will return to state $i$, for the first time, after exactly $n$ steps. What we need for recurrence, however, is the probability that the Markov chain will ever return to state $i$, no matter how long it takes.
$$\sum_{n = 1}^\infty P(X_n=i,X_k \neq i \text{ for } 1\leq k < n \mid X_0=i) = 1$$
This is the probability that the Markov chain will return after 1 step, 2 steps, 3 steps, or any number of steps.
$$p_{ii}^{(n)} = P(X_n = i \mid X_0 = i)$$
This is the probability that the Markov chain is in state $i$ after $n$ steps, but it might have returned to that state earlier. If a state is recurrent, we should expect it to be revisited infinitely often in an infinitely long Markov chain, right? Because if we have visited it once, we will most likely visit it again, and again, and again. The expected number of returns to state $i$ is given by
$$\sum_{n = 0}^\infty p_{ii}^{(n)}$$
and should therefore be infinite for a recurrent state.
