# What does $E_yT_y$ mean?

I am confused about the notation in my Stochastic Processes class, and I can't find a place in the textbook that explicitly defines this notation. He uses $E$ to mean simple expectation i.e. for tail sum formula for expectation it is written $$EX = \sum_{k = 1}^\infty P(X \ge k)$$ Then in the next paragraph, he writes that the probability of returning at least $k$ times $\{N(y) \ge k\}$ is $$E_xN(y) = \sum_{k = 1}P(N(y) \ge k) = \cdots = {\rho_{xy} \over 1 - \rho_{yy}}$$ where I added the \cdots because the calculations aren't relevant.

As a direct consequence of this, I am confused about what the "limiting fraction of time we spend in each state" theorem is saying. It goes as follows:

Theorem 1.21 (Asymptotic Frequency). Suppose our Markov Chains is irreducible and all states are recurrent. If $N_n(y)$ is the number of visits to $y$ up to time $n$, then $${N_n(y) \over n} \to {1 \over E_yT_y}$$

where $T_y$ is defined to be $T_y^1$ for $$T_y^k = \min\{n > T_y^{k-1}: X_n = y\}$$ i.e. the time of first return to $y$.

• $E_y$ is the expected value obtained when the chain is started from state $y$. Commented Sep 23, 2012 at 16:55
• $E_y$ is the expectation when we assume that the Markov chain starts at $y$. Is it the point you misunderstand ? Commented Sep 23, 2012 at 16:57
• @StéphaneLaurent I think you have it. That is exactly what i was about to say. It is apparent that the chain starts at y since "time to first return to y" is mentioned. Commented Sep 23, 2012 at 17:29

As you can see from the remark of Stephane followed by confirming remarks by me and cardinal that E$_y$T$_y$ is the expected value of the first return time to y because E$_y$ mean expectation conditional on
X$_1$=y.