4
$\begingroup$

I am confused about the notation in my Stochastic Processes class, and I can't find a place in the textbook that explicitly defines this notation. He uses $E$ to mean simple expectation i.e. for tail sum formula for expectation it is written $$EX = \sum_{k = 1}^\infty P(X \ge k)$$ Then in the next paragraph, he writes that the probability of returning at least $k$ times $\{N(y) \ge k\}$ is $$E_xN(y) = \sum_{k = 1}P(N(y) \ge k) = \cdots = {\rho_{xy} \over 1 - \rho_{yy}}$$ where I added the \cdots because the calculations aren't relevant.

As a direct consequence of this, I am confused about what the "limiting fraction of time we spend in each state" theorem is saying. It goes as follows:

Theorem 1.21 (Asymptotic Frequency). Suppose our Markov Chains is irreducible and all states are recurrent. If $N_n(y)$ is the number of visits to $y$ up to time $n$, then $${N_n(y) \over n} \to {1 \over E_yT_y}$$

where $T_y$ is defined to be $T_y^1$ for $$T_y^k = \min\{n > T_y^{k-1}: X_n = y\}$$ i.e. the time of first return to $y$.

$\endgroup$
3
  • 2
    $\begingroup$ $E_y$ is the expected value obtained when the chain is started from state $y$. $\endgroup$
    – cardinal
    Commented Sep 23, 2012 at 16:55
  • 1
    $\begingroup$ $E_y$ is the expectation when we assume that the Markov chain starts at $y$. Is it the point you misunderstand ? $\endgroup$ Commented Sep 23, 2012 at 16:57
  • 1
    $\begingroup$ @StéphaneLaurent I think you have it. That is exactly what i was about to say. It is apparent that the chain starts at y since "time to first return to y" is mentioned. $\endgroup$ Commented Sep 23, 2012 at 17:29

1 Answer 1

4
$\begingroup$

As you can see from the remark of Stephane followed by confirming remarks by me and cardinal that E$_y$T$_y$ is the expected value of the first return time to y because E$_y$ mean expectation conditional on

X$_1$=y.

$\endgroup$
4
  • $\begingroup$ I realize this is a meta comment/question, but do you see the comments to the OP in a different order than they show in my browser? $\endgroup$
    – cardinal
    Commented Sep 23, 2012 at 17:38
  • $\begingroup$ Sorry Cardinal. Your comment came in slightly ahead of Stephane. I missed yours and responded to Stephane. So when I noticed your I thought it came later. I did not check the times. You probably were writing your comments at overlapping times. $\endgroup$ Commented Sep 23, 2012 at 18:00
  • $\begingroup$ No problem. The comments do appear to be within a couple minutes of each other. I was mostly just curious if there was a technical reason (i.e., bug), since the order of comments can be important in other situations. $\endgroup$
    – cardinal
    Commented Sep 23, 2012 at 19:41
  • $\begingroup$ @cardinal I think that the order was reversed and that may explain why I thought Stephane's comment came in first but it does seem to be corrected now. $\endgroup$ Commented Sep 23, 2012 at 21:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.