Suppose that the rate at which a Markov chain leaves state i at some time t is $\lambda_{i}$. I.e., What is the rate at which $X_{t}$ leaves state i.

Then, $\lambda_{i} = \sum_{j\neq i}q\left ( i,j \right )$ is the rate at which a Markov chain leaves a state i for j.

From observation, $\lambda_{i} = \sum_{j\neq i}q\left ( i,j \right )$ looks to be the definition of a marginal probability up to some state j.

Could someone kindly explain to me why there is the case? Why would the rate at which the chain moves be determined by how far is it transition previously? It's bizarre.

The notation $q(i,j)$ is also not clarified nor defined by the author.

From Essentials of stochastic processes by Durrett:
enter image description here

  • $\begingroup$ Unclear question: what is the role of $t$? And, given the definition of $\lambda_i$, which is the probability of not staying in $i$, i.e., $1-q(i,i)$, it is not a rate. $\endgroup$ – Xi'an Apr 2 '18 at 8:05
  • $\begingroup$ @Xi'an Edited. Hope that helps $\endgroup$ – Physkid Apr 2 '18 at 8:09
  • $\begingroup$ What is $q(i,j)$? Not a probability, obviously. And please add the reference to the quote you reposted. $\endgroup$ – Xi'an Apr 2 '18 at 8:11
  • $\begingroup$ If that is not a probability, then, I will struggle to know. The author has not defined what that means. $\endgroup$ – Physkid Apr 2 '18 at 8:28

The notation $q(i,j)$ is also not clarified nor defined by the author.

Yes, it's defined in equation (4.1) in that book. It's called the jumping rate.

Looking at that definition you will see that $\lambda_i$ doesn't depend on any past event.

| cite | improve this answer | |
  • $\begingroup$ Could you explain why the definition of rate for a Markov chain at sate i is defined the way it is? I'm trying to build an intuitive understanding of it. $\endgroup$ – Physkid Apr 2 '18 at 13:29
  • $\begingroup$ @Physkid Yes, you should post it as a new question. If you do so, please link that question here so I get notified. $\endgroup$ – Winkelried Apr 5 '18 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.