0
$\begingroup$

Suppose that the rate at which a Markov chain leaves state i at some time t is $\lambda_{i}$. I.e., What is the rate at which $X_{t}$ leaves state i.

Then, $\lambda_{i} = \sum_{j\neq i}q\left ( i,j \right )$ is the rate at which a Markov chain leaves a state i for j.

From observation, $\lambda_{i} = \sum_{j\neq i}q\left ( i,j \right )$ looks to be the definition of a marginal probability up to some state j.

Could someone kindly explain to me why there is the case? Why would the rate at which the chain moves be determined by how far is it transition previously? It's bizarre.

The notation $q(i,j)$ is also not clarified nor defined by the author.

From Essentials of stochastic processes by Durrett:
enter image description here

$\endgroup$
4
  • $\begingroup$ Unclear question: what is the role of $t$? And, given the definition of $\lambda_i$, which is the probability of not staying in $i$, i.e., $1-q(i,i)$, it is not a rate. $\endgroup$
    – Xi'an
    Apr 2, 2018 at 8:05
  • $\begingroup$ @Xi'an Edited. Hope that helps $\endgroup$
    – Physkid
    Apr 2, 2018 at 8:09
  • $\begingroup$ What is $q(i,j)$? Not a probability, obviously. And please add the reference to the quote you reposted. $\endgroup$
    – Xi'an
    Apr 2, 2018 at 8:11
  • $\begingroup$ If that is not a probability, then, I will struggle to know. The author has not defined what that means. $\endgroup$
    – Physkid
    Apr 2, 2018 at 8:28

1 Answer 1

2
$\begingroup$

The notation $q(i,j)$ is also not clarified nor defined by the author.

Yes, it's defined in equation (4.1) in that book. It's called the jumping rate.

Looking at that definition you will see that $\lambda_i$ doesn't depend on any past event.

$\endgroup$
2
  • $\begingroup$ Could you explain why the definition of rate for a Markov chain at sate i is defined the way it is? I'm trying to build an intuitive understanding of it. $\endgroup$
    – Physkid
    Apr 2, 2018 at 13:29
  • $\begingroup$ @Physkid Yes, you should post it as a new question. If you do so, please link that question here so I get notified. $\endgroup$
    – Winkelried
    Apr 5, 2018 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.