1
$\begingroup$

I found the following definition:

"A probabilitly distribution $\pi = \{\pi_x\}_{x \in S}$ on the state space $S$ is called a stationary distribution for the Markov chain if for every $t > 0$,

$$ \pi^T P_t = \pi^T $$

What does $P_t$ mean? I thought it was the t'th step matrix of the transition matrix P but then this would be for discrete time markov chains and not continuous, right?

Oh wait, is it the transition matrix at time t?

$\endgroup$
1
$\begingroup$

You can always get a continuous time version of a discrete one by simply "Poissonizing" it. For example, if you have a discrete time Markov chain with transition matrix $T$ you get a continuous time version by considering $$P_t = \sum_{n\geq 0} \frac{t^n}{n!}\exp(-t)T^n $$ Hence the above definition makes sense in the context of continuous time Markov chains.

$\endgroup$
  • 1
    $\begingroup$ Yeah, but what does $P_t$ mean? Is it the t'th step transition matric or the transition matrix at time t? Or are these two things the same? $\endgroup$ – Kaish Nov 22 '12 at 15:10
  • $\begingroup$ $T^n$ is the $n$-th step transition matrix, i.e. the probability of finding the walker in state $y$ after $n$ steps, given it has started in state $x$ is $(y,T^n x)$. Similarly, the probability of being in state $y$ after time $t$ and starting at $x$ is $(y,P_t x)$. $\endgroup$ – Sven Stodtmann Nov 22 '12 at 15:53
2
$\begingroup$

I am answering rather than commenting due to lack of reputation:

Sven, your claim is incorrect: in your expression, T must be an infinitesimal rate matrix whose rows sum to 0, not a transition matrix whose rows sum to 1.

And now actually answering:

With CTMCs, different things happen to those who wait longer. $P_t$ denotes a transition matrix between observations at time $t_0$ and time $t_0 + t$. When $t$ goes to $0$, it approaches the identity matrix, and when $t$ goes to infinity, it approaches a matrix where every row in $\pi$. Those claim may require regularity conditions such as irreducibility. My entire answer also assumes the process is time-homogenous, i.e. $P_t$ doesn't depend on $t_0$.

$\endgroup$
  • $\begingroup$ +1. Thank you--this question has long needed a useful, relevant answer. $\endgroup$ – whuber Sep 4 '15 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.