I found the following definition:

"A probabilitly distribution $\pi = \{\pi_x\}_{x \in S}$ on the state space $S$ is called a stationary distribution for the Markov chain if for every $t > 0$,

$$ \pi^T P_t = \pi^T $$

What does $P_t$ mean? I thought it was the t'th step matrix of the transition matrix P but then this would be for discrete time markov chains and not continuous, right?

Oh wait, is it the transition matrix at time t?


2 Answers 2


Please read the correct answer from the other post, as mentioned there, this answer incorrectly assumes a rate matrix instead of a transition Matrix.

You can always get a continuous time version of a discrete one by simply "Poissonizing" it. For example, if you have a discrete time Markov chain with transition matrix $T$ you get a continuous time version by considering $$P_t = \sum_{n\geq 0} \frac{t^n}{n!}\exp(-t)T^n $$ Hence the above definition makes sense in the context of continuous time Markov chains.

  • 1
    $\begingroup$ Yeah, but what does $P_t$ mean? Is it the t'th step transition matric or the transition matrix at time t? Or are these two things the same? $\endgroup$
    – Kaish
    Commented Nov 22, 2012 at 15:10
  • $\begingroup$ $T^n$ is the $n$-th step transition matrix, i.e. the probability of finding the walker in state $y$ after $n$ steps, given it has started in state $x$ is $(y,T^n x)$. Similarly, the probability of being in state $y$ after time $t$ and starting at $x$ is $(y,P_t x)$. $\endgroup$ Commented Nov 22, 2012 at 15:53

I am answering rather than commenting due to lack of reputation:

Sven, your claim is incorrect: in your expression, T must be an infinitesimal rate matrix whose rows sum to 0, not a transition matrix whose rows sum to 1.

And now actually answering:

With CTMCs, different things happen to those who wait longer. $P_t$ denotes a transition matrix between observations at time $t_0$ and time $t_0 + t$. When $t$ goes to $0$, it approaches the identity matrix, and when $t$ goes to infinity, it approaches a matrix where every row in $\pi$. Those claim may require regularity conditions such as irreducibility. My entire answer also assumes the process is time-homogenous, i.e. $P_t$ doesn't depend on $t_0$.

  • $\begingroup$ +1. Thank you--this question has long needed a useful, relevant answer. $\endgroup$
    – whuber
    Commented Sep 4, 2015 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.