Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I found the following definition:

"A probabilitly distribution $\pi = \{\pi_x\}_{x \in S}$ on the state space $S$ is called a stationary distribution for the Markov chain if for every $t > 0$,

$$ \pi^T P_t = \pi^T $$

What does $P_t$ mean? I thought it was the t'th step matrix of the transition matrix P but then this would be for discrete time markov chains and not continuous, right?

Oh wait, is it the transition matrix at time t?

share|improve this question
up vote 0 down vote accepted

You can always get a continuous time version of a discrete one by simply "Poissonizing" it. For example, if you have a discrete time Markov chain with transition matrix $T$ you get a continuous time version by considering $$P_t = \sum_{n\geq 0} \frac{t^n}{n!}\exp(-t)T^n $$ Hence the above definition makes sense in the context of continuous time Markov chains.

share|improve this answer
Yeah, but what does $P_t$ mean? Is it the t'th step transition matric or the transition matrix at time t? Or are these two things the same? – Kaish Nov 22 '12 at 15:10
$T^n$ is the $n$-th step transition matrix, i.e. the probability of finding the walker in state $y$ after $n$ steps, given it has started in state $x$ is $(y,T^n x)$. Similarly, the probability of being in state $y$ after time $t$ and starting at $x$ is $(y,P_t x)$. – Sven Stodtmann Nov 22 '12 at 15:53

I am answering rather than commenting due to lack of reputation:

Sven, your claim is incorrect: in your expression, T must be an infinitesimal rate matrix whose rows sum to 0, not a transition matrix whose rows sum to 1.

And now actually answering:

With CTMCs, different things happen to those who wait longer. $P_t$ denotes a transition matrix between observations at time $t_0$ and time $t_0 + t$. When $t$ goes to $0$, it approaches the identity matrix, and when $t$ goes to infinity, it approaches a matrix where every row in $\pi$. Those claim may require regularity conditions such as irreducibility. My entire answer also assumes the process is time-homogenous, i.e. $P_t$ doesn't depend on $t_0$.

share|improve this answer
Welcome to our site! – kjetil b halvorsen Sep 4 '15 at 13:16
+1. Thank you--this question has long needed a useful, relevant answer. – whuber Sep 4 '15 at 13:27

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.