Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I will state the problem in a simplified manner first. Suppose I have two objects, that can be in one of two states. We count the number times when the object is in the second state. The distribution of number of such counts in a time period of length $\Delta t$ is Poisson with intensity $\Delta t\lambda$. The process, that each object generates is a simple Markov chain, with sojourn times (times when object is in the first state) distributed according to an exponential distribution with the same parameter $\lambda$. When we observe both objects, both Markov chains (of continuous time), at some times both chains can be found to be in the same (second) state simultaneously. So, my question is how to calculate the probability that two chains are in the same (second) state at a given time if I just have the rates of Poisson distributions? Would it be easier to simulate this problem? By that I mean that I could simulate two Markov chains for very long time and count how many times both chains simultaneously were in the same (second) state.

I am also interested in the generalization of this problem, when I have N Markov chains with different rates for theirthen Poisson distributions. The problem would be to estimate\calculate the probability of M chains being in the same state simultaneously for some M < N.

share|improve this question
Are the two continuous-time Market Chains independent? If so, you can combine them and get a single CTMC with four states, of which you are interested in only one. This is related to superposition (and thinning) of Poisson processes. The rates do not need to be equal to employ this approach. :-) – cardinal Aug 28 '12 at 12:41
Yes, the chains are independent. So, if I have two chains, each with two states, then after combining them I would get a chain with four states. But since I'm interested just in one state (say fourth, since I need an event when both markov chains are in the second state), I can just analyse a Markov chain with two states with the probabilities being calculated assuming independence? – Tomas Aug 28 '12 at 18:59

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.