Transition rates in continuous time markov chain

Question

A house has 2 rooms of similar sizes with identical air conditioners equipped with thermostats which turn on and off as needed to maintain the temperature in each room to a desired level of 22 degrees. Suppose that a thermostat remains on or off for exponential amounts of time with means $1/\mu$ and $1/\lambda$, respectively, independently of other thermostats. Consider the Markov process $\{X(t), t \ge 0\}$ whose state space is the number of active air conditioners. Write down the matrix of transition rates.

I'm not sure how to exactly approach this type of question. My working is as follows but if someone could clarify my confusion that would be good.

Working:

So clearly there are 3 states, 0 for no air conditioners are on, 1 for one air conditioner is on (active), and 2 for two air conditioners are on (active). Now to work out $q_{01}$, i.e., the transition rate from state 0 to 1, assume currently no air conditioners are on. Consider the two independent poisson processes both with parameter $\lambda$, where the interarrival time is the duration of "off", then merging these two processes gives a poisson process with parameter $2\lambda$, so the transition rate from state 0 to state 1 is $2\lambda$.

Now what about the transition rate from state 0 to 2? I am told that it's 0, but why? Isn't it possible for both air conditioners to both go from "off" to "on"? What is the argument that $q_{02} = 0$?

Answer 1

This question was asked recently on math.SE and the following points were already explained, at least partly. Unsurprisingly, the migration to stats.SE does not change the underlying mathematical structure of the model, so, here we go again.

First, there is no jump from state $0$ to state $2$ or from state $2$ to state $0$ because such a jump would require that both thermostats turn on at the same time or that they both turn off at the same time. Since the times when the thermostats switch are independent and absolutely continuous, they never coincide hence transitions $0\to2$ or $2\to0$ never occur, almost surely.

(Relevant lemma: If $\xi$ and $\eta$ are independent random variables and, say, $P(\xi=x)=0$ for every $x$, then $P(\xi=\eta)=0$.)

Second, except when $\mu=\lambda$, the process $X$ is not a Markov process. True, neither the time before a jump from state $0$ nor the time before a jump from state $2$ depend on the past, but the time before a jump from state $1$ very much does so.

To see why, imagine that $\lambda\ll\mu$ and that $X$ jumps to state $1$ coming from state $0$. This means that both thermostats were off before one of them switched on, causing the jump to state $1$. As is well known, the probability that the thermostat which switched on is the $\lambda$-thermostat is $\lambda/(\lambda+\mu)$ and the probability that it is the $\mu$-thermostat is $\mu/(\lambda+\mu)$ hence, most probably, $X$ arrived at state $1$ because the $\mu$-thermostat switched on. This means that one is more likely to see the $\mu$-thermostat switch again (this time, turning off) before the $\lambda$-thermostat does (thus, turning on), hence $X$ has more chances to go back to state $0$ than to move to state $2$.

Thus, the path $0\to1\to0$ is more probable than the path $0\to1\to2$. A similar reasoning shows that the path $2\to1\to2$ is more probable than the path $2\to1\to0$. This is impossible if $X$ is a Markov process.

The case $\lambda=\mu$ is exceptional, then $X$ is a Markov process, with transition rates $2\lambda$ for $0\to1$ and $2\to1$ and $\lambda$ for $1\to0$ and $1\to2$. Thus, as was to be expected, the stationary distribution is then $(\frac14,\frac12,\frac14)$, that is, binomial $(2,\frac12)$.

A final remark: to start from a Markov process (here, the pair of states of the thermostats) and to lump together some of its states (here, the states on-off and off-on) is a classical way to destroy the Markov property.