Alternating between two states {A, B} each with exp distributed durations. What's the probability of state=A at time t?

Question

Say I have a light bulb that can be on (A) or off (B). It alternates between being state A or B. It will be in state A for a duration a ~ exp(α), and in state B for duration b ~ exp(β), (parameterized by mean), back and forth

I have a hunch that P(state at time t = A) approaches α / (α + β) as t -> infinity.

How can I find the exact probability of being in state A (or B) at time t, given the state at t0?. There are uncountably infinite possible sequences of state durations between now and time t

and what general type of problem is this? poisson something? markov something?

Thanks

Answer 1

If you're looking for an explicit solution, we can use Kolmogorov forward equations for continuous time Markov chains. In particular, we have the following rate matrix,

$$ Q = \begin{pmatrix}-\alpha & \alpha \\ \beta & -\beta \end{pmatrix} $$

Let $P(t)$ represent the transition probability matrix at any given time $t$. Then, the Kolmogorov forward equations satisfy, $\frac{d}{dt}P(t) = QP(t)$ which has the unique solution, $P(t) = e^{Qt}$. To make this calculation simpler, notice that $Q$ has the eigendecomposition, $Q = PDP^{-1}$ where, $$ P = \begin{pmatrix}-\alpha & 1 \\ \beta & 1 \end{pmatrix}, \;\;\;\; D = \begin{pmatrix}-(\alpha + \beta) & 0 \\ 0 & 0 \end{pmatrix}, \;\;\;\; P^{-1} = -\frac{1}{\alpha+\beta}\begin{pmatrix}1 & - 1 \\ -\beta & \alpha \end{pmatrix}$$

Then, $P(t) = e^{Qt} = Pe^{Dt}P^{-1}$. If you work out the matrix algebra here, you should obtain,

$$ P(t) = \frac{1}{\alpha+\beta}\begin{pmatrix}\alpha e^{-(\alpha+\beta)t} + \beta & \alpha e^{-(\alpha+\beta)t} + \alpha \\ -\beta e^{-(\alpha+\beta)t} + \beta & \beta e^{-(\alpha+\beta)t} + \alpha\end{pmatrix} $$

In particular, if you look at the first column of this matrix, they represent the probabilities of being in state $A$ given the initial state $A$ and $B$ respectively. In either case, the limiting behaviour as $t \rightarrow \infty$ (almost) matches your guess that the long run probability of being in state $A$ will be, $$\frac{\beta}{\alpha+\beta}$$

Answer 2

As @LuisCiadella Comments, this is a Markov Chain. The Markovian structure is inherited from the 'memoryless' property of exponential distributions. Because of its simple structure, it is easy to give an intuitive solution to the proportion of time spent in State A.

Beginning in State A we will wait for average time $\alpha$ before making a transition to State B. Then we remain in B for average time $\beta$ before transitioning back to A.

This cycle between A and B repeats endlessly. The average cycle lasts for time $\alpha + \beta,$ and of $\alpha$ of this time is spent in A.

Thus, as you suggest, the long-run proportion of time in State A is $$\frac{\alpha}{\alpha + \beta}.$$

---

Notes: (1) There are many ways to find the long-run behavior of this Markov chain. Methods of solution depend on the mathematical rigor with which the chain is described. The long-run behavior can also be well approximated by simulation. Perhaps other Answers will define the chain more rigorously and explore some of the alternate methods of solution.

(2) If we think of this chain as a shop with one clerk ('server') who is busy in State A and idle in state B, this process is an M/M/1/1 queue. The server remains busy for average time $\alpha.$ No customers can arrive when the server is busy. (Any customers who arrive when the server is busy are 'lost to the system' and not explicitly accounted for in this simple model.)

When the server is not busy, customers arrive at an exponential rate $1/\beta.$ An arriving customer begins service immediately, so that the average time the server is idle is $\beta.$

The description M/M/1/1 has the following meaning: The first M stands for memoryless or Markov (exponential) arrivals; the second M for Markov departures, the first $1$ stands for one server and the second $1$ stands for the limitation of only one customer at a time.