3
$\begingroup$

Say I have a light bulb that can be on (A) or off (B). It alternates between being state A or B. It will be in state A for a duration a ~ exp(α), and in state B for duration b ~ exp(β), (parameterized by mean), back and forth

I have a hunch that P(state at time t = A) approaches α / (α + β) as t -> infinity.

How can I find the exact probability of being in state A (or B) at time t, given the state at t0?. There are uncountably infinite possible sequences of state durations between now and time t

and what general type of problem is this? poisson something? markov something?

Thanks

$\endgroup$
1
  • 2
    $\begingroup$ I would describe this as a markov chain, as the next state depends only on the current state, ie the process has no "memory". $\endgroup$ Jun 9, 2019 at 1:22

2 Answers 2

3
$\begingroup$

If you're looking for an explicit solution, we can use Kolmogorov forward equations for continuous time Markov chains. In particular, we have the following rate matrix,

$$ Q = \begin{pmatrix}-\alpha & \alpha \\ \beta & -\beta \end{pmatrix} $$

Let $P(t)$ represent the transition probability matrix at any given time $t$. Then, the Kolmogorov forward equations satisfy, $\frac{d}{dt}P(t) = QP(t)$ which has the unique solution, $P(t) = e^{Qt}$. To make this calculation simpler, notice that $Q$ has the eigendecomposition, $Q = PDP^{-1}$ where, $$ P = \begin{pmatrix}-\alpha & 1 \\ \beta & 1 \end{pmatrix}, \;\;\;\; D = \begin{pmatrix}-(\alpha + \beta) & 0 \\ 0 & 0 \end{pmatrix}, \;\;\;\; P^{-1} = -\frac{1}{\alpha+\beta}\begin{pmatrix}1 & - 1 \\ -\beta & \alpha \end{pmatrix}$$

Then, $P(t) = e^{Qt} = Pe^{Dt}P^{-1}$. If you work out the matrix algebra here, you should obtain,

$$ P(t) = \frac{1}{\alpha+\beta}\begin{pmatrix}\alpha e^{-(\alpha+\beta)t} + \beta & \alpha e^{-(\alpha+\beta)t} + \alpha \\ -\beta e^{-(\alpha+\beta)t} + \beta & \beta e^{-(\alpha+\beta)t} + \alpha\end{pmatrix} $$

In particular, if you look at the first column of this matrix, they represent the probabilities of being in state $A$ given the initial state $A$ and $B$ respectively. In either case, the limiting behaviour as $t \rightarrow \infty$ (almost) matches your guess that the long run probability of being in state $A$ will be, $$\frac{\beta}{\alpha+\beta}$$

$\endgroup$
1
  • $\begingroup$ My guess was based on alpha being the mean rather than the rate parameter, so the answers are equivalent. You said "if you're looking for an explicit solution"...the wiki page notes that explicit solutions are hard for larger matrices (more states). But clearly it's fairly doable for 2 states. That said, is it uncommon to use these exact probabilities directly even in the case of 2 states? $\endgroup$ Jun 12, 2019 at 7:25
5
$\begingroup$

As @LuisCiadella Comments, this is a Markov Chain. The Markovian structure is inherited from the 'memoryless' property of exponential distributions. Because of its simple structure, it is easy to give an intuitive solution to the proportion of time spent in State A.

Beginning in State A we will wait for average time $\alpha$ before making a transition to State B. Then we remain in B for average time $\beta$ before transitioning back to A.

This cycle between A and B repeats endlessly. The average cycle lasts for time $\alpha + \beta,$ and of $\alpha$ of this time is spent in A.

Thus, as you suggest, the long-run proportion of time in State A is $$\frac{\alpha}{\alpha + \beta}.$$


Notes: (1) There are many ways to find the long-run behavior of this Markov chain. Methods of solution depend on the mathematical rigor with which the chain is described. The long-run behavior can also be well approximated by simulation. Perhaps other Answers will define the chain more rigorously and explore some of the alternate methods of solution.

(2) If we think of this chain as a shop with one clerk ('server') who is busy in State A and idle in state B, this process is an M/M/1/1 queue. The server remains busy for average time $\alpha.$ No customers can arrive when the server is busy. (Any customers who arrive when the server is busy are 'lost to the system' and not explicitly accounted for in this simple model.)

When the server is not busy, customers arrive at an exponential rate $1/\beta.$ An arriving customer begins service immediately, so that the average time the server is idle is $\beta.$

The description M/M/1/1 has the following meaning: The first M stands for memoryless or Markov (exponential) arrivals; the second M for Markov departures, the first $1$ stands for one server and the second $1$ stands for the limitation of only one customer at a time.

$\endgroup$
3
  • $\begingroup$ But is it possible to calculate the exact probability of being in state A at time t? given the state at t0 $\endgroup$ Jun 10, 2019 at 0:22
  • $\begingroup$ They purport to derive it at the bottom of the page numbered 148: u.math.biu.ac.il/~amirgi/CTMCnotes.pdf But I get confused when they say λ (x)h + o(h) as h -> 0. Is that some sort of linear approximation? and why? $\endgroup$ Jun 10, 2019 at 0:34
  • $\begingroup$ Looks like there's a solution under Transient Behavior here: en.wikipedia.org/wiki/Markov_chain#Transient_behaviour but they don't explain it very well for non-experts $\endgroup$ Jun 10, 2019 at 1:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.