Full Problem: A continuous-time Markov chain has generator matrix

$$Q= \begin{pmatrix} -1 & 1 & 0 \\ 1 & -2 & 1 \\ 2 & 2 & -4 \\ \end{pmatrix} $$

(i) Exhibit the transition matrix of the embedded Markov chain.

(ii) Exhibit the holding time parameters for each state.

OK, I must be misunderstanding something. I have the following for the embedded chain transition probabilities:

$$P_{ij} = \frac{q_{ij}}{q_i}$$

where $q_i$ is the parameters of the exponential length of time that the process stays in i.

The problem is that I know $q_i$ is the minimum with exponential distribution and parameter $\sum_k q_{ik}$.

But when I calculate $P_{11}$ that gives me $-1$ which is impossible..

Obviously there is an error in my thinking but can anyone shine light on what I'm misunderstanding? Thank you greatly in advance.

  • $\begingroup$ I discovered my mistake. The diagonals are always zero. I'll leave this here in case anyone finds themselves equally confused. $\endgroup$ – Nicklovn Apr 23 '18 at 14:44
  • $\begingroup$ It's unclear what you mean by "the diagonals are always zero," because that's not the case either with $Q$ or with the answer to (i). $\endgroup$ – whuber Apr 23 '18 at 15:41
  • 1
    $\begingroup$ The only way your readers would understand that would be for you to edit your post to elaborate on these requirements and restrictions. $\endgroup$ – whuber Apr 23 '18 at 22:14
  • 1
    $\begingroup$ It seems like you might be confusing a generator with the transition matrix itself. I'm not reading anything that distinguishes your question (in its current form) from the duplicate. $\endgroup$ – whuber Apr 23 '18 at 22:28
  • 1
    $\begingroup$ No no. I see where the confusion stems from. The question you linked to mine is asking for $P(t)$ which is the transition function. I was asking how to find the embedded chain transition matrix. I.e. the transition function with 0 waiting time. $\endgroup$ – Nicklovn Apr 23 '18 at 22:33

Indeed $P_{ij} = \frac{q_{ij}}{q_j}$ for $i\ne j$, so the transition matrix for the embedded Markov chain is $$ P=\pmatrix{ 0&1&0\\ \frac12&0&\frac12\\ \frac12&\frac12&0 }. $$ The holding time for state $0$ has $\mathsf{Exp}(1)$ distribution since $q_{01}=1$ and $q_{02}=2$. The holding time for state $1$ has $\mathsf{Exp}(2)$ since $q_{10}=q_{12}=1$ and the minimum of two independent exponential random variables is again exponential with rate equal to the sum of the two. A similar argument shows that the holding time for state $2$ has $\mathsf{Exp}(4)$ distribution.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.