I'm learning markov chains in order to compute estimations of transition probabilities, and I found an example of the estimator construction for continuous time markov chains:


(pages 10 -12)

In its presentation, the author specifies that it's necessary to fix matrix diagonal and replace it by the negative of row sums without considering it's initial value:

    D <- rep(0, dim(Lambda.hat)[2])
    Lambda.hat <-rbind(Lambda.hat,D)
    diag(Lambda.hat) <- D
    rowsums <- apply(Lambda.hat,1,sum)
    diag(Lambda.hat) <- -rowsums

After that, he computes estimated transition probabilities:

    P.hat <- expm(Lambda.hat)

The output is consistent because each row adds up to one so it seems it works. Although, I don't understand why it was necessary to make these two steps and would really appreciate any help.


There is no "initial" value of the diagonal entries in the infinitesimal generator matrix, a.k.a. transition rate matrix, I'll call it Q, of a continuous time Markov Chain. The author shows how to estimate the off-diagonal transition entries of the infinitesimal generator matrix. As is always the case with an infinitesimal generator matrix, each row must sum to zero, as described in https://en.wikipedia.org/wiki/Transition_rate_matrix . Therefore, the diagonal entries of the infinitesimal generator matrix are set equal to the negative of the sum of the entries in the corresponding row.

As described at https://en.wikipedia.org/wiki/Continuous-time_Markov_chain#Transient_behaviour , the transition probability matrix is calculated as $e^{t Q}$, where t is the length of time, in units corresponding to the intensity parameter entries in Q. The author in your link performed such calculation with annual intensity rates and t = 1, and therefore the transition probability matrix in his example corresponds to a one year period. The mathematics works out that if the row sums of Q equal zero (as they must), then the row sums of the transition probability matrix equal one, as is the case in the example in the link.

| cite | improve this answer | |
  • $\begingroup$ when you said "The mathematics works out that if the row sums of A equal zero (as they must)", you meant the generator matrix "Q"? $\endgroup$ – José Vallejo Apr 26 '16 at 22:11
  • $\begingroup$ @José Vallejo , Yes, Q. Thanks for catching that typo. I just fixed it. $\endgroup$ – Mark L. Stone Apr 26 '16 at 22:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.