For a given panel dataset I have used the MSM package to estimate transition probabilities between states. Using the pnext.msm function I can obtain the probabilities of getting from a given state $r$ to a state $s$. However, I am also interested in the transition probability of going to $r$ from $r$. The help page for pnext.msm states that

For a continuous-time Markov process in state r, the probability that the next state is s is $-q_{rs} / q_{rr}$, where $q_{rs}$ is the transition intensity (qmatrix.msm).

This seems not well-defined when $r = s$.

I know that I can obtain the transition probabilities for a given time window using the pmatrix.msm function, but then the question is what time window I should pick so that I can extract the right $r \rightarrow r$ transition probabilities.

How can I extract the right probability of going from any state $r$ to itself?


Short Answer

Use the Kolmogorow-Smirnov test to identify a time window such that the eigenvalues of the Transition matrix is not sensitive to the precise choice of window.

Longer Answer

In Markov theory, there are two ways of formulating the dynamics.

  1. Continuous time: We talk about transition rates in units of inverse time. So $Q_{rs}$ is the number of transitions from $r$ to $s$ expected per unit time (e.g. 5 transitions per second).
  2. Discrete time. Here we talk about the probability of transitioning from one state to another in some specified interval of time. So $T_{rs}(\delta t)$ would be the probability to be in state $s$ at time $t+\delta t$ given that we were in state $r$ at time $t$.

One can show that these two formulations are related through a matrix exponential.

\begin{equation} T = e^{Q\delta t} \end{equation}

Continuous Time Being Well Defined

So regarding your first point regarding self transition being well-defined:

  • The continuous time formulation requires that the diagonal elements $Q_{rr}$ are equal to the negative sum of all the other transitions ($-\sum_{i\neq r}Q_{ri}$). You can read more about this here.
  • The pnext.msm quantity gives the probability of which state will come next. This is achieved by the equation in your question which just calculates the relative probabilities of transitions.
  • Notice that since the diagonal is the negative sum, the equation becomes: \begin{equation} \frac{-Q_{rs}}{Q_{rr}}=\frac{-Q_{rs}}{-\sum_{i\neq r}Q_{ri}}=\frac{Q_{rs}}{\sum_{i\neq r}Q_{ri}} \end{equation}
  • So now it is clearly just a relative likelihood!
  • Since the continuous time approach asks "Which state comes next?", the answer to that question can never be the current state.

Discrete Time Choice of Time Window

Now, let's say we want to look at discrete time and talk about "Which state will we be in 1 second from now?". Then we can easily find our $T$ from $Q$ if we know what our $\delta t$ is.

How do we choose the time window? Well we want a window such that the process is memoryless (Markovian). Often real world processes are non-Markovian on short time scales.

To check for Markovianity, we compute the matrix at a range of windows and examine that the dominant eigenvalue $\lambda$ is insensitive to the precise choice of window.

\begin{equation} \frac{log(\lambda(\delta t))}{\delta t}\approx \frac{log(\lambda(2\delta t))}{2\delta t} \end{equation}

  • $\begingroup$ Thank you very much for your extensive answer. So if I understand you correctly I need to do the following: (i) find $\delta t$ such that the largest eigenvalue doesn't change much, (ii) compute the transition probability matrix $T_{rs}(\delta t) = \exp{Q\delta t}$ and present these. Is this correct? $\endgroup$
    – Stijn
    Oct 1 '21 at 14:11
  • $\begingroup$ The time window selection to ensure Markovianity (using my final equation) is if you are calculating your $T$ matrix directly from raw data. If you are just computing $T$ from your calculated $Q$ then Markovianity will be ensured by construction. $\endgroup$
    – Adam Kells
    Oct 1 '21 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.