# How could I estimate a transition probability matrix that varies over time?

I have multiple Markov chains with twelve states. I want to estimate a transition probability matrix for each time point (except for the last time point) that can vary over time using all Markov chains. I found a function in R to do this. It is in the TraMineR package and is called seqtrate. However, it isn't clear to me how they estimate a transition probability matrix that can vary over time for each time point.

The transition probability matrix their function infers would in this case have three dimensions, [states,states,time], where the first two dimensions would correspond to a transition probability matrix.

You can use a multinomial regression model (using previous state and time as independent variables), and you can use any typical regression modelling techniques such as regularization, feature engineering...

• Are there any resources you can refer me to that could show me how to do this? I'm attempting this, but I'm having some trouble Commented Mar 30, 2023 at 20:49
• Never mind! I think I got the hang of it Commented Mar 30, 2023 at 23:15
• I'm not sure if I should post this as a question, but in this case time is discrete. So, would you include time in your multinomial regression model as an independent variable that the model regards as an integer rather than as a factor? Commented Apr 7, 2023 at 1:21
• yes I would add time as a real variable ( but its up to you if eg you want to add interactions etc.).look up discrete time survival models ( this is single state) and discrete time multistate models Commented Apr 7, 2023 at 5:10

If the transition probability matrix varies over time then your stochastic process is not a Markov chain (i.e., it does not obey the Markov property). In order to estimate transition probabilities at each time you would need to make some structural assumptions about how these transition probabilities can change (e.g., how rapidly they can change, etc.). Without any structural assumptions, the MLE for the transition probability matrix at each time period will estimate a probability of one for the transition that actually occurred and a probability of zero for all other transitions, which is not very helpful.

If you have a look at the documentation for the seqtrate function, you will see that it references Gabadinho et al (2011). This paper gives further information on the analysis of state-sequence objects using the TraMineR package. It includes discussion of the transition rates and the "turbulence" of transitions in the sequence. The paper contains references to relevant mathematical and statistical literature that discusses this type of analysis, so you might need to do a bit of a deep-dive to learn the relevant models and methods for this type of analysis.

• Thank you for your response. It is helpful. I just read through Gabadinho et al (2011). Unfortunately, they don't describe how they estimate time-varying transition probability/rate. When I apply the function, however, my output includes values between 0 and 1 (e.g., 0.3,0.4, ect.). So, its output appears to be helpful. I may look through the authors' references, email them, or just implement a different method for estimating time-varying transition probabilities, a method whose assumptions I can control. Commented Mar 21, 2023 at 5:04
• Could the method they use be a generalization of the hazard function for time to a single type of event? It's a function that comes from survival analysis. Commented Mar 21, 2023 at 16:45
• This is the only information the authors of Gabadinho et al (2011) include when describing how they estimate time-varying transition probabilities: "Time-varying transition rates can be obtained with option time.varying=TRUE, in which case a 3-dimensional array with a distinct transition rate matrix for each of the positions t = 1, 2, . . . , L − 1 is returned. The matrix for position t is computed by considering only the states at t and t + 1. The third dimension of the array corresponds to the position t index." Commented Mar 21, 2023 at 17:09
• I recommend reading back into the other literature they cite to see if you can find the relevant models, etc. there. Failing that, contact them and ask for more information.
– Ben
Commented Mar 21, 2023 at 21:06
• Sounds like a good plan. Thank you for the help Commented Mar 22, 2023 at 0:37