I have a matrix for data that (supposedly) follows a Markov process with an absorbing state; I have 3 possible states and 50 periods (discrete states, discrete time). Element [t,s] of the matrix tells me how much of my population is in state s at time t. Something like:

90 10 0
80 10 10

The question is: how can I estimate the transition matrix?

I use Python but might use R or Julia for this - or I'd be happy to consider converting an algorithm to Python if not too complex.

Note that I only have this matrix as described - I do not have the underlying individual observations; in other words, I do not know which item went from which state to which state - only the total number of items in each state at each time.

The answers/comments I have found refer to cases where you know the underlying observations, which I don't in this case. I understand there is a package for R: https://cran.mtu.edu/web/packages/markovchain/ But, if I understood it correctly, it requires the actual observations, and cannot really be used with my kind of data - is that right?

I have put together a few lines of Python to simulate the kind of data I want to estimate.

Thoughts / ideas / suggestions? Thanks!

import numpy as np
import pandas as pd

# the initial state

# One matrix is applied in 15% of the cases; I do this to introduce
# some 'noise'
m1=np.array([[0.91,0.09,0 ], [0.3,0.4,0.3],[0,0,1] ])
m2=np.array([[0.9,0.1,0 ], [0.32,0.4,0.28],[0,0,1] ])

periods = 50
ran = np.random.rand(periods)

evol= np.zeros((periods,3))

for t in np.arange(1, periods):
    if ran[t]<0.15:
        trans = m1
        trans = m2
    evol[t,:]=np.matmul( evol[t-1,:], trans )
  • $\begingroup$ Can you better explain the first few lines of your question. What are you trying to achieve ? $\endgroup$ – rgk Mar 14 at 17:06
  • $\begingroup$ Is the structure of the data I have clear? Like I said, I am trying to estimate the transition matrix. Let me try to rephrase. Let's suppose I have data on the medical status of some patients; there are 3 states: healthy, sick and dead. Dead is an absorbing state (obviously!). I do not have the individual observations that tell me that, say, patient 1 went from HHHHHHSSSSD while patient 2 went from SSHHHHHHSSHHHD All I have is, for each period, the total count of my population broken down among these 3 states. So I know that, at time 0, I have 90 healthy, 10 sick and 0 dead, etc. $\endgroup$ – Pythonista anonymous Mar 14 at 17:58
  • $\begingroup$ In light of all this, how can one estimate the transition matrix? $\endgroup$ – Pythonista anonymous Mar 14 at 18:00
  • $\begingroup$ Thank you for your comments. As far as I understand, since there is an absorbing state in your problem, the markov chain is not ergodic which means there is no n-step transition probability matrix. $\endgroup$ – rgk Mar 14 at 22:01
  • $\begingroup$ I'm not sure I am following. Yes, the chain is not ergodic, but how does this affect whether/how the transition matrix can be estimated? $\endgroup$ – Pythonista anonymous Mar 14 at 23:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.