# First-order Discrete Markov Chain with time lag

I want to estimate the first-order transition matrix of a sequence in discrete time, e.g.

$$s = 1,0,1,0,1,1,0,1,0,0, \dots$$

but states are not evenly spaced in time. So that even if $s_{t=1} = 1$ and $s_{t = 2} = 0$, and $s_{t=3} = 1$ and $s_{t = 4} = 0$, the time lag between $t_{1}$ and $t_{2}$ is one day, whereas the time lag between $t_{3}$ and $t_{4}$ is one week.

How can I estimate the correct transition matrix accounting for the time lag between states? References are well accepted.

• If the time intervals are generally of one unit (day) and sometimes a multiple of the unit (e.g. 2 days, 7 days), then just replace the transition matrix, say $\mathbf{P}$, by a suitable power of it (e.g. $\mathbf{P}^2$, $\mathbf{P}^7$). It the time design is more complex with no clear unit, then use a Continuous Time Markov Chain model. – Yves May 13 '15 at 12:10

Taking "one day" as your time unit, consider the complete sequence $x_1,\dots,x_n$ and sum over the "missing data" days for which you don't have an observation. More specifically, if you have only information about $k$ days $I=\{i_i,\dots,i_k\}\subset\{1,\dots,n\}$, then, defining $I^c=\{1,\dots,n\}\setminus\{i_i,\dots,i_k\}$, and using the marginal $$p(x_{i_1},\dots,x_{i_k}) = \sum_{i\in I^c}\sum_{x_i\in\{0,1\}}\left(p(x_1)\prod_{j=1}^n p(x_j\mid x_{j-1})\right),$$ you have the likelihood of the transition matrix elements based only on the observed days. Maximization of this (in general intractable) likelihood looks like a job for the EM algorithm.