5
$\begingroup$

I have a situation where data from the following process is observed:

For $i = 1, \dots, n$ let $(X_{i,1}, \dots, X_{i,m_i})$ be a sequence of $m_i$ random variables coming from a discrete-space Markov chain with $N \times N$ transition matrix $P$. For simplicity's sake I'll assume that each $X_{ij} \in \{1, \dots, N\}$. For $i \neq i'$ assume that the chains $(X_{i,1}, \dots, X_{i,m_i})$ and $(X_{i',1}, \dots, X_{i',m_{i'}})$ are independent of each other and both have the same transition matrix $P$. In general $m_i \neq m_{i'}$.

A big complication: $m_i$ is really small. The largest $m_i$ is 9, the smallest is 3, and about 95% of the $m_i = 5$. I have around 50 of these chains (i.e. $n = 50$).

For a single chain the MLE $\hat P_i$ of $P$ is easy to compute: it's just the proportions of times that each transition was observed.

My question: how to get a single estimate of $P$ from these $\hat P_i$? Each $\hat P_i$ estimates the same thing, and due to differences in $m_i$ some do better than others. But because each $m_i$ is small, none of them do very well. There are lots of 0's and 1's which I don't think reflect reality. What should I do here?

$\endgroup$
5
  • $\begingroup$ I wonder whether there is any useful information in the starting states of each chain. (For instance, if these mini-chains are random segments selected from a single larger chain, the distribution of starting states might reflect a stationary distribution.) Could you tell us how those starting states were determined? $\endgroup$
    – whuber
    May 11, 2016 at 22:17
  • $\begingroup$ @whuber Thanks for the comment. Each chain corresponds to a totally separate process which was started and stopped; I don't know much about why the initial values for each chain happened to be what they are, but they are certainly random. Overall, I really expected this to be a well-studied problem with a tidy answer. I'm surprised that that doesn't seem to be the case. $\endgroup$
    – alfalfa
    May 12, 2016 at 1:47
  • 2
    $\begingroup$ Your problem appears to be no different than estimating the transition probabilities in a single chain. $\endgroup$
    – whuber
    May 12, 2016 at 4:44
  • $\begingroup$ @whuber Can you elaborate on that? If I observed chain 1 to be (0,1,1,0,1) and chain 2 to be (1,1,0,1), should I just estimate $P$ by counting all of the $(i,j)$ for $i,j=0,1$, without paying attention to which chain a given $(i,j)$ comes from? $\endgroup$
    – alfalfa
    May 12, 2016 at 13:36
  • 3
    $\begingroup$ According to all the information you have provided, the answer is of course. You have observed three independent transitions from the $0$ state, governed by its Binomial distribution, and four independent transitions from the $1$ state, governed by its (possibly different) Binomial distribution. That reduces the question to estimating Binomial probabilities. Why should it matter that the transitions occurred within different chains? $\endgroup$
    – whuber
    May 12, 2016 at 13:41

1 Answer 1

4
$\begingroup$

Just so that this question has an answer, I'm going to try to summarize @whuber's comments here:

Basically, because these chains are independent and have the same transition matrix $P$, I can just count the number of instances of each possible transition and get the MLE of $P$ from that. It doesn't matter that these counts come from different Markov chains.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.