I'm trying to find the asymptotic covariance matrix $\Sigma$ for the MLE of a Markov Chain. Let $S$ be the states of the chain. If $\mathbf{y}$ is a path of the Markov Chain and $n_{ij}$ are the number of times observed moving from state $i$ to state $j$, I see that
(1) $p(\mathbf{y}|P,y_0)=\prod_{i\in S} \prod_{j\in S}p_{ij}^{n_{ij}}$
Furthermore, I understand how
(2) $\hat{p}_{ij}=\frac{n_{ij}}{\sum_k^m n_{ik}}$,
So I put these two together in the formula for covariance:
(3) $\int_\Omega (\hat{\theta}-\theta)(\hat{\theta}-\theta)^T dp(\mathbf{y}|\theta)$.
(where $\theta$ is just a vectorized version of my transition probabilities).

However, am I summing probabilities over all possible paths?
(4) $\Omega:=\{\mathbf{y}:y_0=i_0, \mathbf{y}\in S^\infty\}$,
Or am I summing over all possible values of $n_{ij}$, i.e.
(5) $\Omega:=\{n_{ij}\in\mathbb{Z_+}\}$?

 My intuition tells me that these are equivalent in the asymptotic case, but i can't seem to strike the argument. Something about how an infinite realization of the chain contains all possible finite realizations of the chain as subsequences, but I'd like to be formal about it.

 I'd also like some insight into how to determine $\Omega$ for a finite sample (i.e. not the asymptotic case). Perhaps there needs to be a counting coefficient since there may be multiple paths that follow the same steps and yeild the same $\theta$, but in a different order?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.