11
$\begingroup$

I am currently working with Markov chains and calculated the Maximum Likelihood Estimate using transition probabilities as suggested by several sources (i.e., number of transitions from a to b divided by number of overall transitions from a to other nodes).

I now want to calculate the log-likelihood of the MLE.

Improve this question
$\endgroup$
1
  • $\begingroup$ You've already computed the maximum likelihood estimate of the transition probabilities and now you want to compute the log-likelihood of what exactly? $\endgroup$
    – Nick
    Jan 14, 2013 at 15:14

1 Answer 1

Reset to default
15
$\begingroup$

Let $ \{ X_i \}_{i=1}^{T}$ be a path of the markov chain and let $P_{\theta}(X_1, ..., X_T)$ be the probability of observing the path when $\theta$ is the true parameter value (a.k.a. the likelihood function for $\theta$). Using the definition of conditional probability, we know

$$ P_{\theta}(X_1, ..., X_T) = P_{\theta}(X_T | X_{T-1}, ..., X_1) \cdot P_{\theta}(X_1, ..., X_{T-1})$$

Since this is a markov chain, we know that $P_{\theta}(X_T | X_{T-1}, ..., X_1) = P_{\theta}(X_T | X_{T-1} )$, so this simplifies this to

$$ P_{\theta}(X_1, ..., X_T) = P_{\theta}(X_T | X_{T-1}) \cdot P_{\theta}(X_1, ..., X_{T-1})$$

Now if you repeat this same logic $T$ times, you get

$$ P_{\theta}(X_1, ..., X_T) = \prod_{i=1}^{T} P_{\theta}(X_i | X_{i-1} ) $$

where $X_0$ is to be interpreted as the initial state of the process. The terms on the right hand side are just elements of the transition matrix. Since it was the log-likelihood you requested, the final answer is:

$$ {\bf L}(\theta) = \sum_{i=1}^{T} \log \Big( P_{\theta}(X_i | X_{i-1} ) \Big) $$

This is the likelihood of a single markov chain - if your data set includes several (independent) markov chains then the full likelihood will be a sum of terms of this form.

Improve this answer
$\endgroup$
14
  • $\begingroup$ Wow, thanks a lot for the answer. In this case $P_{\theta}$ is the "transition" probability taken from the MLE, right? $\endgroup$
    – fsociety
    Jan 14, 2013 at 15:42
  • $\begingroup$ @ph_singer, you are very welcome. $P_{\theta}(X_i|X_{i-1})$ is the probability of moving from state $X_{i-1}$ to $X_i$, given the parameter value, $\theta$. If you imposed no structure on the transition matrix (which is what it sounds like) then $\theta$ just denotes the vector of transition probabilities (and the MLEs are just the sample proportions, as you correctly indicated in your question statement), so, yes: $P_{\hat{\theta}_{{\rm MLE}}}(X_i|X_{i-1})$ will just be the sample proportion of moves from state $X_{i-1}$ that ended up in state $X_{i}$. $\endgroup$
    – Macro
    Jan 14, 2013 at 15:48
  • $\begingroup$ Thanks again! Just one more question: If I use another order (e.g., k=2), how would this process work then? $\endgroup$
    – fsociety
    Jan 14, 2013 at 16:56
  • $\begingroup$ Can you please clarify what you mean by "order"? $\endgroup$
    – Macro
    Jan 14, 2013 at 18:07
  • $\begingroup$ (+1) The OP probably means $k=2$ to denote a second-order MC, i.e., depending on the previous two states $X_{i-1},X_{i-2}$ rather than just the most recent one $X_{i-1}$. $\endgroup$
    – cardinal
    Jan 14, 2013 at 18:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.