Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am currently working with Markov chains and calculated the Maximum Likelihood Estimate using transition probabilities as suggested by several sources (i.e., number of transitions from a to b divided by number of overall transitions from a to other nodes).

I now want to calculate the log-likelihood of the MLE.

share|improve this question
You've already computed the maximum likelihood estimate of the transition probabilities and now you want to compute the log-likelihood of what exactly? – Nick Jan 14 '13 at 15:14
up vote 5 down vote accepted

Let $ \{ X_i \}_{i=1}^{T}$ be a path of the markov chain and let $P_{\theta}(X_1, ..., X_T)$ be the probability of observing the path when $\theta$ is the true parameter value (a.k.a. the likelihood function for $\theta$). Using the definition of conditional probability, we know

$$ P_{\theta}(X_1, ..., X_T) = P_{\theta}(X_T | X_{T-1}, ..., X_1) \cdot P_{\theta}(X_1, ..., X_{T-1})$$

Since this is a markov chain, we know that $P_{\theta}(X_T | X_{T-1}, ..., X_1) = P_{\theta}(X_T | X_{T-1} )$, so this simplifies this to

$$ P_{\theta}(X_1, ..., X_T) = P_{\theta}(X_T | X_{T-1}) \cdot P_{\theta}(X_1, ..., X_{T-1})$$

Now if you repeat this same logic $T$ times, you get

$$ P_{\theta}(X_1, ..., X_T) = \prod_{i=1}^{T} P_{\theta}(X_i | X_{i-1} ) $$

where $X_0$ is to be interpreted as the initial state of the process. The terms on the right hand side are just elements of the transition matrix. Since it was the log-likelihood you requested, the final answer is:

$$ {\bf L}(\theta) = \sum_{i=1}^{T} \log \Big( P_{\theta}(X_i | X_{i-1} ) \Big) $$

This is the likelihood of a single markov chain - if your data set includes several (independent) markov chains then the full likelihood will be a sum of terms of this form.

share|improve this answer
Wow, thanks a lot for the answer. In this case $P_{\theta}$ is the "transition" probability taken from the MLE, right? – fsociety Jan 14 '13 at 15:42
@ph_singer, you are very welcome. $P_{\theta}(X_i|X_{i-1})$ is the probability of moving from state $X_{i-1}$ to $X_i$, given the parameter value, $\theta$. If you imposed no structure on the transition matrix (which is what it sounds like) then $\theta$ just denotes the vector of transition probabilities (and the MLEs are just the sample proportions, as you correctly indicated in your question statement), so, yes: $P_{\hat{\theta}_{{\rm MLE}}}(X_i|X_{i-1})$ will just be the sample proportion of moves from state $X_{i-1}$ that ended up in state $X_{i}$. – Macro Jan 14 '13 at 15:48
Thanks again! Just one more question: If I use another order (e.g., k=2), how would this process work then? – fsociety Jan 14 '13 at 16:56
Can you please clarify what you mean by "order"? – Macro Jan 14 '13 at 18:07
(+1) The OP probably means $k=2$ to denote a second-order MC, i.e., depending on the previous two states $X_{i-1},X_{i-2}$ rather than just the most recent one $X_{i-1}$. – cardinal Jan 14 '13 at 18:14

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.