# Calculating log-likelihood for given MLE (Markov Chains)

I am currently working with Markov chains and calculated the Maximum Likelihood Estimate using transition probabilities as suggested by several sources (i.e., number of transitions from a to b divided by number of overall transitions from a to other nodes).

I now want to calculate the log-likelihood of the MLE.

• You've already computed the maximum likelihood estimate of the transition probabilities and now you want to compute the log-likelihood of what exactly? – Nick Jan 14 '13 at 15:14

Let $\{ X_i \}_{i=1}^{T}$ be a path of the markov chain and let $P_{\theta}(X_1, ..., X_T)$ be the probability of observing the path when $\theta$ is the true parameter value (a.k.a. the likelihood function for $\theta$). Using the definition of conditional probability, we know

$$P_{\theta}(X_1, ..., X_T) = P_{\theta}(X_T | X_{T-1}, ..., X_1) \cdot P_{\theta}(X_1, ..., X_{T-1})$$

Since this is a markov chain, we know that $P_{\theta}(X_T | X_{T-1}, ..., X_1) = P_{\theta}(X_T | X_{T-1} )$, so this simplifies this to

$$P_{\theta}(X_1, ..., X_T) = P_{\theta}(X_T | X_{T-1}) \cdot P_{\theta}(X_1, ..., X_{T-1})$$

Now if you repeat this same logic $T$ times, you get

$$P_{\theta}(X_1, ..., X_T) = \prod_{i=1}^{T} P_{\theta}(X_i | X_{i-1} )$$

where $X_0$ is to be interpreted as the initial state of the process. The terms on the right hand side are just elements of the transition matrix. Since it was the log-likelihood you requested, the final answer is:

$${\bf L}(\theta) = \sum_{i=1}^{T} \log \Big( P_{\theta}(X_i | X_{i-1} ) \Big)$$

This is the likelihood of a single markov chain - if your data set includes several (independent) markov chains then the full likelihood will be a sum of terms of this form.

• Wow, thanks a lot for the answer. In this case $P_{\theta}$ is the "transition" probability taken from the MLE, right? – fsociety Jan 14 '13 at 15:42
• @ph_singer, you are very welcome. $P_{\theta}(X_i|X_{i-1})$ is the probability of moving from state $X_{i-1}$ to $X_i$, given the parameter value, $\theta$. If you imposed no structure on the transition matrix (which is what it sounds like) then $\theta$ just denotes the vector of transition probabilities (and the MLEs are just the sample proportions, as you correctly indicated in your question statement), so, yes: $P_{\hat{\theta}_{{\rm MLE}}}(X_i|X_{i-1})$ will just be the sample proportion of moves from state $X_{i-1}$ that ended up in state $X_{i}$. – Macro Jan 14 '13 at 15:48
• Thanks again! Just one more question: If I use another order (e.g., k=2), how would this process work then? – fsociety Jan 14 '13 at 16:56
• Can you please clarify what you mean by "order"? – Macro Jan 14 '13 at 18:07
• (+1) The OP probably means $k=2$ to denote a second-order MC, i.e., depending on the previous two states $X_{i-1},X_{i-2}$ rather than just the most recent one $X_{i-1}$. – cardinal Jan 14 '13 at 18:14