Skip to main content
2 of 2
added 27 characters in body
jmb
  • 744
  • 1
  • 6
  • 17

Compute likelihood of mixture distribution while avoiding floating point problems

I have a mixture distribution with likelihood function

$$ L(\theta) = \prod_{i=1}^N \sum_{k=1}^K f(X_i|\theta_k) \lambda_k $$

where $N$ is the sample size, $K$ is the number of component, $\theta_k$ is the parameter vector of component $k$, and $\lambda_k$ is the mixing probability of component $k$.

If I take the log of this I get

$$ \log \left[ L(\theta) \right] = \sum_{i=1}^N \log \left[ \sum_{k=1}^K f(X_i|\theta_k) \lambda_k \right] $$

My question is how do I evaluate the log-term for given $X$, $\theta$ and $\lambda$? Theoretically, I could compute the likelihoods $f(X_i|\theta_k)$ and sum them up weighted by the $\lambda_k$ and after take the log. However, I need to work with the log likelihoods to avoid floating point problems. My question therefore is how to compute the log term in the log likelihood above, if I can only compute $\log \left[ f(X_i|\theta_k) \right]$.

jmb
  • 744
  • 1
  • 6
  • 17