4
$\begingroup$

I have a mixture data. I used EM to estimate the model parameters. When I calculate the log likelihood function, I found that the values is positive. So, is that ok. Can the log likelihood function be positive?

To be more clear:

$ll = \sum_{n=1}^{N}\log(\sum_{k=1}^{K} \pi_{k} f(x_n;\theta_{k}))$

$\endgroup$
  • $\begingroup$ To my understanding, likelihood function is usually a product of probabilities and log-likelihood is a sum of logs. Because probabilities are less than 1, theirs logs should be less than 0. And because you sum those logs, you should get a negative number. $\endgroup$ – Celdor Dec 21 '17 at 10:32
  • 4
    $\begingroup$ The comment above is wrong. The likelihood function of continuous parameters (such as your case, I guess) is based on probability densities, which can be greater than 1 depending on the domain and the density, so the log likelihood can occasionally be positive. Still, you should check that everything else is correct. $\endgroup$ – lacerbi Dec 21 '17 at 10:35
  • $\begingroup$ yes, my density is continuous. ' $\endgroup$ – Alice Dec 21 '17 at 10:37
8
$\begingroup$

Simply (just summarizing the comments):

  • when using probabilities (discrete outcome), the log likelihood is the sum of logs of probabilities all smaller than 1, thus it is always negative
  • when using probability densities (continuous outcome), the log likelihood is the sum of logs of densities that can be greater than 1, thus is can be positive.
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ If all the discrete probability is in a single outcome, log-likelihood would be 0 rather than negative. $\endgroup$ – Glen_b Sep 3 at 5:41

Not the answer you're looking for? Browse other questions tagged or ask your own question.