I do some optimization problem in R. I minimize the loglikelihood function. I found that the log-likelihood has a negative value. For example, I have this: -34.5. Then, when I count the AIC, I will get, a positive value for AIC. My question, Is it ok to have negative loglikelihood values (I am working with continuous data).
$\begingroup$
$\endgroup$
3
-
$\begingroup$ Is this probably a duplicate? I would not be surprised to find that the question has been asked before. $\endgroup$– Richard HardyCommented May 17, 2018 at 10:40
-
$\begingroup$ It's certainly a duplicate. $\endgroup$– Glen_bCommented May 17, 2018 at 13:53
-
$\begingroup$ For example, stats.stackexchange.com/questions/172846/… (the fact that it's marginal changes nothing on this issue); that one is closed in turn because its a duplicate of the oft-asked question about densities exceeding 1. That one is probably sufficient but I think there's better ones to be had. $\endgroup$– Glen_bCommented May 17, 2018 at 15:08
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
The likelihood function is defined as
$$ \mathcal{L}(\theta|X) = \prod_{i=1}^n f_\theta(X_i) $$
and is a product of probability mass functions (discrete variables) or probability density functions (continuous variables) $f_\theta$ parametrized by $\theta$ and evaluated at the $X_i$ points.
Probability densities are non-negative, while probabilities also are less or equal to one. It follows that their product cannot be negative. The natural logarithm function is negative for values less than one and positive for values greater than one. So yes, it is possible that you end up with a negative value for log-likelihood (for discrete variables it will always be so).
-
$\begingroup$ Thank you so much for your great answer. I am working on a project and I would like to clearly understand it. Could you advise me on some peer-review paper or a book about this, please? this will be very helpful to discuss it with my supervisor. $\endgroup$– MaryamCommented May 17, 2018 at 10:26
-
$\begingroup$ @Maryam this should be explained in any mathematical statistics handbook, since follows from the definitions. $\endgroup$– TimCommented May 17, 2018 at 10:29
-
1$\begingroup$ In principle any probability or probability density being 0 means that the likelihood is 0 and the log-likelihood is indeterminate. In principle also saying that the likelihood is zero amounts to saying that some observed data are impossible and that surely means a misspecified model. So in practice only positive values for $f$ enter your equation. $\endgroup$– Nick CoxCommented May 17, 2018 at 12:35