3
$\begingroup$

AIC values (from a fitted model, for example) are positive. So are the likelihood values. Are the log-likelihood values positive or negative? Here, in Wikipedia page concerning likelihood ratio test the log-likelihood values are negative and the less negative value indicates better fit.

But in this page, there is -(log-likelihood) (meaning negative of the log-likelihood) and it says that more negative value indicates better fit. In R the logLik-function from a model gives negative values, but if I have only the likelihood and I want a log-likelihood from it, I have to take negative of the logarithm???

So my question is fairly simple, when comparing those values (AIC, likelihood, logarithmic likelihood) from 2 different models, which (of which kind, more or less negative) value indicates better model?

Here is also some statistic as an example, could some one "translate" this info as words for me =) (I am using msts and tbats from forecast-function)

> fit1$likelihood
   [1] 90871.47
> fit2$likelihood
[1] 90785.92
> fit1$AIC
[1] 90909.47
> fit2$AIC
[1] 90839.92
# AIC from likelihood, par1 refers to number of fitted parameters
> 2*par1-2*log(fit1$likelihood) 
[1] -14.8344
> 2*par2-2*log(fit2$likelihood) # AIC from likelihood
[1] -10.83252

So why I wont get the same AIC values when calculating "by hand"? Which one are the correct ones?

|cite|improve this question
$\endgroup$
  • $\begingroup$ AIC can be negative. $\endgroup$ – Firebug Dec 9 '16 at 12:07
  • $\begingroup$ okay, I didn't know that. But still, is bigger AIC value then indicating better model or how can I compare the values? $\endgroup$ – praseodyymi Dec 9 '16 at 12:09
  • 1
    $\begingroup$ The smallest AIC is the best. $\endgroup$ – Firebug Dec 9 '16 at 12:11
  • $\begingroup$ Ah, that explains a lot $\endgroup$ – praseodyymi Dec 9 '16 at 12:12
  • 2
    $\begingroup$ I think the confusion may be due to thinking of the log-likelihood when you are really looking at the log of the likelihood ratio (likelihood for model1/likelihood for model2). So depending on which is the better model the likelihood ratio can be > 1 or < 1.hence the log-likelihood ratio can be .> 0 or < 0 respectively. $\endgroup$ – Michael Chernick Dec 9 '16 at 12:27
4
$\begingroup$

AIC will be negative whenever $k<\ln(L)$ (see this question and answers Negative values for AIC in General Mixed Model). Also keep in mind the log-likelihood can be positive, since the likelihood function is usually a probability density, i.e. it's always positive and can exceed $1$.

The interpretation, when comparing models, is the same though. The smallest AIC provides the "best" model, in the sense that it tends towards models with small $k$ and large $\ln{(L)}$.

Regarding likelihood, the higher the "better". Also, as log-likelihood is a monotonic transformation of likelihood, the same applies. This makes sense, since we want to minimize $\text {AIC}$:

$$\text {AIC} = 2k -2\ln(L)$$

Regarding your example in R, I'm pretty sure fit1$likelihood and fit2$likelihood are actually two times the negative log-likelihoods. The reason is simple:

fit1$likelihood - fit1$AIC
#[1] -38
fit2$likelihood - fit2$AIC
#[1] -54

If the AIC calculation is right, this means fit1 has 19 parameters and fit2 has 27 parameters.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ A minor point, the likelihood is always positive. $\endgroup$ – Michael Chernick Dec 9 '16 at 12:32
  • $\begingroup$ @MichaelChernick True, will state explicitly. $\endgroup$ – Firebug Dec 9 '16 at 12:33
  • $\begingroup$ I added an example to my question, but thanks, this was also helpful $\endgroup$ – praseodyymi Dec 9 '16 at 12:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.