4
$\begingroup$

I am ranking candidate models using Akaike information criterion (AIC). All my models have positive -2*LL (log likelihood) values which as far as I understand is expected under certain circumstances and not much of a problem...

...unless - and this is just me wondering - I want to rank models using AIC=-2*LL+(2k) with k being the number of parameters in the model. With large positive -2LL there appears little penalty for more complex models, which suddenly rank surprisingly high.

Can I still use AIC for model comparison?

Thanks heaps!!

$\endgroup$
4
$\begingroup$

You certainly may! AIC is not really an absolute measure, it's intended for model comparison, and is a relative measure.

For example, AIC in a normal linear model is driven by the variance of the errors. If the entire problem gets rescaled by, for example, dividing the target variable by 100, the AIC will change, but the relative magnitude of AIC with and without an extra variable will not:

> # Generate random example
> x <- rnorm(100)
> z <- rnorm(100)
> y <- x + rnorm(100)
> 
> # Base case
> AIC(lm(y~x), lm(y~x+z))
              df      AIC
lm(y ~ x)      3 290.7473
lm(y ~ x + z)  4 292.7220
>  
> # Rescale y
> AIC(lm(y/100~x), lm(y/100~x+z))
                  df       AIC
lm(y/100 ~ x)      3 -630.2867
lm(y/100 ~ x + z)  4 -628.3121
> 
> # Rescale y again
> AIC(lm(y*10000~x), lm(y*10000~x+z))
                      df      AIC
lm(y * 10000 ~ x)      3 2132.815
lm(y * 10000 ~ x + z)  4 2134.790

So your comparison comes out the same regardless of the -2LL part.

Edit:

Note also that the difference between the AICs in each of the three examples is exactly the same, except for rounding in the last digit. You can think of it as that, because we are using the log likelihood, the "scale" of the problem becomes an additive term, and just serves to move the log likelihood up and down the real number line; all the models move the same distance (because the term is additive) so preserve the relative differences between them. A loose explanation, admittedly.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ ...but what if I compare models with just 2 vs models with 5 or up to 10 parameters? Ideally we'd be after the simplest model - however, with large -2*LL values the penalty (2k) for more compex models is relatively small and hence those more complex models seem to rank higher than simple models?! $\endgroup$ – ulnberg Apr 24 '12 at 10:02
  • $\begingroup$ Assuming the small model is correct, well as correct as a model ever is, the -2*LL term will be roughly the same across all the models - even the more complex ones - because the added terms will likely have little impact on the likelihood, although for smaller samples chance will play an important role. So the 2k term will still be important. Note in the example that the difference between the AICs in the "base" case and the rescaled case is exactly the same. I'll alter the example to make this clearer and edit the answer a little too. $\endgroup$ – jbowman Apr 24 '12 at 13:59
  • $\begingroup$ Wow - thanks heaps! That's convincing. Much appreciated! $\endgroup$ – ulnberg Apr 25 '12 at 21:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.