0
$\begingroup$

I am new on the subject... I am computing AIC in R:

fitM1 = lm(data$M1~data$BM)
logM1 = logLik(fitM1)
aicM1 = 2*(n+1) - 2*logM1

First I was very concerned with negative AIC values, but what concerns me now is the high differences among AIC values of different models. By comparing some models with different input parameters to a benchmark model, I have got very discrepant (negative) AIC values. For instance, for one example the AIC values for the models are:

-105885.1, -105121.2, -109740.6, -117007, -105858.8, -108601.9, -108856.9

With such values set it is not even possible to compute the Akaike weight (wi).
Could it be correct?

$\endgroup$
3
  • 1
    $\begingroup$ You could do AIC(fitM1) to get the AIC value directly. $\endgroup$ – Richard Hardy Aug 12 '16 at 13:34
  • $\begingroup$ Are you worried that the difference between the AIC for one model and another is large? $\endgroup$ – gung - Reinstate Monica Aug 12 '16 at 13:36
  • $\begingroup$ Yes, using AIC(fitM1) returns roughly the same values. It concerns me because I can not even compute wi (it goes to infinity). And until now I have not seen such differences in the papers I have found. Thus, as I am new on the subject I am not so keen. $\endgroup$ – Marcos Alex Aug 12 '16 at 13:44
5
$\begingroup$

1) There is nothing wrong with negative AIC values. AIC values are arbitrary; even an AIC of 0 isn't directly interpretable.

2) There is nothing wrong with large differences in AIC values between models. Indeed, this is often what you are looking for with AIC to begin with. Without more details of what exactly your models are it is difficult to be more specific. If you are really concerned about wildly shifting AIC values between models, you may want to take a look at the assumptions of your models themselves. Maybe you have variables with very different ranges and should consider standardization, maybe you have misspecified the functional relationship between outcome and predictors, etc.

3) Why can't you compute the Akaike weight? The weight is just the value exp(-0.5*$\Delta$AIC) averaged across all pairwise model comparisons (where $\Delta$AIC is just the difference between two AIC values).

EDIT: In light of your comment on the OP, you say the weight is going to infinity. My answer above actually contains an error, the weight is not that value averaged across all pairwise model comparisons. The weight is the average of those values where $\Delta$AIC is the difference between each model and the SMALLEST AIC in the group of models of interest. So, if:

$w_i(AIC) = \frac{exp\{-0.5*\Delta_i\text{AIC}\}}{\sum_k^K{exp\{-0.5*\Delta_k\text{AIC}\}}}$

Then:

> AIC <- c(-105885.1, -105121.2, -109740.6, -117007, -105858.8, -108601.9, -108856.9)
> delta <- x-min(x)
> denominator <- sum(exp(-0.5*delta))
> weights <- exp(-0.5*delta)/denominator
> weights
[1] 0 0 0 1 0 0 0

The weights aren't being driven towards infinity, but towards 0. Even so, this doesn't seem to be a huge issue (any more so than having large pairwise AIC differences, anyway). Weights are just a way of comparing relative likelihoods between models; if the differences in likelihood (and thus AIC) between models are large, the weights will trend towards 0 and 1. In any case, the weights just give you the same interpretation as choosing the smallest AIC. Honestly, weights would only seem to be useful to begin with in the case where the differences between AIC values for all the models are relatively small, anyway.

$\endgroup$
3
  • $\begingroup$ Just a heads-up: regarding (3), see the comment by the OP under the OP. $\endgroup$ – Richard Hardy Aug 12 '16 at 13:49
  • $\begingroup$ Thanks, @RichardHardy. That must have been posted while I was writing my answer. $\endgroup$ – Ryan Simmons Aug 12 '16 at 13:51
  • $\begingroup$ (3) it is because the $exp(-0.5\Delta)$ tends to zero. Regarding (2), the outcome variable varies considerably. It varies with depth, generally with maximum at the surface and minimum at the bottom. This pattern follows for several days. I am comparing the models for the whole dataset. Calculating now AIC for just one depth, AIC values are (absolute) much lower, around -2000, and $\Delta AIC$ much closer. $\endgroup$ – Marcos Alex Aug 12 '16 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.