3
$\begingroup$

In looking at the formula for the AIC=-2*(LL)-2k and the formula for log likelihood, LL=-n/2*log(2*pi) - n/2*log(sse/n) - n/2, I notice that the term with sse is sensitive to the scale of the dependent variable but the second term is not.

This seems to mean there might be a case where changing measuring units of the dependent variable from, say, kilograms to grams would cause the term (sse/n) to increase while the other terms would remain constant. Depending on the values of the other terms, this change of measuring units could potentially change which model has the lowest AIC.

I have two questions: First, is my reasoning mistaken? Second, assuming I have not made a mistake, how should someone use the AIC to select a model when the result is sensitive to choice of measuring units?

$\endgroup$
  • 1
    $\begingroup$ Usually you are examining the likelihood ratio, so scale changes should not affect this. $\endgroup$ – user75138 Jun 30 '15 at 2:53
  • 1
    $\begingroup$ This is a good argument showing why AIC is meaningless in and of itself: it can only be used in (additive) comparison to another (related) AIC. $\endgroup$ – whuber Jun 30 '15 at 13:13
  • 1
    $\begingroup$ Is AIC even useful in comparisons? I think it would be simple to construct a numerical example where I have some variables to predict how far a runner can run in an hour and the subset of the variables that generates the lowest AIC would depend on whether the distance travelled is measured in meters or km. If that's really the case, then AIC based feature selection would seem to fall under the category of "not even wrong." $\endgroup$ – user4945913 Jul 1 '15 at 2:09
  • $\begingroup$ You could try to construct that example and see what you get! $\endgroup$ – kjetil b halvorsen Mar 5 '17 at 18:12
1
$\begingroup$

Relative AIC is independent of the scale used for the data. Say, $c$ is the scaling factor. The sse term would become $$ \frac{n}{2} \log\left(\frac{c\times \textrm{sse}}{n}\right) = \frac{n}{2} \log\left(c\right) + \frac{n}{2} \log\left(\frac{\textrm{sse}}{n}\right). $$ The first term on the right is clearly a constant. Hence, it doesn't matter as long as you are comparing models trained on the same data-set with the same number of data points.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.