6
$\begingroup$

What is the term $K$ in Akaike information criterion? The AIC is defined as $2K-2log(L)$, where $L$ is the maximized value of the likelihood function for the estimated model.

On the internet, I found three competing candidates:

  1. Number of parameters + error term (for simple linear one-predictor model, intercept, slope and error term: $K=3$)
  2. Number of parameters (for the linear one-predictor model, intercept and slope: $K=2$)
  3. Number of predictors (for the linear one-predictor model, the slope: $K=1$)

Which one is correct and why?

$\endgroup$
4
$\begingroup$

This is how the original 1974 paper by Hirotugu Akaike defines the AIC:

AIC = (-2)log(maximum likelihood) + 2(number of independently adjusted parameters within the model)

The error term is not a parameter which you're independently trying to adjust, but the intercept is (e.g. your slope might be zero and the data best fit by a horizontal line). The correct answer for your simple univariate regression is $K=2$ (intercept and slope).

$\endgroup$
  • $\begingroup$ "The error term is not a parameter which you're independently trying to adjust, but the intercept is (e.g. your data might best be fit by a horizontal line)." I think you mean "...fit by a line through the origin," since omitting the intercept term constrains the intercept to be zero, but yields a line with slope $\beta$. $\endgroup$ – Sycorax says Reinstate Monica Sep 26 '13 at 13:34
  • $\begingroup$ The point I was trying to make was that the intercept is an independently adjusted parameter. That is, you could have a slope of zero, and an intercept of, say, one. This would draw a straight line through the data. Or you can start with a slope $\beta$ and then adjust the intercept until you get the best fit. The point is that the intercept should be counted. $\endgroup$ – Cristian Dima Sep 26 '13 at 13:38
  • $\begingroup$ I agree that the intercept is an independently-adjusted parameter. The sentence seems to indicate that the intercept parameter determines whether or not the line is horizontal, which is not the case. $\endgroup$ – Sycorax says Reinstate Monica Sep 26 '13 at 13:47
  • $\begingroup$ Thanks for the quick answer. I was just wondering whether we must add a variance component to the estimated parameters (the first one above with error term perhaps implies that), bacause it has been counted in in some cases e.g. www4.ncsu.edu/~shu3/Presentation/AIC.pdf stats.stackexchange.com/questions/67786/… $\endgroup$ – Timo J. Marjomäki Oct 2 '13 at 8:36
  • $\begingroup$ So in the multivariate case, would we increment K by 1 for each column/attribute added to the regression model, after the initial one? This is a great answer, but I want to make sure I understand it clearly; asking a separate question would probably qualify as a duplicate. $\endgroup$ – SQLServerSteve May 20 '17 at 0:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.