8
$\begingroup$

The formula for the AICc is:

AICc = AIC - 2k(k+1) / (n-k-1)

where k is the number of parameters and n the number of samples.

Is it somehow possible to calculate the AICc for n=k+1? Why does the formula not allow calculation of AICc in such case?

Thanks.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
9
$\begingroup$

The case $n = k + 1$ corresponds to a saturated model,

$$ \# \textrm{parameters} = \# \textrm{observations} $$ which is why you are seeing effectively an "infinite" penalisation.

One of the contexts in which Akaike's Information Criterion along with a host of others were developed, and is used frequently today, is linear regression. It's not always clear when the intercept or noise variance are counted or not, hence the "off by one" confusion.

Reference:

Two different formulas for AICc

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ The context in which AIC has originally been developed (i.e. Akaike's papers from 1972 and 1974) in the maximum likelihood (ML) framework and information theory, not linear regression. For some specific assumptions and choices, the ML estimator reduces to linear regression and hence formulas for AIC can be cast into a regression framework, which is what many people are interested in and that some references are not that clear in which parameters they include in their "k" count (most likely the noise variance is omitted, rather than an intercept). $\endgroup$
    – Egon
    Commented Nov 29, 2016 at 18:25
  • $\begingroup$ @Egon Fair enough, but since the topic is corrected AIC, even Sugiura's 1978 paper explicitly gives the formula in the case of linear regression immediately after the general definition. The correction is also motivated by a logit regression. Thanks for the clarification re: noise variance. $\endgroup$
    – Maximilian Aigner
    Commented Nov 29, 2016 at 20:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.