1
$\begingroup$

I'm considering using the AICc instead of the AIC to select models because my sample size is not much larger than my number of parameters (n=214, K=16 - which is not enough, according to Burnham and Anderson page 66).

However, whilst they don't really say much as to the assumptions we need to make in their book, wikipedia seem to indicate some of the assumptions we need to make on the model so that the formule for AICc holds: enter image description here

I don't understand what assuming that the model is univariate mean. I'm doing variable selection, across 16 variables (I plan on fixing many of them, because otherwise I'll just have overfitting) and I know that my model is not univariate. Otherwise I would just compare the 16 models with one variable.

Furthermore, I'm not sure how I can verify the other assumptions (them being: Assuming that the model [...] is linear in its parameters, and has normally-distributed residuals (conditional upon regressors)).

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ i suspect they mean the response y is univariate i.e. not vector regression. $\endgroup$
    – John Madden
    Jul 28, 2022 at 15:16
  • $\begingroup$ Like predicting a float rather than a vector? $\endgroup$
    – FluidMechanics Potential Flows
    Jul 28, 2022 at 15:26
  • 2
    $\begingroup$ yes (comment is now of sufficient length to post thanks to this parenthetical that may be ignored). $\endgroup$
    – John Madden
    Jul 28, 2022 at 15:28

1 Answer 1

Reset to default
2
+50
$\begingroup$

Here they are referring to a model in which the response variable is univariate (as opposed to a response vector) (hat tip to John Madden for pointing this out in comments). The formula given covers simple and multiple linear regression, but not multivariate regression.

In terms of the other assumptions, you should note that the AIC and AICc can be written in terms of the underlying maximised log-likelihood function for your model. Consequently, if you are using a regression model that is nonlinear, or which uses a non-Gaussian error distribution, you can still derive the formulae for the AIC and AICc, but you need to go through the process of looking at the log-likelihood function for your model. An example of the derivation of the AIC (for a Gaussian regression) is in this related question.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thanks for answering. Basically means that if I'm using a "standard" linear regression model that predicts one variable, my model is linear in its parameters, and has normally-distributed residuals? What if I'm doing a logistic regression? Would that change anything? $\endgroup$
    – FluidMechanics Potential Flows
    Aug 12, 2022 at 20:19
  • 1
    $\begingroup$ Yes, it would change things. If you do logistic regression then your lieklihood function changes, with flow-on effects to the AIC. $\endgroup$
    – Ben
    Aug 13, 2022 at 1:06
  • $\begingroup$ So the definition of the AIC in wikipedia (en.wikipedia.org/wiki/Akaike_information_criterion#Definition) is only valid for a linear regression model? Phrased as it is, it sounds like it's valid for any model. Or are the flow-on effects only on the AICc? $\endgroup$
    – FluidMechanics Potential Flows
    Aug 13, 2022 at 10:14
  • 1
    $\begingroup$ No, that is the general definition; you will see that the definition depends on the maximised log-likelihood function. If you change to a different model then you change your likelihood function, which changes the AIC. $\endgroup$
    – Ben
    Aug 13, 2022 at 13:10
  • 1
    $\begingroup$ The usual definition does not have the scaling factor $1/N$. However, as you point out, we use this statistic only for relative comparisons, so a scaling factor does not mess this up. $\endgroup$
    – Ben
    Aug 14, 2022 at 1:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.