1
$\begingroup$

I've been advised by a colleague that when performing backwards elimination for model selection using AIC as my criterion, I should remove terms individually - starting from the most complex interaction, and working down to the main effects, comparing the AIC for the reduced model and comparing it to the full model (the original model containing all terms).

For example, if I had 3 main effects, and their interactions in my model I could write the full model $y_1$ and remove the three way interaction to make model $y_2$:

$y_1 = (a \times b \times c) + (a \times b) + (a \times c) + (b \times c) + a + b + c$

$y_2 = (a \times b) + (a \times c) + (b \times c) + a + b + c$

I find that removing the term reduces the AIC by >-2, so I should remove the three way interaction. I then reduce $y_2$ further by removing the first two way interaction ($a \times b$).

$y_3 = (a \times c) + (b \times c) + a + b + c$

Following the advice given to me by my colleague I would then compare AIC of $y_3$ to that of $y_1$ and reject it if the AIC reduces by >-2. However, it strikes me that if the three way interaction, which is missing from both $y_2$ and $y_3$, is extremely uninformative such that $\Delta$AIC >>-2 then I will remove the $a \times b$ interaction unless it is highly informative such that it overcomes the effect of removing the three way interaction.

This brings me to my question: in backwards elimination, should one compare AIC of the reduced model to:

a) the full model (e.g. $AIC_{y1} - AIC_{y3}$)

b) the model upon which the reduced model is based (e.g. $AIC_{y2} - AIC_{y3}$)

c) something else...

$\endgroup$
0
$\begingroup$

I think choice b) is probably what your colleague meant. Choice a) doesn't seem sensible at all.

$\endgroup$
  • $\begingroup$ @peter_flom could you expand on your answer? I'm keen to see some material which supports this, I'd like something which gives a concrete statement that model comparison is between the reduced model and the model from which it was reduced (i.e. not the full model, and not other possible models e.g. the one with the lowest AIC). $\endgroup$ – rg255 Mar 16 '17 at 11:31
  • 1
    $\begingroup$ Any book or material that deals with model selection via AIC or other similar methods should have this, but I can't cite, offhand, some specific place that I am absolutely sure says this. But .. just think about it! a) doesn't make sense! $\endgroup$ – Peter Flom - Reinstate Monica Mar 16 '17 at 11:48
  • 1
    $\begingroup$ To be honest, I'm more concerned as to whether the comparison should be to the model with the lowest AIC, but I'd say that doesn't make sense because the DeltaAIC threshold would depend on the difference in number of terms - e.g. if I had a model with 6 terms that has an AIC of 112.4, a model with 2 terms could have an AIC of 120.3999 and still be considered a better fit right? $\endgroup$ – rg255 Mar 16 '17 at 13:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.