2
$\begingroup$

I've set up code to give me a graphical depiction of AIC vs BIC parsimony over various degrees of polynomial models. On the rare occassion AIC does not match BIC trends, which parsimonious model would you select and why?

$\endgroup$
2
$\begingroup$

I would not make an automatic choice. Sometimes you want a more parsimonious model, sometimes you want a less parsimonious model. Sometimes you want a model that is less parsimonious than either AIC or BIC recommends.

In almost all cases you will want to look at several models and see which is the most useful to you, for your particular purposes, with your knowledge of the substantive area, with the amount of data you have.

Attempts to automate these decisions are ways of saying to your boss "Don't pay me so much, the computer does my thinking".

:-)

$\endgroup$
  • $\begingroup$ Sorry, more context might be appropriate--I'm trying to select a polynomial degree for the "best" fit to run a predictive polynomial model. I do need some criterion for best. I thought that was the purpose for tools like AIC, BIC, and crossvalidation? $\endgroup$ – Jeff May 14 '13 at 19:04
  • 1
    $\begingroup$ Well.... you have to make up your mind as to whether more or less parsimony is "better" for whatever your definition of "good" is. Personally, I think polynomial terms higher than 2 are hard to interpret and, if they are needed, I prefer splines. But AIC and BIC (and their variants) all have fierce partisans - if you search Google, you can find many papers. But the reason software offers both choices is that neither is that the decision on which is "better" isn't in yet. $\endgroup$ – Peter Flom - Reinstate Monica May 14 '13 at 19:07
1
$\begingroup$

The AIC can select models that are over-parameterized, because the model-size penalty is pretty low. The BIC also increases the penalty as the sample size increases, which seems like a desirable feature. Personally, I would favor the BIC.

$\endgroup$
0
$\begingroup$

You should read the original journal articles. AIC is and was intended to be an approximation as a ready or obvious solution was not available. The paper behind the AIC is quite logical and well done, but the AIC was solved in a non-proof-like manner. The BIC follows as a theorem from a proof. It is not an approximation, it is the exact answer, subject to the assumptions. It is proportionate to the posterior density subject to a cost function and any statistical assumptions regarding the models involved. They should move together.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.