I've set up code to give me a graphical depiction of AIC vs BIC parsimony over various degrees of polynomial models. On the rare occassion AIC does not match BIC trends, which parsimonious model would you select and why?
1
-
$\begingroup$ It depends on what you're trying to achieve $\endgroup$– Glen_bCommented Dec 5, 2016 at 6:35
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
2
I would not make an automatic choice. Sometimes you want a more parsimonious model, sometimes you want a less parsimonious model. Sometimes you want a model that is less parsimonious than either AIC or BIC recommends.
In almost all cases you will want to look at several models and see which is the most useful to you, for your particular purposes, with your knowledge of the substantive area, with the amount of data you have.
Attempts to automate these decisions are ways of saying to your boss "Don't pay me so much, the computer does my thinking".
:-)
answered May 14, 2013 at 18:54
-
$\begingroup$ Sorry, more context might be appropriate--I'm trying to select a polynomial degree for the "best" fit to run a predictive polynomial model. I do need some criterion for best. I thought that was the purpose for tools like AIC, BIC, and crossvalidation? $\endgroup$– JeffCommented May 14, 2013 at 19:04
-
1$\begingroup$ Well.... you have to make up your mind as to whether more or less parsimony is "better" for whatever your definition of "good" is. Personally, I think polynomial terms higher than 2 are hard to interpret and, if they are needed, I prefer splines. But AIC and BIC (and their variants) all have fierce partisans - if you search Google, you can find many papers. But the reason software offers both choices is that neither is that the decision on which is "better" isn't in yet. $\endgroup$ Commented May 14, 2013 at 19:07
$\begingroup$
$\endgroup$
The AIC can select models that are over-parameterized, because the model-size penalty is pretty low. The BIC also increases the penalty as the sample size increases, which seems like a desirable feature. Personally, I would favor the BIC.
$\begingroup$
$\endgroup$
You should read the original journal articles. AIC is and was intended to be an approximation as a ready or obvious solution was not available. The paper behind the AIC is quite logical and well done, but the AIC was solved in a non-proof-like manner. The BIC follows as a theorem from a proof. It is not an approximation, it is the exact answer, subject to the assumptions. It is proportionate to the posterior density subject to a cost function and any statistical assumptions regarding the models involved. They should move together.
