AIC, BIC parsimony I've set up code to give me a graphical depiction of AIC vs BIC parsimony over various degrees of polynomial models.  On the rare occassion AIC does not match BIC trends, which parsimonious model would you select and why?
 A: I would not make an automatic choice. Sometimes you want a more parsimonious model, sometimes you want a less parsimonious model. Sometimes you want a model that is less parsimonious than either AIC or BIC recommends.
In almost all cases you will want to look at several models and see which is the most useful to you, for your particular purposes, with your knowledge of the substantive area, with the amount of data you have. 
Attempts to automate these decisions are ways of saying to your boss "Don't pay me so much, the computer does my thinking".
:-)
A: The AIC can select models that are over-parameterized, because the model-size penalty is pretty low. The BIC also increases the penalty as the sample size increases, which seems like a desirable feature. Personally, I would favor the BIC.
A: You should read the original journal articles.  AIC is and was intended to be an approximation as a ready or obvious solution was not available.  The paper behind the AIC is quite logical and well done, but the AIC was solved in a non-proof-like manner.  The BIC follows as a theorem from a proof.  It is not an approximation, it is the exact answer, subject to the assumptions.  It is proportionate to the posterior density subject to a cost function and any statistical assumptions regarding the models involved.  They should move together.  
