# Is it possible that the AIC and BIC give totally different model selections?

I'm performing a Poisson Regression model with 1 response variable and 6 covariates. Model selection using AIC results in a model with all covariates as well as 6 interaction terms. The BIC however, results in a model with only 2 covariates and no interaction terms. Is it possible that the two criteria, that look very similar, yield totally different model selections?

• If it wasn't possible to get different model recommendations from the two metrics, then there wouldn't be two metrics, we'd just always use one. May 14 '18 at 23:59
• The word "totally different" is hard to interpret when models are the result of choosing from a set of discrete parameters. May 15 '18 at 3:20

It is possible indeed. As explained at https://methodology.psu.edu/AIC-vs-BIC, "BIC penalizes model complexity more heavily. The only way they should disagree is when AIC chooses a larger model than BIC."

If your goal is to identify a good predictive model, you should use the AIC. If your goal is to identify a good explanatory model, you should use the BIC. Rob Hyndman nicely summarizes this recommendation at
https://robjhyndman.com/hyndsight/to-explain-or-predict/:

"The AIC is better suited to model selection for prediction as it is asymptotically equivalent to leave-one-out cross-validation in regression, or one-step-cross-validation in time series. On the other hand, it might be argued that the BIC is better suited to model selection for explanation, as it is consistent."

The recommendation comes from Galit Shmueli’s paper “To explain or to predict?”, Statistical Science, 25(3), 289-310 (https://projecteuclid.org/euclid.ss/1294167961).

• The only way they should disagree is when AIC chooses a larger model than BIC.” Technically BIC could choose a larger model if $\ln n < 2$, i.e. $n \le 7$. Hopefully samples of size 7 aren’t too much of an issue, though. :p May 15 '18 at 4:01