# AIC BIC Mallows Cp Cross Validation Model Selection

If you have several linear models, say model1, model2 and model3, how would you cross-validate it to pick the best model?

(In R)

I'm wondering this because my AIC and BIC for each model are not helping me determine a good model. Here are the results:

Model - Size (including response) - Mallows Cp - AIC - BIC
Intercept only - 1 - 2860.15 - 2101.61 - 2205.77
1 - 5 - 245.51 - 1482.14 - 1502.97
2 - 6 - 231.10 - 1472.88 - 1497.87
3 - 7 - 179.76 - 1436.29 - 1465.45
4 - 8 - 161.05 - 1422.43 - 1455.75
5 - 9 - 85.77 - 1360.06 - 1397.55
6 - 10 - 79.67 - 1354.79 - 1396.44
7 - 17 - 27.00 - 1304.23 - 1375.04
All Variables - 25 - 37.92 - 1314.94 - 1419.07


Note - assume the models are nested.

• why isn't you're AIC helping? AIC and BIC appear even to coincide here. – charles Mar 10 '14 at 0:34
• The lowest AIC/BIC is obtained on the 17 variable model - so this is the best? Shouldn't I do some calculations to check if the 'additional AIC or BIC' is worth an extra variable? Also - would you use LASSO to get several models, subsets regression, or neither? I used Subsets to create 6 models but now I'm wondering if its better to use LASSO? The question also is - is this method even correct for justifying which model is the best 'compromise'? – Dino Abraham Mar 10 '14 at 2:09