# What to choose from BIC/AIC/ridge/elastic net?

I have the following regression problem

I have about 60 independent variables; some of them have a high correlation with others. I have around 3 million observations

(1) - My main goal is out-of-sample-prediction, so my main question is: which regularization method should I use in this case?

Some more questions (assumptions I have, probably a little confused)

(2) - Ridge regression, while not completely removing coefficients, would keep those coefficients low that lasso/elastic net/BIC would completely remove; is that correct? (If it doesn't, would that be a problem?)

(3) - If I wanted to use AIC/BIC in this case, I would have to test all possible combinations of the 60 independent variables?

(4) - Would it make sense to start with AIC/BIC, then later do ridge regression with the remaining independent variables? (I guess ridge regression after AIC/BIC might make sense because some of the independent variables correlate with others?)

Thanks

• With 3 million observations, and 60 features, there is no ostensible reason to perform variable selection, unless you are running into numerical problems. Particularly if you are interested in prediction. – tchakravarty Nov 30 '12 at 11:24
• You don't specify why you need parsimony in your model. Is it difficult or expensive to collect all the 60 variables? Just like @fgnu asked, are you running into numerical problems? – user765195 Nov 30 '12 at 19:06

An alternative to this (which also works well for nonlinear models), if you are using R, is the perturb package.