I have the following regression problem
I have about 60 independent variables; some of them have a high correlation with others. I have around 3 million observations
(1) - My main goal is out-of-sample-prediction, so my main question is: which regularization method should I use in this case?
Some more questions (assumptions I have, probably a little confused)
(2) - Ridge regression, while not completely removing coefficients, would keep those coefficients low that lasso/elastic net/BIC would completely remove; is that correct? (If it doesn't, would that be a problem?)
(3) - If I wanted to use AIC/BIC in this case, I would have to test all possible combinations of the 60 independent variables?
(4) - Would it make sense to start with AIC/BIC, then later do ridge regression with the remaining independent variables? (I guess ridge regression after AIC/BIC might make sense because some of the independent variables correlate with others?)
Thanks