Where is the divide between information criterion (AIC, BIC, etc...) and cross validation? I've taken a regression class and am now in a machine learning class.  In regression, we talk about model selection using adj-R2 and AIC/BIC.  In my machine learning class, we primarily select models using cross validation (or sample splitting) and the validation error.
I am not seeing the connection between the two, or more specifically the divide between the two.  Seems like cross validation/sample splitting can work on any model, so why even bother with adj-R2 or AIC/BIC?  
When would we want to use one over the other?  And are there situations where AIC/BIC wouldn't work?
Thanks
 A: The question is quite broad, but I will give some starting points:
Why bother with AIC/BIC: using cross validation (CV) is (much) more computationally expensive than using AIC/BIC, except for some special cases like leave-one-out cross validation (LOOCV) for regression where it is computationally as cheap as AIC/BIC.
Situations where AIC/BIC would not work: AIC/BIC are only available for models estimated using maximum likelihood estimation (MLE), and this is a relative small class of models in the context of machine learning.
Connection between CV and AIC/BIC: under some assumptions, AIC is asymptotically equivalent to LOOCV while BIC is asymptotically equivalent to k-fold CV with a specific fold size that depends on the sample size. So under these assumptions, you can save a lot of computations by replacing CV with AIC/BIC.
On $R^2_{adj.}$: According to "Justification for and optimality of $R^2_{adj.}$ as a model selection criterion", it is questionable whether $R^2_{adj.}$ can be regarded as an optimal model selection criterion. Personally, I would not use it when other alternatives like AIC, BIC or CV are available.
