Among many methods used to detect and avoid overfitting, I am particularly interested in those two:
My question is: what is their relation in the context of adaptive data analysis and model selection? To be more concrete, let's focus on developing a multivariate logistic classifier. We are given a dataset of $N$ $D$-dimensional points $x$ associated with binary labels $y$. $N$ and $D$ are large. We want to select which dimensions (i.e. variables) of $x$ should be used by the classifier. Is it possible to say which method of the above two would in general be better for this purpose? If yes, which one?