1
$\begingroup$

Among many methods used to detect and avoid overfitting, I am particularly interested in those two:

  1. replica method
  2. reusable holdout

My question is: what is their relation in the context of adaptive data analysis and model selection? To be more concrete, let's focus on developing a multivariate logistic classifier. We are given a dataset of $N$ $D$-dimensional points $x$ associated with binary labels $y$. $N$ and $D$ are large. We want to select which dimensions (i.e. variables) of $x$ should be used by the classifier. Is it possible to say which method of the above two would in general be better for this purpose? If yes, which one?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.