0
$\begingroup$

I've studied many questions and answers on the theme of nested cross-validation. I understand why we need it and how I can, after that part, find the optimal hyperparameters and any other things I'm interested in.

But here I'm interested in the following scenario: I want to build a classification model using, say, a RandomForest. So, for my model I want to pick the hyper-parameters, choose between two different preprocessing schemes and a get an unbiased estimated of the generalization error. Let's also say that I consider preprocessing to be part of the 'model building process'.

My question is do I need a nested nested CV (inner loop: hyperparameters, outer loop: preprocessing, outmost loop: generalization error) or can I stick with a nested CV and treat the different preprocessing schemes as just another hyperparameter? In other words, can the inner loop treat both hyperparameter and preprocessing selection?

An answer is given by @cbeleites here, but it is about selecting an algorithm and not a preprocessing scheme. Does this hold for preprocessing as well?

$\endgroup$
1
$\begingroup$

Yes it holds for preprocessing as well.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.