1
$\begingroup$

I felt a little bit puzzled about cross-validation. CV is a method to find the best parameter to check the generalization ability of the classifier in the classification problem.

  1. I am really not sure whether I should apply cross-validation on the training set or on the entire dataset? I prefer on the training set because if you apply it to the entire set, it will lead to overfitting again.
  2. Next question is: what is the correct step to use cross-validation in the classification problem. My view is (1)separate the data set as training set and testing set;(2)apply CV to the training set to get the best parameter; (3)use the best parameter to train a model;(4)use this model to test the testing set.

Am I correct?

$\endgroup$
0
$\begingroup$

CV as a method to find the best hyperparameter should be done on the training set, as you indicated.

CV as a method of evaluating a modeling process should be done on the whole set, however.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.