7
$\begingroup$

By increasing the size of the training set the model memorize more data. Thus, will using leave one out increase the chance of overfitting?

$\endgroup$
1
$\begingroup$

ML model will start to "memorize" data with increasing complexity of your algorithm (and its parameters), not the size of training set.

Cross-validation is used to estimate your model performance on data that was not used to train. If you use LOOCV (k=n) then your k models will be (almost) identical. This gives you a high variance in your model evaluation and low bias regarding the final model trained on the entire data set.

Use 10-times 10-fold stratified CV if you unsure about a good value for k.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ The part about high variance is not always true and is an incorrect generalization of a special case for unstable algorithms - see recent posts about bias variance trade-off around the site $\endgroup$ – Xavier Bourret Sicotte Oct 26 '18 at 14:47
0
$\begingroup$

The main reason to not use leave-one-out would be that it is computationally heavy in comparison to k-fold cross validation. (Although it can be done more directly Proof of LOOCV formula)

The larger training set is in general not causing a stronger overfitting. In fact, it has the opposite effect. The noise will be canceled out more and is less likely to create overfitting (the model remembers the average of the training data). Also, in comparison to k-fold CV there are more ways in which the data is split up into training and test data, which should in general be better as well.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.