By increasing the size of the training set the model memorize more data. Thus, will using leave one out increase the chance of overfitting?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
ML model will start to "memorize" data with increasing complexity of your algorithm (and its parameters), not the size of training set.
Cross-validation is used to estimate your model performance on data that was not used to train. If you use LOOCV (k=n) then your k models will be (almost) identical. This gives you a high variance in your model evaluation and low bias regarding the final model trained on the entire data set.
Use 10-times 10-fold stratified CV if you unsure about a good value for k.
-
2$\begingroup$ The part about high variance is not always true and is an incorrect generalization of a special case for unstable algorithms - see recent posts about bias variance trade-off around the site $\endgroup$ Oct 26, 2018 at 14:47