Why LOOCV(Leave-One-Out Cross-Validation) has less bias than K fold Cross Validation ? Please explain with example if possible

  • $\begingroup$ There's a good explanation at Introduction to Statistical Learning, section 5.1.4 on bias-variance trade-off. There's some examples on chapter 5, none of them focused on your question though. $\endgroup$ – lrnzcig Apr 5 '16 at 14:34
  • $\begingroup$ @lrnzcig Yes I have gone through that but did not find my answer. Any further help on this ? $\endgroup$ – user110244 Apr 5 '16 at 16:22
  • $\begingroup$ Ok. Could you be more concrete? What is it you don't understand in that book? Do you understand the main point they are making -using less info when fitting the model, i.e. using less % of samples for training, will lead to a worse estimate of the test error. Thus, "simply", the more % of samples you use, the less bias you have. $\endgroup$ – lrnzcig Apr 5 '16 at 16:48
  • $\begingroup$ @lrnzcig I had this point in my mind but not sure how much correctly I interpreted this . Now seems like I am right. Thanks for your suggestion. Are you on social media? lets connect $\endgroup$ – user110244 Apr 5 '16 at 17:01
  • 1
    $\begingroup$ Note that in some cases (particularly with very small sample size), LOO can have a decidedly higher pessimisic bias than a well-chosen k for k-fold CV. E.g., if the classifier incorporates the frequency of the classes, leaving one sample out will always cause the tested class to have less cases in the training set. As you ask for examples: hyperspec.r-forge.r-project.org/blob/Beleites2005.pdf $\endgroup$ – cbeleites unhappy with SX Apr 6 '16 at 9:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.