0
$\begingroup$

I have a cross validation error higher than independent test set error (3-10% difference in error). It is not a training error but a cross-validation error using LOOCV. Does anyone have similar kind of situation. Could someone provide me some intuition behind it.

$\endgroup$
1
$\begingroup$

Two possibilities come immediately to my mind:

$\endgroup$
  • $\begingroup$ Thank you for your comments. I have balance sample in both the training (112) and testing (30). It is binary classification problem. I use accuracy, sensitivity and specificity as a performance measure for both LOOCV and Test set. I do calculate C.I in both the LOOCV and testing. The accuracy lies between 95% C.I in both LOOCV and test set. However, testing accuracy does not lies in between the 95% C.I of LOOCV. Meanwhile, I will read the papers you have refereed. $\endgroup$ – TmN Mar 13 '17 at 9:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.