I have a dataset with a categorical outcome no/yes, 8 predictors, in 31 examples, and I'm trying to classify the examples using different algorithms in the caret package. There are 9 "no" and 22 "yes". The code I'm using is as follows:

## SVM
ctrl= trainControl(method= "LOOCV",sampling="up", classProbs = 
TRUE,savePredictions = TRUE, summaryFunction = twoClassSummary)
svm = train(remission ~ ., data = num.m, method = "svmLinear", 
trControl=ctrl, metric="ROC")

I've read that oversampling should only be applied after cross-validating when using loocv, but I'm not sure how to do this. How would be the proper way to solve this problem?


1 Answer 1


You can use oversampling with cross-validation; any form of cross-validation. However, ensure that test set is not changed. So, the basic idea is to divide the data into k-folds; use oversampling or undersampling (eg. SMOTE) on the samples in all folds except the k^th fold, i.e., training data. Train your classifier on training data which is oversampled and evaluate your classifier on the test set which is not sampled.

In leave one out cross-validation, leave one sample which you will use as test set and oversample the other remaining set. Train your classifier on all the oversampled data and test your classifier on test set.

  • 1
    $\begingroup$ I believe the third sentence, last words should be "i.e. testing data" $\endgroup$
    – Underminer
    Commented Mar 24, 2017 at 16:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.