So, as far as I understand, all folds in k-fold cross-validation need to be the same size, or as close as possible. However, if one uses stratified cross-validation, each class should be represented in same proportion as they are in the total dataset.

My issue is that I have 23 classes, each with 35 images, making the total dataset size 805 images. As I'm trying to replicate a study that used 10-fold cross-validation on the same dataset, I kind of want to stick to 10-fold cross-validation, even though I know I could use 5 or 7 folds or even leave-one-out.

So, dividing the complete data into 10 folds gives me five folds with 80 and five folds with 81 images each, picked randomly. However, now my classes are not represented in a balanced manner.

Alternatively, I have to divide each class into ten folds, resulting in the dataset being split into five folds of 69 (3 images from each of 23 classes) and five folds of 92 images (4 images from each of 23 classes).

Is either of these cross-validation strategies better than the other? Should I just not bother with balanced cross-validation and go with the first method? Am I misunderstanding something?

  • $\begingroup$ @user2974951 what are you basing that information on? Is there a minimum number of samples needed for cross-validation? $\endgroup$ – kelkka Feb 22 '19 at 13:32
  • $\begingroup$ @user2974951 in that case it is a meaningless comment as I haven't given any information about what I'm doing with the data (except cross-validation). The data I'm using has been published in a journal article and used in a similar way to what I am attempting. The validity of the size of the dataset was not the question. $\endgroup$ – kelkka Feb 22 '19 at 14:55
  • $\begingroup$ Never mind then. What is the purpose of creating these folds? $\endgroup$ – user2974951 Feb 22 '19 at 15:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.