So, as far as I understand, all folds in k-fold cross-validation need to be the same size, or as close as possible. However, if one uses stratified cross-validation, each class should be represented in same proportion as they are in the total dataset.
My issue is that I have 23 classes, each with 35 images, making the total dataset size 805 images. As I'm trying to replicate a study that used 10-fold cross-validation on the same dataset, I kind of want to stick to 10-fold cross-validation, even though I know I could use 5 or 7 folds or even leave-one-out.
So, dividing the complete data into 10 folds gives me five folds with 80 and five folds with 81 images each, picked randomly. However, now my classes are not represented in a balanced manner.
Alternatively, I have to divide each class into ten folds, resulting in the dataset being split into five folds of 69 (3 images from each of 23 classes) and five folds of 92 images (4 images from each of 23 classes).
Is either of these cross-validation strategies better than the other? Should I just not bother with balanced cross-validation and go with the first method? Am I misunderstanding something?