1
$\begingroup$

I have a dataset of 900 images, distributed across 6 classes, with 150 images per class. To develop a classifier and assess its performance, I will utilize k-fold cross-validation. In this case, I will employ 3-fold cross-validation.

For each fold, I will allocate 70% of the data for training purposes, leaving the remaining 30% for testing. Consequently, within the training partition, there will be 105 images per class. However, during the training phase, I will only select 20 images per class to train the model. When evaluating the model, I will assess its performance on the entire test partition.

To report the overall performance of the model, I will calculate the average test accuracy across the 3 folds and present this averaged accuracy as the final performance metric.

Despite utilizing a subset of the training partition, can this approach still be referred to as "3-fold cross validation"?

$\endgroup$
1
  • 1
    $\begingroup$ It is certainly not what people would intuitively expect from "3-fold cross-validation". They would expect the training set to be split into three subsets of roughly equal size A,B,C. Then train on all of A and B comparing that model to C, train on all of B and C comparing that model to A, and train on all of C and A comparing that model to B. $\endgroup$
    – Henry
    Jul 5, 2023 at 13:21

1 Answer 1

1
$\begingroup$

In $k$-fold cross-validation there are no "train" and "test" buckets. You put all the data into one bucket, split it into $k$ "folds" and on each iteration you train on $k-1$ folds and test on the remaining one.

That said, there are scenarios where you may want to combine it with classical cross-validation, i.e. do $k$-fold cross-validation on the training data, and then test the final model on the test set. This may be needed to avoid overfitting when you used $k$-fold cross-validation for model selection or picking the best hyperparameters.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.