# Numerical leave-some-out, k-fold cross validation example please

I am the kind that understands much easier on numerical examples.

Assume we will use 5-fold cross-validation. Can you calculate number of examples in training, test and validation sets for a data that have 100 examples in total?

Thanks a lot.

• Give us your go on the numbers, and we'll correct it ;-) Feb 18, 2016 at 11:38
• Ok. Assume training, test, validation are the number of examples in training, test, and validation sets. My guess is test=validation=100/5=20. Remaining becomes training set: training=100-2*20=60. But I am not sure whether test and validation should be equal. Feb 18, 2016 at 13:36

k-fold cross validaton splits into 2 sets, i.e. for 100 cases 5-fold CV

• 5 x (20 validation/optimization + 80 trainig), or
• 5 x (20 test/final validation + 80 optimized training)

Splitting into 3 (training + validation/optimization + test/final validation) sets e.g. by nesting k-fold cross validation. You can choose the inner and outer validation schemes independently, including k-fold CV for both, e.g.:

• 5-fold within 5-fold:
5 x (20 test/final validation + 80 optimized training)
= 5 x (20 test/final validation + 5 x (16 validation/optimization + 64 training))
• 4-fold within 5-fold:
5 x (20 test/final validation + 80 optimized training)
= 5 x (20 test/final validation + 4 x (20 validation/optimization + 60 training)

There's no special reason why the optimization/validation and test/final validation sets need to have the same number of cases. Comparison (particularly, multiple comparison) may require huge numbers of test cases in order to achieve a testing precision that allows meaningful model selection. It then may sound weird to have fewer test cases for the validation of the finally chosen model.