Assume I have a small sample size, e.g. N=100, and two classes. How should I choose the training, cross-validation, and test set sizes for machine learning?
I would intuitively pick
- Training set size as 50
- Cross validation set size 25, and
- Test size as 25.
But probably this makes more or less sense. How should I really decide these values? May I try different options (though I guess it is not so preferable... increased possibility of over learning)?
What if I had more than two classes?