# Isn't leave-1-out insufficient for proper classification evaluation?

I encountered several papers that used some classification method (for instance, LDA), with leave-1-out validation, and posted the classification results as an aggregation of all results (for all samples 'left out').

I was wondering if this weren't a little problematic. I'm used to putting, say, 20% of samples as test and then training the classifier on the remaining 80%. But in the leave-1-out, the case is more similar to 99% training and 1% testing (in the case of 100 samples, for the sake of example)... so how can I properly compare my method to leave-1-out?

Thanks.

The leave-one-out method is a cross-validation technique. If you have a data set with N=100 samples, you will run 100 train/test iterations where, in each iteration, the train set will have 99 test examples and 1 training example. This way you find out, for each sample, what the generated classification is for the sample when training on all other samples. You get a final error value by averaging the error over all iterations.
It is a particular case of k-fold cross validation (k=N) where you divide your data set into k equal-sized subsets and train on k-1 of them while you test on the remaining subset, and repeat that scenario for all combinations of k-1 training subsets and 1 test subset. You then get an error value by averaging the error of all iterations.