I am trying to classify two types of objects, which unfortunately have high-dimensional features with few samples. (230 features 12 samples from each group).
As a first step: To reduce the dimension, I have tested three different approaches of PCA (each with slightly different parameters) and use the scores of the first 3 PCs as the features instead of the original 230 features.
As a second step: in order to classify those objects, I have tested three different classifiers (SVM, 3-NN, Naive Bayes) where I use the leave one out method, i.e.: training the classifier on 23 objects and tested it on the one leave out, for testing.
Sum up: I have use 3 different dimensional reduction methods and three use different classifiers (nine combinations overall).
In one combination (out of 9), I got excellent classification performance.
My question: Is there some statistic approach I can use here in order to prove that this option was indeed robust and the correct one, and didn't happen by chance?