I am trying to classify a small dataset (around 500 records) into two classes. I used various methods like SVM, Naive Bayes and k-nn classifier. Now, I would like to set the results from one of the classifiers are my baseline and perform a statistical hypothesis testing. I am not much familiar with this field of statistical testing, and I wonder how to proceed on this.
I have been thinking of setting the SVM classifier as my baseline, but I am not sure how to perform a t-test (or similar) on the data. The input dataset has 10 attributes. Should I use the classification results from two classifiers and do a paired t-test on them? For example, I could take the result from Naive Bayes and perform the paired t-test with the SVM results (which is the baseline). Is this the right approach?
Also, I am confused with the explanation for null and alternate hypothesis. Could someone be willing to give an idea about how to fix the null and alternate hypothesis.