For a potential emotion recognition bachelor-project I was wondering what statistical test I have to perform when I get my results to test whether it's significant. I will be testing which combination of feature extraction and machine learning algorithm will give me the best percentage of correct classified. The results will exist out of Combination A gives ...% classified, Combination B ...%, Combination C ...% and so on. Which statistical test should I use to test whether Combination ... is significantly better than the others and why?
For example: 6 emotions have to be recognized in a database with 100 faces for each emotion (600 total). Every machine learning algorithm will use 2/3 for training and 1/3 for testing. Which face per emotion is in the training set and which one is in the test set is randomly selected every epoch for 100 epochs. The end result is for example: Combination A classified 93.4% correct, Combination B 91.2%, Combination C 86.3% and so on. Which statistical test should I use to test whether Combination A is significantly better than Combination B (and C) and why?
Also is it dichotomous? As the probability of successful selecting the right emotion is 16.67%.