I am comparing the performance of five classifiers over 100 datasets. I have a table of data where each column is a classifier and each row is a dataset, so the cell (i,j) is the performance (accuracy) of classifier j over sample i.
I am using the Friedman test for statistical comparison, but my question relates to visualization.
I want to generate some sort of plot that gives an indication of which classifier performs best.
So far, my idea has been to separate the original matrix in five 100x2 matrixes, each corresponding to a classifier, where the first column indicates the dataset's label (1-100) and the second column the performance.
I then sort all columns in ascending order based on the values in the second column, so for example:
1 80 2 85 3 79
3 79 2 85 1 80
I then plot the sorted performance vector for each classifier, such that in the resulting plot I can compare best performances, second-best performances, etc.
This allows me to draw conclusions such as "The best performance of Classifier A is better than the best performance of Classifier B."
However, I am not convinced this provides meaningful information. For example, if the best performance of A was on dataset 1 and the best performance of B on dataset 2, it might still be that B outpeformed A on dataset 2.