Timeline for Area Under ROC Curve for Multiple Classes
Current License: CC BY-SA 3.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 18, 2013 at 16:11 | comment | added | Elvis | I would just, for each class, report the three probabilities of being classified in A, B, C. It makes a big table which can be difficult to read, but at least all the information is available. To get synthetic comparison between your classifiers, I think you should really return to the concrete problem you're trying to tackle. You can for example attribute a (relative) cost to each type of missclassification, then compute the cost expected when using each of your classifier, for example. | |
| Nov 18, 2013 at 16:06 | comment | added | Elvis | I don’t think you can use the definition of $AUC_{total}$ for a classifier that doesn’t output probabilities to belog to each class, or something similar (a score for each class) which allows to derive binary classifiers. | |
| Nov 18, 2013 at 15:46 | comment | added | Prometheus | But then how do I calculate $AUC(A,B)$ without disregarding class $C$ ? | |
| Nov 18, 2013 at 13:36 | comment | added | Elvis | No wonder you have $AUC < 0.5$ then... to make the AUC make sense, you have to define a binary classifier. You can’t have ``unclassified'' elements. | |
| Nov 18, 2013 at 11:30 | comment | added | Prometheus | For $AUC(A,B)$, I simply disregard all labels with class $C$. In other words, if $A$ is "positive" and $B$ is "negative", then $FPR = \frac{\mbox{No. of B classified as A}}{\mbox{No. of B}}$ and $TPR=\frac{\mbox{No. of A classified as A}}{\mbox{No. of A}}$. | |
| Nov 18, 2013 at 11:10 | comment | added | Elvis | Note that the classifier I described (predict the class with maximum probability) is discrete. The point is that you have to build two-class classifiers from each multi-class classifier you consider. When you compute $AUC(A,B)$, what do you do when the classifier predicts "class C"? | |
| Nov 18, 2013 at 9:49 | comment | added | Prometheus | Actually, no. I used the multi-class variants of all 10 classifiers. Since not all classifiers give probabilities (like SVM), I treated all 10 as 'discrete' classifiers. As you may know, discrete classifiers only give a single point on the ROC. In the paper that I linked in the OP, the author recommends calculating $AUC$ for discrete classifiers by connecting this single point to $(0,0)$ and $(1,1)$ and taking the area enclosed. I did the same. For three classes I get 3 ROCs (AB, BC and AC) and hence 3 AUCs for each classifier. $AUC_total$ is simply the average of the three. | |
| Nov 17, 2013 at 22:57 | history | answered | Elvis | CC BY-SA 3.0 |