1
$\begingroup$

I've been looking into using ROC curves as a evaluation tool of a multi-class classification. The only data I have about this classification is in form of 7-by-7 confusion matrix.

Visualisation of the ROC curves is not important, result in form of AUC is sufficient.

Is this possible?

Edit: Simplified the question, omitted unnecessary presumption I have made.

$\endgroup$
4
  • $\begingroup$ Welcome to CV. First, this site is not intended to be a resource for software specific questions. Then, it appears that you have assumed everyone is familiar with Hand and Till's paper. This is definitely not the case. Please elaborate and explain the key points from that paper relevant to your question. $\endgroup$
    – user78229
    Commented Jul 14, 2016 at 11:08
  • $\begingroup$ Do you have one 7-by-7 confusion matrix or all possible confusion matrices at the different thresholds? $\endgroup$
    – Calimo
    Commented Jul 14, 2016 at 14:05
  • $\begingroup$ @Calimo Currently I have one one matrix. Assuming that I could produce all possible confusion matrices at different thresholds, how would I go about computing the estimated AUC in multi-class case? $\endgroup$
    – Vladimir M
    Commented Jul 14, 2016 at 14:40
  • $\begingroup$ If you go for the Hand & Till method, I'd say you'd need to take all the 2x2 sub-matrices (should be 21 of them if I can count), calculate sensitivity and specificity, compute all the 21 AUCs and then average... $\endgroup$
    – Calimo
    Commented Jul 14, 2016 at 15:10

2 Answers 2

10
$\begingroup$

You cannot construct a ROC curve from the confusion matrix alone. A confusion matrix represents a single point in the ROC space, and you need all possible confusion matrices at all thresholds to build a full curve and compute the AUC.

This holds true for multi-class ROC analysis.

$\endgroup$
0
$\begingroup$

You can compute a ROC curve with a confusion matrix. You'll need to compute the true positive rate and the false positive rate. This will give you a point on the ``ROC'' space. The two other points are always $(0,0)$ and $(1,1)$, as varying the threshold positive $0$ to $1$ will always give you these points :

  • A threshold of $0$ will consider everything as positive so the $tpr$ is $1$ and the $fpr$ is $1$ ;
  • A threshold of $1$ will consider everything as negative so the $tpr$ is $0$ and the $fpr$ is $0$.

As a reminder :

  • True positive rate: $tpr = tp / p$
  • False positive rate: $fpr = fp / n$

Where:

  • $tp$ : true positive ;
  • $p$ : all positives values (false and true) ;
  • $fp$ : false positives ;
  • $n$ : all negatives values (false and true).
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.