17
$\begingroup$

I have a problem with 6 classes. So I build a multiclass classifier, as follows: for each class, I have one Logistic Regression classifier, using One vs. All, which means that I have 6 different classifiers.

I can report a confusion matrix for each one of my classifiers. But, I would like to report a confusion matrix for ALL the classifiers, as I've seen in a lot of examples here.

How can I do it? Do I have to change my classification strategy, using a One vs. One algorithm instead of One vs. All? Because on these confusion matrices, the reports says the false positives for each class.

Example of a multiclass confusion matrix

Multiclass Confusion Matrix

I would like to find the number of misclassified items. In the first row, there are 137 examples of class 1 that were classified as class 1, and 13 examples of class 1 that were classified as class 2. How to get this number?

Improve this question
$\endgroup$
4
  • $\begingroup$ The number of misclassified items is the sum of all elements in the matrix minus the trace of the matrix...but I don't think this is what you mean. $\endgroup$
    – user75138
    Nov 2, 2015 at 19:23
  • 1
    $\begingroup$ Mechanically, you get this matrix by first separating your test set by their actual class (say, Target =1, Target = 2 etc), then apply your trained classifier to each point in each group. So, for Target = 1, you would be filling in the top row of the matrix, based on how many members of this group were assigned to each class. $\endgroup$
    – user75138
    Nov 2, 2015 at 19:24
  • $\begingroup$ This is exactly the way it should be done.... So mechanical as you said. Thanks! $\endgroup$
    – Victor Leal
    Nov 10, 2015 at 12:51
  • 1
    $\begingroup$ no problem. I mentioned this more formally in my post as well, but sometimes it helps to see the actual recipe. $\endgroup$
    – user75138
    Nov 10, 2015 at 19:49

3 Answers 3

Reset to default
5
$\begingroup$

Presumably, you are using these classifiers to help choose one particular class for a given set of feature values (as you said you are creating a multiclass classifier).

So, lets say you have $N$ classes, then your confusion matrix would be an $N\times N$ matrix, with the left axis showing the true class (as known in the test set) and the top axis showing the class assigned to an item with that true class. Each element $i,j$ of the matrix would be the number of items with true class $i$ that were classified as being in class $j$.

This is just a straightforward extension of the 2-class confusion matrix.

Improve this answer
$\endgroup$
10
  • $\begingroup$ Yes! I know about that! But, how to say the false positives? I mean, there are examples where the number of items misclassified are shown....and my classifiers just say "Hey, there are 60 items of class A, and 40 are of another class (I just can't say which one it is...)" $\endgroup$
    – Victor Leal
    Nov 2, 2015 at 18:10
  • 1
    $\begingroup$ @VictorLeal I don't follow, a confusion matrix will tell you false positive, true positive, true negatives, false negatives..what is missing? $\endgroup$
    – user75138
    Nov 2, 2015 at 18:46
  • 1
    $\begingroup$ @VictorLeal see here: en.wikipedia.org/wiki/Confusion_matrix $\endgroup$
    – user75138
    Nov 2, 2015 at 18:47
  • $\begingroup$ I know the information that we have in a Confusion Matrix. Maybe an image can represents better what I'm talking about: Confusion Matrix Multiclass $\endgroup$
    – Victor Leal
    Nov 2, 2015 at 18:57
  • $\begingroup$ @VictorLeal It looks like a normal confusion matrix to me...LHS shows the actual class the the top shows the assigned class...am I missing something? Also, you should add this image to your post..it will be helpful $\endgroup$
    – user75138
    Nov 2, 2015 at 18:59
15
$\begingroup$

While there are some answers already on this forum I thought I'd give the explicit equations to make it more definite:

Assuming you have a multi-class confusion matrix of the form, \begin{align} C=\text{Actual}\begin{matrix} & \text{Classifed} & \\ c_{11} & ... & c_{1n}\\ \vdots & \ddots & \\ c_{n1} & & c_{nn} \end{matrix} \end{align}

The confusion elements for each class are given by:

$tp_i = c_{ii}$

$fp_i = \sum_{l=1}^n c_{li} - tp_i$

$fn_i = \sum_{l=1}^n c_{il} - tp_i$

$tn_i = \sum_{l=1}^n \sum_{k=1}^n c_{lk} - tp_i - fp_i - fn_i$

Improve this answer
$\endgroup$
3
  • $\begingroup$ what is l and L? $\endgroup$
    – girl101
    May 28, 2019 at 7:18
  • 1
    $\begingroup$ also, what is the tp,tn,fp,fn for all the classes together $\endgroup$
    – girl101
    May 28, 2019 at 7:21
  • $\begingroup$ tp = true positive , fp = false positive, fn = false negative , tn = true negative. I suppose that index i is reference to each class. $\endgroup$
    – albert
    Aug 20, 2019 at 22:25
4
$\begingroup$

Using the matrix attached in the question and considering the values in the vertical axis as the actual class, and the values in the horizontal axis the prediction. Then for the Class 1:

  • True Positive = 137 -> samples of class 1, classified as class 1
  • False Positive = 6 -> (1+2+3) samples of classes 2, 3 and 4, but classified as class 1
  • False Negative = 18 -> (13+3+1+1) samples of class 1, but classified as classes 2, 3, 6 and 7
  • Ture Negative = 581 -> (55+1+6...+2+26) The sum of all the values in the matrix except those in column 1 and row 1
Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Your calculation's interpretation of false positive is wrong. It should be: 6 -> (1+2+ 3 ) $\endgroup$
    – Rawnak Yazdani
    Dec 4, 2020 at 12:03
  • 1
    $\begingroup$ yes! correct @RAWNAKYAZDANI. Thanks for pointing out $\endgroup$
    – Ignacio Alorre
    Dec 4, 2020 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.