Let's say I compute a confusion matrix, in the sense defined here: http://www.gabormelli.com/RKB/Confusion_Matrix

I could easily compute the number of True Negatives (TN), True Positives (TP), False Negatives (FN) and False Positives (FP) but I feel it would be a bit awkward: in my case, all objects belong to at least one class: so a misclassification (e.g. my classifier put an actual "A" in a "B" bin) is not only a FP, it is a FN in the same time. And what if an actual "A+B" object is classified as "B" only: in my current study, it's definitely better than classified in "C" or not classified at all. I have other examples like this...

As you could guess, for me, those are confusing matrix.


1 Answer 1


Although it's not stated directly in your question, I guess you have a multiclass classification problem. As always, there are many performance metrics available, each emphasizes another property of the classifier. My favourite generic metric for this type of problems is multiclass $F1$ metric.

Let's say for a data point $x_i$ the set of true labels is $S_i$, and your classifier output is set of labels $C_i$. Then :

F1 measure on that example is $\frac{2 \cdot |S_i \cap C_i|}{|S_i| + |C_i|}$

And F1 measure on the whole data set is :

$\frac{1}{M} \cdot \sum_{i=1}^{M} \frac{2 \cdot |S_i \cap C_i|}{|S_i| + |C_i|}$

Notice it's 1 iff the classifier is perfect, and drops if the classifier makes either FP or FN mistakes (it dislikes FN mistakes a little more).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.