1
$\begingroup$

I have a few models doing prediction with 4 classes, with the output precision and recall varying with different labels.

For example I have (with the class labels being 0, 1, 2, 3 on the x axis):

enter image description here

I understand from a previous question (Recall and precision in classification) that the differences a model can have at predicting different labels can be related to the cost for different mistakes, but I would like to understand in the context of multilabel classification - why is it the models differ for different labels in precision and recall? Are they just showing that they are better at recognising specific patterns that relate to specific labels? Or is there some way I can investigate further as to why this is happening?

Also, generally speaking, would consistency in precision and recall across labels (for example in my graph it seems the DeepSuperLearner is the most consistent/nearest to a straight line for both) make a model more worthwhile than a model which is for example, really good at labels 1 and 3 but bad/medicore at predicting labels 0 and 2?

$\endgroup$
1

1 Answer 1

0
$\begingroup$

Precision and recall can be easily obtained from a confusion matrix, simply by counting the true positives etc.

For example, think about the following confusion matrix (I took it from the Internet):



Precision and recall are calculated as follows:

${\displaystyle {\text{Precision}}={\frac {tp}{tp+fp}}\,}$

${\displaystyle {\text{Recall}}={\frac {tp}{tp+fn}}\,}$


So, notice that, each label has its tp, fp and fn. For example...

toetouches has tp=0.92, fp=0.08 and fn=0.27

squats has tp=0.73, fp=0.27 and fn=0


So, that means that, in the case of squats, the precision is lower than in the case of toetouches. But, on the other hand, recall is much higher (every time our model says that it is squats, it certainly is. Doesn't happen the same in the case of toetouches)

$\endgroup$
2
  • $\begingroup$ How come a confusion matrix has float numbers? Maybe in some problems you can claim to have half a true positive, but generally I think this example is more confusing than helpful. Could you add a standard confusion matrix with integers and redo your explanation? $\endgroup$ Sep 21, 2022 at 15:57
  • $\begingroup$ you can normalize the values in confusion matrix. Upon normalization you'll get float numbers $\endgroup$
    – Aman Singh
    Oct 17, 2022 at 7:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.