0
$\begingroup$

A machine learning model is outputting precision and recall for a two-class classification problem (0 and 1) like this:

Confusion matrix:
[[136  21]
  41   6]]

Precision: [0.768 0.128]
Recall: [0.866 0.222]
Accuracy: 0.696

There are two measures for both precision and recall: The first measure for the 0 class and the second for 1 class. Is it okay to take the average of these? E.g. precision as a whole is 0.768+0.128 / 2 = 0.448? And similarly with recall?

$\endgroup$
1
  • $\begingroup$ Can you share the code outputting these numbers? or just confirm that these are from sklearn. also are these two ok/not ok predictions or conceptually these are different? $\endgroup$
    – Sadra
    Apr 3, 2022 at 15:16

1 Answer 1

1
$\begingroup$

WARNING: Average of precision/recall is totally different concept from Average Precision(AP) link.

Based on the question, we will talk about the Average of precision and recall.

you are partially correct; if

      Predicted 0 , Predicted 1 
True 0 [[136             21]     [[TP  FN]
True 1   41              6]]       FP  TN]]

Then precision for each class(row) is ( Mi,i / sigma(j) Mji), So for:

class 0: 136/ 136+41 (0.76)

class 1: 6 / 6+21 ( 0.22)

for recall, the same happens, but the denominator will be on rows, i.e. ( Mi,i / sigma(j) Mij)

class 0: 136/ 136+21 (0.86)

class 1: 6 / 6+41 ( 0.12)

Then you can average on each group to have overall precision/recall.

Check Table III of this paper (referred to as Precision_M and recall_M):

enter image description here

More precisely, you are doing macro-averaging.

in code, you can have :

cm = confusion_matrix(labels, predictions)
recall = np.diag(cm) / np.sum(cm, axis = 1)
precision = np.diag(cm) / np.sum(cm, axis = 0)

#overall precision/recall
np.mean(precision)
np.mean(recall)

Readmore

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.