A machine learning model is outputting precision and recall for a two-class classification problem (0 and 1) like this:
Confusion matrix:
[[136 21]
41 6]]
Precision: [0.768 0.128]
Recall: [0.866 0.222]
Accuracy: 0.696
There are two measures for both precision and recall: The first measure for the 0 class and the second for 1 class. Is it okay to take the average of these? E.g. precision as a whole is 0.768+0.128 / 2 = 0.448? And similarly with recall?
WARNING: Average of precision/recall is totally different concept from Average Precision(AP) link.
Based on the question, we will talk about the Average of precision and recall.
you are partially correct; if
Predicted 0 , Predicted 1
True 0 [[136 21] [[TP FN]
True 1 41 6]] FP TN]]
Then precision for each class(row) is ( Mi,i / sigma(j) Mji), So for:
class 0: 136/ 136+41 (0.76)
class 1: 6 / 6+21 ( 0.22)
for recall, the same happens, but the denominator will be on rows, i.e. ( Mi,i / sigma(j) Mij)
class 0: 136/ 136+21 (0.86)
class 1: 6 / 6+41 ( 0.12)
Then you can average on each group to have overall precision/recall.
Check Table III of this paper (referred to as Precision_M and recall_M):
More precisely, you are doing macro-averaging.
in code, you can have :
cm = confusion_matrix(labels, predictions)
recall = np.diag(cm) / np.sum(cm, axis = 1)
precision = np.diag(cm) / np.sum(cm, axis = 0)
#overall precision/recall
np.mean(precision)
np.mean(recall)
