14
$\begingroup$

I did a number of machine learning experiments to predict a binary classification. I measured precision, recall and accuracy.

I noticed that my precision is generally quite high, and recall and accuracy are always the same numbers.

I used the following definitions:

$\text{Precision} = \frac{TP}{(TP + FP)}$

$\text{Recall} = \frac{TP}{(TP + FN)}$

$\text{Accuracy} = \frac{(TP + TN)}{(P + N)}$

I have some difficulties to interpret accuracy and recall. What does it mean if these two number are always the same in my case?

$\endgroup$

4 Answers 4

14
$\begingroup$

I suspect that you're measuring the micro-averages of precision, recall and accuracy for your two classes. If you're doing so instead of considering one class "positive" and the other "negative", you'll always get equal values for Recall and Accuracy, because the values of FP and FN will be always the same (you can check with more details here: http://metaoptimize.com/qa/questions/8284/does-precision-equal-to-recall-for-micro-averaging )

$\endgroup$
2
  • $\begingroup$ Hmm I'm not so sure about that. If I understand it correctly, my precision values should also be the same as my accuracy and recall values. But it's only accuracy and precision that are the same. Precision is most often way higher. $\endgroup$ May 22, 2014 at 14:26
  • $\begingroup$ Was for me the case. After setting averaging mode to macro in Pytorch the numbers differ. Thanks. $\endgroup$ Jul 2, 2022 at 11:24
3
$\begingroup$

It might be a coincidence.

If we have to say something about it, then it indicates that sensitivity (a.k.a. recall, or TPR) is equal to specificity (a.k.a. selectivity, or TNR), and thus they are also equal to accuracy. TP / P = TN / N = (TP+TN) / (P+N), where P = TP+FN, N = TN+FP.

This means your model is somehow "balanced", that is, its ability to correctly classify positive samples is same as its ability to correctly classify negative samples.

However, the importance of sensitivity and specificity may vary from case to case, so being "balanced" is not necessarily good.

$\endgroup$
0
$\begingroup$

As OP has mentioned, this is just a coincidence. It's highly likely that number of instances in each class is balanced. Recall = TP/P and Acc = (TP + TN)/(P+N), so in your case TP/P = TN/N. This can happen, and is more likely to happen when |P| = |N|

Try following: Print upto 7-8 places of decimal and you may see some difference.

Second try to imbalance the problem. Like set positive class as just 20% of total and let rest be 80%, you should definitely see the difference.

$\endgroup$
2
  • $\begingroup$ Uhm, I was shocked at first to see that recall always equals accuracy in my case, until I see this question. Mine is a multi-class imbalanced classification: Class0: 28.71%, Class1: 3.04%, Class2: 14.33%, Class3: 50.65%, Class4: 3.27% RESULTS: Accuracy: 0.5060665102067927, Recall: 0.5060665102067927, F1: 0.3403620268955662 $\endgroup$
    – arilwan
    Sep 2, 2020 at 10:37
  • $\begingroup$ Can you provide any references for your suggestion? $\endgroup$ Nov 19, 2021 at 22:26
0
$\begingroup$

It means TN and FP are close to 0.

Therefore, precision is close to TP/TP=1.

The recall formula doesn't change since neither TP nor FN is close to 0.

Accuracy which is (TP+TN)/(TP+TN+FP+FN) is close to TP/(TP+FN) which is recall.

Having TN and FP close to 0 means that you have an imbalanced dataset with an inverted imbalance compared to the standard for positive and negative. You have very few negative while it is standard to have very few positive when using precision and recall.

Typically, people use precision and recall because TN is extremely common (and makes the accuracy very high) and you don't care much about it. In your case, TP is common and TN is rare.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.