9
$\begingroup$

I did a number of machine learning experiments to predict a binary classification. I measured precision, recall and accuracy.

I noticed that my precision is generally quite high, and recall and accuracy are always the same numbers.

I used the following definitions:

$\text{Precision} = \frac{TP}{(TP + FP)}$

$\text{Recall} = \frac{TP}{(TP + FN)}$

$\text{Accuracy} = \frac{(TP + TN)}{(P + N)}$

I have some difficulties to interpret accuracy and recall. What does it mean if these two number are always the same in my case?

$\endgroup$
8
$\begingroup$

I suspect that you're measuring the micro-averages of precision, recall and accuracy for your two classes. If you're doing so instead of considering one class "positive" and the other "negative", you'll always get equal values for Recall and Accuracy, because the values of FP and FN will be always the same (you can check with more details here: http://metaoptimize.com/qa/questions/8284/does-precision-equal-to-recall-for-micro-averaging )

$\endgroup$
1
  • $\begingroup$ Hmm I'm not so sure about that. If I understand it correctly, my precision values should also be the same as my accuracy and recall values. But it's only accuracy and precision that are the same. Precision is most often way higher. $\endgroup$ – RoflcoptrException May 22 '14 at 14:26
2
$\begingroup$

It might be a coincidence.

If we have to say something about it, then it indicates that sensitivity (a.k.a. recall, or TPR) is equal to specificity (a.k.a. selectivity, or TNR), and thus they are also equal to accuracy. TP / P = TN / N = (TP+TN) / (P+N), where P = TP+FN, N = TN+FP.

This means your model is somehow "balanced", that is, its ability to correctly classify positive samples is same as its ability to correctly classify negative samples.

However, the importance of sensitivity and specificity may vary from case to case, so being "balanced" is not necessarily good.

$\endgroup$
0
$\begingroup$

As OP has mentioned, this is just a coincidence. It's highly likely that number of instances in each class is balanced. Recall = TP/P and Acc = (TP + TN)/(P+N), so in your case TP/P = TN/N. This can happen, and is more likely to happen when |P| = |N|

Try following: Print upto 7-8 places of decimal and you may see some difference.

Second try to imbalance the problem. Like set positive class as just 20% of total and let rest be 80%, you should definitely see the difference.

$\endgroup$
1
  • $\begingroup$ Uhm, I was shocked at first to see that recall always equals accuracy in my case, until I see this question. Mine is a multi-class imbalanced classification: Class0: 28.71%, Class1: 3.04%, Class2: 14.33%, Class3: 50.65%, Class4: 3.27% RESULTS: Accuracy: 0.5060665102067927, Recall: 0.5060665102067927, F1: 0.3403620268955662 $\endgroup$ – arilwan Sep 2 '20 at 10:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.