What does it imply if accuracy and recall are the same?

Question

I did a number of machine learning experiments to predict a binary classification. I measured precision, recall and accuracy.

I noticed that my precision is generally quite high, and recall and accuracy are always the same numbers.

I used the following definitions:

$\text{Precision} = \frac{TP}{(TP + FP)}$

$\text{Recall} = \frac{TP}{(TP + FN)}$

$\text{Accuracy} = \frac{(TP + TN)}{(P + N)}$

I have some difficulties to interpret accuracy and recall. What does it mean if these two number are always the same in my case?

Answer 1

I suspect that you're measuring the micro-averages of precision, recall and accuracy for your two classes. If you're doing so instead of considering one class "positive" and the other "negative", you'll always get equal values for Recall and Accuracy, because the values of FP and FN will be always the same (you can check with more details here: http://metaoptimize.com/qa/questions/8284/does-precision-equal-to-recall-for-micro-averaging )

Answer 2

It might be a coincidence.

If we have to say something about it, then it indicates that sensitivity (a.k.a. recall, or TPR) is equal to specificity (a.k.a. selectivity, or TNR), and thus they are also equal to accuracy. TP / P = TN / N = (TP+TN) / (P+N), where P = TP+FN, N = TN+FP.

This means your model is somehow "balanced", that is, its ability to correctly classify positive samples is same as its ability to correctly classify negative samples.

However, the importance of sensitivity and specificity may vary from case to case, so being "balanced" is not necessarily good.

Answer 3

As OP has mentioned, this is just a coincidence. It's highly likely that number of instances in each class is balanced. Recall = TP/P and Acc = (TP + TN)/(P+N), so in your case TP/P = TN/N. This can happen, and is more likely to happen when |P| = |N|

Try following: Print upto 7-8 places of decimal and you may see some difference.

Second try to imbalance the problem. Like set positive class as just 20% of total and let rest be 80%, you should definitely see the difference.

Answer 4

It means TN and FP are close to 0.

Therefore, precision is close to TP/TP=1.

The recall formula doesn't change since neither TP nor FN is close to 0.

Accuracy which is (TP+TN)/(TP+TN+FP+FN) is close to TP/(TP+FN) which is recall.

Having TN and FP close to 0 means that you have an imbalanced dataset with an inverted imbalance compared to the standard for positive and negative. You have very few negative while it is standard to have very few positive when using precision and recall.

Typically, people use precision and recall because TN is extremely common (and makes the accuracy very high) and you don't care much about it. In your case, TP is common and TN is rare.