Despite having seen these terms 502847894789 times, I cannot for the life of me remember the difference between sensitivity, specificity, precision, accuracy, and recall. They're pretty simple concepts, but the names are highly unintuitive to me, so I keep getting them confused with each other. What is a good way to think about these concepts so the names start making sense?
Put another way, why were these names chosen for these concepts, as opposed to some other names?
Personally I remember the difference between precision and recall (a.k.a. sensitivity) by thinking about information retrieval:
For precision and recall, each is the true positive (TP) as the numerator divided by a different denominator.
Mnemonics neatly eliminate man’s only nemesis: insufficient cerebral storage.
There is SNOUT SPIN:
I imagine a pig spinning around in a centrifuge, perhaps in preparation for going into space, to help me remember this mnemonic. Humming the theme to Tail Spin with the words appropriately changed can help the musically inclined from a certain generation.
I am not aware of any others.
I agree that the terms are very non-intuitive and hard to correspond to their formulas. Here are some diagrams with the mnemonic tricks that I have developed, and now I have them all solidly memorized.
Classification matrix
First, here are the mnemonics summarized in two common variations of the classification matrix (also called confusion matrix). My mnemonics work for either variation, so just focus on the variation of the matrix with which you are more familiar.
Note:
Short version of mnemonics
Here is the short form of my mnemonics; the details below explain the logic underlying them, which should help in memorizing what they mean:
Detailed explanation of logic underlying the mnemonics, corresponding to their intrinsic meaning
Accuracy: overall results
Accuracy is actually quite intuitive and usually presents no difficulty in memorization. It is simply the correct (true) predictions divided by all predictions, whether true or false.
$$ Accuracy = \frac{correct\:predictions}{all\:predictions} = \frac{TP + TN}{TP + TN + FP + FN} $$
Precision and Recall: focus on true positives
The essence of precision and recall is that they both consider the proportion of true positive results; that is, they are two different ways of measuring how many times the model correctly guessed the class of interest (that is, the positive class). So, TP is always the numerator. The difference between them is in the denominator: whereas precision considers all the values that were predicted to be positive (whether correctly or not), recall considers all the values that actually are positive (whether correctly predicted or not).
Precision is the proportion of positive predictions that were correct: $$ Precision = \frac{true\:positive}{\pmb P\pmb R\pmb E dicted\:positive} = \frac{TP}{TP + FP} $$
Recall is the proportion of real or actual positives that were predicted correctly: $$ Recall = \frac{true\:positive}{\pmb R\pmb E\pmb A l\:positive} = \frac{TP}{TP + FN} $$
To differentiate them, you can remember:
Sensitivity and Specificity: focus on correct predictions
The essence of sensitivity and specificity is that they both focus on the proportion of correct predictions. So, the numerator is always a measure of true predictions and the denominator is always all the total of corresponding predictions of that class. Whereas sensitivity measures the proportion of correctly predicted positives out of all actual positive values, specificity measures the proportion of correctly predicted negatives out of all actual negative values.
Sensitivity is the proportion of actual positives that were correctly predicted: $$ Sensitivity = \frac{true\:\pmb Positive}{real\:\pmb Positive} = \frac{TP}{TP + FN} $$ (Note that the although the formulas for Recall and Sensitivity are mathematically identical, when recall is paired with precision and sensitivity is paired with specificity, the interpretations and applications of the two measures are rather different.)
Specificity is the proportion of actual negatives that were correctly predicted: $$ Specificity = \frac{true\:\pmb Negative}{real\:\pmb Negative} = \frac{TN}{TN + FP} $$
To remember which is which, remember "snip and spin", but note that the letters P and N are swapped (that is, they spin around and the ends of the long names are snipped):
In the context of binary classification:
Accuracy - How many instances did the model label correctly?
Recall - How often was the model able to find positives?
Precision - How believable the model is when it says an instance is a positive?
The following article helps me a lot
https://medium.com/swlh/how-to-remember-all-these-classification-concepts-forever-761c065be33
I created an interactive confusion table to help me understand the difference between these terms: http://zyxue.github.io/2018/05/15/on-the-p-value.html#interactive-confusion-table. I post the link here in case someone may find it helpful, too.
I'll try and explain how I remember what recall is.
Definition: Recall = True positives/All real world positives. OR Recall = True positives/True Positives and False Negatives.
Imagine an automobile company that wants to recall some of its cars for a manufacturing defect (hard to imagine, right?). This company obviously wants to get in all the cars that have the issue. That's our denominator. The total number of faulty cars.
It may indeed get hold of all of them, by calling every single car it ever manufactured. So here, it's recall would be perfect, a value of 1. There cannot be a false negative (part two of the denominator) since we labelled everything as positive!
In this case, the owner is obviously a multi-billionaire who doesn't care about the cost of the *recall exercise.
But what if a corporate entity wanted to cut costs (again, just go with me on this) by getting only the faulty cars in. Well, then, they would want to figure out something like, let's only call in cars that were manufactured in January this year as they have the maximum chances of this problem.
This creates our false negatives, that is, cars that have the problem but do not meet the January criterion. Therefore the second part of the denominator (FN) now becomes non-zero, which then reduces the overall fraction.
Key takeaways - it is the false negatives that fiddle with the recall metric. The mnemonic, if you really need it, is that cars get recalled. Hope this helps, somewhat.
I had a similar problem and came across Andrew Ng's slide, which I found helpful, although there are good answers here as well.
As highlighted by other answers the key is remembering the confusion matrix.
Positives are on the first row and negatives are on the bottom row.
Andrew Ng Explanation:
For both precision and recall, True Positive is on top of the division symbol (i.e. numerator). For the Prescison we divide this by Predicted positive (which is the first row) and for reCAll, we divide this by ACtual positive (which is the first column)
High precision would mean that if a diagnosis of patients
have that rare disease, probably the patient does have it and it's an
accurate diagnosis. High recall means that if there's a patient with
that rare disease, probably the algorithm
will correctly identify that they do
have that disease.
Microsoft explanation:
Precision is the ability of a model to avoid labeling negative samples as positive (by looking at the above eq, precision formula, we need the false positive to be zero meaning do not tell people that they have the rare disease)
Recall is the ability of a model to detect all positive samples (by looking at the above eq, recall formula, we need the false negative to be zero meaning do not miss patients who have the rare disease)
I use the word TARP to remember the difference between accuracy and precision.
TARP: True=Accuracy, Relative=Precision.
Accuracy measures how close a measurement is to the TRUE value, as the standard/accepted value is the TRUTH.
Precision measures how close measurements are RELATIVE to each other, or how low the spread between various measurements is.
Accuracy is truth, precision is relativity.
Hope this helps.
Specificity tackles false positive. High specificity means a low false-positive rate. (Specificity = 1 - false-positive rate)
Sensitivity tackles false negative. High sensitivity means a low false-negative rate. (Sensitivity = 1 - false-negative rate)
That's why specificity is also called true negative rate, and sensitivity is also called true positive rate. The reason why we don't call them these ways is probably because the term "true negative rate" can be misleading to laymen as the denominator can be confusing. true negative rate = true negatives/actual negatives, NOT predicted negatives. The same goes for "true positive rate".
P.S. The answer above is tightly connected with the mnemonics "spout" and "spin", but I think it makes the mnemonics more understandable, plus I don't need to remember two extra words.
I have recently published an article in the Journal of Classification that deals with exactly this question. Take a look at https://rdcu.be/dL1wK In short, instead of using those confusing terms, use more intuitive ones that won’t leave you confused.
