71
$\begingroup$

Despite having seen these terms 502847894789 times, I cannot for the life of me remember the difference between sensitivity, specificity, precision, accuracy, and recall. They're pretty simple concepts, but the names are highly unintuitive to me, so I keep getting them confused with each other. What is a good way to think about these concepts so the names start making sense?

Put another way, why were these names chosen for these concepts, as opposed to some other names?

$\endgroup$
6
  • 4
    $\begingroup$ The best way to remember is to recall a real life study where this or that characteristic was in the focus. I.e. contextual flesh helps. $\endgroup$ – ttnphns Oct 31 '14 at 19:37
  • 4
    $\begingroup$ For me, the best way to remember these concepts is through the 2×2 contingency table within the Wikipedia link. $\endgroup$ – Randel Oct 31 '14 at 21:22
  • 1
    $\begingroup$ @ttnphns: "contextual flesh" is a great typo! $\endgroup$ – amoeba Oct 31 '14 at 23:26
  • 5
    $\begingroup$ Recall is sensitivity, there you have one fewer to deal with. :) $\endgroup$ – Penguin_Knight Nov 1 '14 at 1:24
  • 1
    $\begingroup$ Just to keep it here, this post offers a nice explanation: uberpython.wordpress.com/2012/01/01/… $\endgroup$ – Maxim.K Nov 18 '17 at 20:51

11 Answers 11

16
$\begingroup$

Personally I remember the difference between precision and recall (a.k.a. sensitivity) by thinking about information retrieval:

  • Recall is the fraction of the documents that are relevant to the query that are successfully retrieved, hence its name (in English recall = the action of remembering something).
  • Precision is the fraction of the documents retrieved that are relevant to the user's information need. Somehow you take a few shots and if most of them got their target (relevant documents) then you have a high precision, regardless of how many shots you fired (number of documents that got retrieved).
$\endgroup$
1
  • $\begingroup$ When you say "regardless of how many shots you fired (number of documents that got retrieved)", it seems that the precision is independant of the number of total positive prediction, but : Precision=TP/Predicted Positive. $\endgroup$ – Xiaoshi May 23 at 13:32
47
$\begingroup$

For precision and recall, each is the true positive (TP) as the numerator divided by a different denominator.

  • Precision: TP / Predicted positive
  • Recall: TP / Real positive
$\endgroup$
1
  • 1
    $\begingroup$ TP/TN/FN/FP are well-named, self-explanatory if you remember that True/False refers to the correctness and Positive/Negative is the real world truth. $\endgroup$ – user4446237 Jan 4 at 16:49
16
$\begingroup$

Mnemonics neatly eliminate man’s only nemesis: insufficient cerebral storage.

There is SNOUT SPIN:

  • A Sensitive test, when Negative rules OUT disease
  • A Specific test, when Positive, rules IN a disease.

I imagine a pig spinning around in a centrifuge, perhaps in preparation for going into space, to help me remember this mnemonic. Humming the theme to Tail Spin with the words appropriately changed can help the musically inclined from a certain generation.

I am not aware of any others.

$\endgroup$
1
  • $\begingroup$ The SNOUT and SPIN rules are deceptively simple. You really ought to have good estimates of sensitivity, specificity, and prevalence before putting your trust in a positive or negative test result, no matter how sensitive or specific the test is. Check out this website: kennis-research.shinyapps.io/Bayes-App. For example, entering a prevalence of 5 per 1,000. sensitivity = .90, specificity = .99 produces (via Bayes Rule) a relatively low positive predictive value of .2857. $\endgroup$ – RobertF Sep 29 '16 at 16:08
7
$\begingroup$

In the context of binary classification:

Accuracy - How many instances did the model label correctly?

Recall - How often was the model able to find positives?

Precision - How believable the model is when it says an instance is a positive?

$\endgroup$
3
$\begingroup$

I created an interactive confusion table to help me understand the difference between these terms: http://zyxue.github.io/2018/05/15/on-the-p-value.html#interactive-confusion-table. I post the link here in case someone may find it helpful, too.

$\endgroup$
3
$\begingroup$

The following article helps me a lot

https://medium.com/swlh/how-to-remember-all-these-classification-concepts-forever-761c065be33

enter image description here

  • accuracy: Double-A rule
  • precision: Triple-P rule
$\endgroup$
1
$\begingroup$

I'll try and explain how I remember what recall is.

Definition: Recall = True positives/All real world positives. OR Recall = True positives/True Positives and False Negatives.

Imagine an automobile company that wants to recall some of its cars for a manufacturing defect (hard to imagine, right?). This company obviously wants to get in all the cars that have the issue. That's our denominator. The total number of faulty cars.

It may indeed get hold of all of them, by calling every single car it ever manufactured. So here, it's recall would be perfect, a value of 1. There cannot be a false negative (part two of the denominator) since we labelled everything as positive!

In this case, the owner is obviously a multi-billionaire who doesn't care about the cost of the *recall exercise.

But what if a corporate entity wanted to cut costs (again, just go with me on this) by getting only the faulty cars in. Well, then, they would want to figure out something like, let's only call in cars that were manufactured in January this year as they have the maximum chances of this problem.

This creates our false negatives, that is, cars that have the problem but do not meet the January criterion. Therefore the second part of the denominator (FN) now becomes non-zero, which then reduces the overall fraction.

Key takeaways - it is the false negatives that fiddle with the recall metric. The mnemonic, if you really need it, is that cars get recalled. Hope this helps, somewhat.

$\endgroup$
0
$\begingroup$

I use the word TARP to remember the difference between accuracy and precision.

TARP: True=Accuracy, Relative=Precision.

Accuracy measures how close a measurement is to the TRUE value, as the standard/accepted value is the TRUTH.

Precision measures how close measurements are RELATIVE to each other, or how low the spread between various measurements is.

Accuracy is truth, precision is relativity.

Hope this helps.

$\endgroup$
0
$\begingroup$

I agree that the terms are very non-intuitive and hard to correspond to their formulas. Here are the mnemonic tricks that I have developed.

Note:

  • TP = true positives
  • TN = true negatives
  • FP = false positives
  • FN = false negatives

Here is the short form of my mnemonics; the details below explain the logic underlying them, which should help in memorizing what they mean:

  • Accuracy: correct predictions divided by all predictions: TP+TN/(TP+FP+FP+FN)
  • Precision and Recall: focus on true positives
    • PREcision is TP divided by PREdicted positive: TP/(TP+FP)
    • REcAll is TP divided by REAl positive: TP/(TP+FN)
  • Sensitivity and Specificity: focus on correct predictions
    • SNIP (SeNsitivity Is Positive): TP/(TP+FN)
    • SPIN (SPecificity Is Negative): TN/(TN+FP)

The details that follow provide the underlying logic for these, based on the essential meaning of each measure.

Accuracy: overall results

Accuracy is actually quite intuitive and usually presents no difficulty in memorization. It is simply the correct (true) predictions divided by all predictions, whether true or false.

$$ Accuracy = \frac{correct\:predictions}{all\:predictions} = \frac{TP + TN}{TP + TN + FP + FN} $$

Precision and Recall: focus on true positives

The essence of precision and recall is that they both consider the proportion of true positive results; that is, they are two different ways of measuring how many times the model guessed right. So, TP is always the numerator. The difference between them is in the denominator: whereas precision considers all the values that were predicted to be positive (whether correctly or not), recall considers all the values that actually are positive (whether correctly predicted or not).

Precision is the proportion of positive predictions that were correct: $$ Precision = \frac{true\:positive}{\pmb P\pmb R\pmb E dicted\:positive} = \frac{TP}{TP + FP} $$

Recall is the proportion of real or actual positives that were predicted correctly: $$ Recall = \frac{true\:positive}{\pmb R\pmb E\pmb A l\:positive} = \frac{TP}{TP + FN} $$

To differentiate them, you can remember:

  • PREcision is TP divided by PREdicted positive
  • REcAll is TP divided by REAl positive

Sensitivity and Specificity: focus on correct predictions

The essence of sensitivity and specificity is that they both focus on the proportion of correct predictions. So, the numerator is always a measure of true predictions and the denominator is always all the total of corresponding predictions of that class. Whereas sensitivity measures the proportion of correctly predicted positives out of all actual positive values, specificity measures the proportion of correctly predicted negatives out of all actual negative values.

Sensitivity is the proportion of actual positives that were correctly predicted: $$ Sensitivity = \frac{true\:\pmb Positive}{real\:\pmb Positive} = \frac{TP}{TP + FN} $$ (Note that the although the formulas for Recall and Sensitivity are mathematically identical, when recall is paired with precision and sensitivity is paired with specificity, the interpretations and applications of the two measures are rather different.)

Specificity is the proportion of actual negatives that were correctly predicted: $$ Specificity = \frac{true\:\pmb Negative}{real\:\pmb Negative} = \frac{TN}{TN + FP} $$

To remember which is which, remember "snip and spin", but note that the letters P and N are swapped (that is, they spin around and the ends of the long names are snipped):

  • SNIP (SeNsitivity Is Positive)
  • SPIN (SPecificity Is Negative)
$\endgroup$
0
$\begingroup$

Specificity tackles false positive. High specificity means a low false-positive rate. (Specificity = 1 - false-positive rate)

Sensitivity tackles false negative. High sensitivity means a low false-negative rate. (Sensitivity = 1 - false-negative rate)

That's why specificity is also called true negative rate, and sensitivity is also called true positive rate. The reason why we don't call them these ways is probably because the term "true negative rate" can be misleading to laymen as the denominator can be confusing. true negative rate = true negatives/actual negatives, NOT predicted negatives. The same goes for "true positive rate".

P.S. The answer above is tightly connected with the mnemonics "spout" and "spin", but I think it makes the mnemonics more understandable, plus I don't need to remember two extra words.

$\endgroup$
-2
$\begingroup$

https://newbiettn.github.io/2016/08/30/precision-recall-sensitivity-specificity/

for further study see this link

https://newbiettn.github.io/2016/08/30/precision-recall-sensitivity-specificity/

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.