2
$\begingroup$

I just watched a video claiming that for a test for property X that has a 99% success rate, candidate A tests positive, but the probability of candidate A having property X is NOT 99%.

Instead I am supposed to consider the probability of any given candidate from the population having property X. The video uses 0.1% as an example.

Giving:

$$\frac{0.99 * 0.001}{0.001 * 0.99 + 0.999 * 0.01}$$

Which comes out around 9% for the chances of candidate A having property X.

It makes no sense to me! I know it's Bayes Law, but I don't understand the need to consider the probability for an arbitrary member of the population having property X.

I would have thought the following argument is sufficient to describe the situation:

  • A test for property X has a 99% success rate
  • Candidate A tests positive for property X
  • Therefore the probability of candidate A having property X is 99%.

Why am I wrong?

Regardless of the chances of candidate A being tested, or of any random person having property X, in this set up it is candidate A specifically who tested positive.

Why does the probability of any given candidate from the population having property X affect the probability for Candidate A, who we know was tested and who tested positive? Isn't being part of the group who were tested enough to put the candidate under the influence of the probabilities at play within that group?


Below is an attempt at reasoning this through with actual numbers - it ends in a contradiction, but may help to illustrate where my confusion lies.

Using hypothetical numbers to illustrate this hypothetical situation.

  • Assume a population of 10,000 candidates.
  • From prior knowledge, 10 people in the population have property X (0.001 * 10,000)
  • We test all 10,000 candidates for property X
  • Of these, assume 1000 tested positive (arbitrary value to facilitate reasoning)
  • From the group who tested positive, 990 have property X and 10 don't.

OK, so I just picked some numbers and ran with it - but this example seems to show that the prior knowledge must be wrong anyway, as we have 990 true positives, whereas prior knowledge was that there were 10 candidates with property X. (Let's assume the 99% success rate for the test has been rigorously validated in multiple independent studies.)

Candidate A was one of the 1000 from the population who tested positive. What is the relevance of the fact that we tested 10,000 people in total or that in that population a certain prevalence of property X exists?

$\endgroup$
2
  • 2
    $\begingroup$ You’re confusing $p(\text{has-condition} \mid \text{test-positive})$ with $p(\text{test-positive} \mid \text{has-condition})$. To say that the test is 99% effective means $p(\text{test-positive} \mid \text{has-condition}) = 0.99$. You’re asked to reason in the other direction, though: given that I tested positive, what is the probability that I have the condition? $\endgroup$ Commented Mar 27, 2021 at 21:17
  • $\begingroup$ Your penultimate bullet point is where you go wrong. It's very unlikely that you would get as many as 1000 positive test results - on average you would get 9990 * 0.01 + 10 * 0.99 = 109.8 positive test results, composed of on average 99.9 false positives and only 9.9 true positives. Similar question here: stats.stackexchange.com/questions/513324/… $\endgroup$
    – fblundun
    Commented Mar 27, 2021 at 22:08

1 Answer 1

2
$\begingroup$

An extreme example: suppose that the property X is "is infected with smallpox", and the test is from before smallpox was eradicated. In the present day, nobody has smallpox. But 1% of people without smallpox test positive, so if you administered the test to everybody in the world, millions would get positive results. Should those people believe they have a 99% chance of having smallpox?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.