# Understanding Medical Testing

While reading a textbook on probability I came across a question:

There is a probability that a certain percentage of the population has a disease. Then a test for this disease could have a certain specificity (let's say .98) and sensitivity (let's say .99).

How do we know what percentage of population will have a disease, unless we test it? What other information/knowledge is used to estimate the percentage of population which will contract a disease?

• I suspect the textbook's subsequent argument did not rely on knowing what proportion of the population has a disease, but required only that such a proportion exists.
– whuber
Jul 12, 2020 at 16:57
• Sensitivity is based on info gained from testing subjects known to have the disease, and specificity on info from testing subjects known not to have the disease. Prevalence is based on the entire population. Bayes' Theorem allows computation of conditional probability P(Disease | Pos Test), given prevalence, sensitivity, and specificity. // Estimating prevalence via screening tests can be challenging, esp. with a new disease and recently developed tests. Perhaps see this. Jul 12, 2020 at 18:26
• @whuber In a way it did, because we were supposed to calculate the predictive power of the test and it was dependent on the proportion of population having the disease. Jul 14, 2020 at 20:33
• Yes, but that still doesn't require knowing the proportion: the proportion is only given hypothetically and that's enough to do the calculation.
– whuber
Jul 14, 2020 at 20:59
• @whuber Yes the hypothetical proportion might help with solving the textbook question, but my question was more about reality, how do we find this proportion in real life, as this proportion affects the predictive power of the test. Jul 14, 2020 at 21:41

You are correct to suspect that prevalence of a disease is sometimes estimated from test results.

Denote prevalence $$\pi = P(D),$$ sensitivity $$\eta = P(+|D),$$ specificity $$\theta = P(-|D^c),$$ and the probability of a positive test in the population as $$\tau = P(+).$$ Then $$\tau = \pi\eta + (1-\pi)(1-\theta).$$ Upon solving for $$\pi,$$ this implies $$\pi = \frac{\tau+\theta -1}{\eta+\theta - 1}.$$ So if you get the proportion $$t = a/n$$ of positive tests among $$n$$ randomly selected members of the population, you can estimate $$\tau$$ by $$t$$ and $$\pi$$ by $$p = \frac{t+\theta -1}{\eta+\theta - 1}.$$

If you want a confidence interval for $$\pi,$$ begin by getting the usual binomial confidence interval for $$\tau$$ and then use the second displayed equation on the endpoints of the CI for $$\tau$$ to get endpoints of a CI for $$\pi.$$

Note: Unfortunately, for tests with poor sensitivity or specificity or for prevalence $$\pi$$ near to $$0$$ or $$1,$$ the CI for $$\pi$$ can have nonsensical endpoints outside of $$(0,1).$$ Then a Gibbs sampler may provide a useful Bayesian probability interval for the prevalence $$\pi$$ of the disease.

• Thank you for the detailed mathematics, it helps. As you mention, sometimes we estimate prevalence of a disease from test result, can you please explain in simpler terms that how do we estimate what proportion of the population will be infected without testing? It seems confusing as without testing we can't know who is infected and without knowing the proportion of infected population we can't measure how good a testing method is. Jul 14, 2020 at 20:36
• With some diseases physicians can easily identify a high proportion of infected subjects based on overt symptoms (without needing a test): chicken pox, whooping cough, mumps, measles, hepatitis. // If by 'simpler terms' you mean a trivial appeal to intuition, then you should probably abandon that quest. The interplay of conditional probabilities referring to distinct subpopulations makes this an inherently nonintuitive topic. // As widespread nonsensical declarations about Covid-19 have shown, even some health professionals lack reliable intuition on interpretation of medical screening tests. Jul 14, 2020 at 20:49
• Thank you very much, really appreciate it, although it seems a bit discouraging, but it makes me aware of the reality, how hard sometimes doing actual science is.I will accept your answer. Cheers! Jul 14, 2020 at 20:55
• Don't give up. With some time to get familiar with a few specific applications of medical screening tests and careful attention to the implications of sensitivity and specificity, you can be among the (apparently small) fraction of professionals who understand the issues. Not impossible, just not immediately trivial. Jul 14, 2020 at 21:48
• @BruceET: "With some diseases physicians can easily identify a high proportion of infected subjects based on overt symptoms (without needing a test)" I'd say a physician judging symptoms is a test, certainly in the context of discussing statistics of medical testing. Jul 15, 2020 at 12:57

You never actually know the truth.

Looked at one way, the logic of test accuracy, the 2×2 table, sensitivity, specificity, positive predictive value and all that is predicated upon a testing method under study compared to a "gold standard". For example, an oncology test based on lab work for a urinary biomarker might be compared to a specific protocol for biopsy and histology of related tissue. Of course, the protocol for biopsy and histology itself has some sensitivity and some specificity! In practice, though we might take such a protocol as ground truth, and rely on it as a gold standard... at least until something we feel is better can come along.

Looked at another way, the logic of test accuracy, the 2×2 table, sensitivity, specificity, positive predictive value and all that, irrespective of the quality or existence of any "gold standard", helps us organize our understanding of trade-offs, and the behavior of tests. For example, regardless of how close we can actually come to knowing The Truth of whatever we are trying to test, we can know that making the definition of a positive test easier necessarily increases our false positives (i.e. of $$\alpha$$ error/Type I error). Likewise, this logic can help us understand that positive predictive value of a test tanks when the prevalence of the thing we are testing for is tiny.

• Thank you this adds more clarity! Jul 16, 2020 at 22:28
• "You never actually know the truth." well, there are a number of scenarios where we can know the truth - if we're willing to wait long enough (e.g. by post-mortem autopsy, or by long-time follow-up). I.e., there is a difference between what we can know when measuring how well a test does (where we may wait) and between a diagnostic situation where we need to decide now what to do with the patient. Jul 17, 2020 at 21:27
• @cbeleitesunhappywithSX Looks like you have solved longstanding epistemological questions! ;) Jul 17, 2020 at 22:52
• @cbeleitesunhappywithSX I just came across your interesting comment regarding the notion that if we're welling to wait long enough (e.g. by post-mortem autopsy...) [we will know the truth]. It reminds me of a "fact" long-accepted by scientists regarding a specific disease and the belief that it was correlated with a difference in size of a specific brain region. Long story short, it turned out scientists had been performing the measurements incorrectly, so all the "facts" that were believed for years (and the theories based thereof) were incorrect. [continued...] Oct 9, 2022 at 4:57
• [...continued] So "long enough" turned out to be decades longer than what everyone thought! Oct 9, 2022 at 4:57