I reckon that utilizing Bayesian method often produces counterintuitive results, but the example below seems counterintuitive in an extent too extreme for me.

Suppose there is a specific disease and a company is trying to make a kit which can detect it, and the probability of such case is as below. ( "+" indicates that the kit has showed positive.)

got Disease did not get Disease
+ .009 .099
- .001 .891

Then, the equations go

P(+|D)= 0.9 [which indicates that the performance of the kit is decent in the perspective of the company];

yet, P(D|+)= 0.083 [which indicates that an individual who got the positive result do not have any reasonable evidence to believe she actually has a disease.]

I understand that this dilemma appeared because of the huge asymmetry between the probabilities, and I can grasp it mathematically by using diagrams. In addition, one might say that the probability of getting disease is updated and increased from .009 to .083 due to the happening of event (+).

However, considering that social science needs to interpret the numeric statistical results into a logical and intuitive explanations, the latter result to me is utterly unexplainable.

While the former result implies to the kit manufacturer that they have invented reliable kit, the latter - which is what we as individuals face - may intuitively tell people - say, those who are diagnosed positive with COVID test - that the possibility that they actually got infected is under 10%. On the basis of this explanation, perhaps there is no need for one to think that they should be hospitalized or get medically treated just because they got + sign on their kit.

Can this gap between mathematical result and logical(i.e. intuitive) explaination be harmonized?

  • 3
    $\begingroup$ Frank Harrell likes to talk about these “backwards in time” probabilities and why the sensitivity score you mention is not as useful as people think. Check out his blog. $\endgroup$
    – Dave
    Mar 11, 2021 at 2:05
  • 2
    $\begingroup$ BTW this involves Bayes law, not Bayesian method. $\endgroup$
    – carlo
    Mar 12, 2021 at 9:14

3 Answers 3


To understand why such an "unreliable" test is still useful for society, and hence, the individuals, you need to take into account the alternative: no widely available test. Also, keep in mind that risk analysis is a composition of the probability of something happening, and the extent of the consequences. If the consequences are severe we consider it a large risk even though the probability is low.

There are a lot of similar tests in medicine, for example for breast cancer detection. The first examination is ultrasound/mammogram, with similar numbers as in your example. What happens with those who receive the "+"? They don't start treatment, but a needle biopsy (which is more invasive, expensive and time consuming) with better sensitivity and specificity. Those who receive the "+" will go for a third examination, removal of the tumour, and if the final diagnosis is cancer, then treatment starts. This is very simplified, of course, but reflects bigger picture. The point being that the first test is constructed to have high sensitivity, and still be widely available, so it comes with the expense of low specificity.

Let's stick to your Covid example. The "+" test results will not hospitalise/treat an otherwise healthy person. Why do it at all? Let's have a look at the alternative.

Scenario 0: No widely available test, and only those with severe symptoms get a test before treatment starts. The proportion of people with Covid is 1%, $P(Cov) = 0.01$, and to stop the Covid to spread, all people have to live under restrictions.

Scenario 1: A widely available test with high sensitivity and low specificity. People with no/mild symptoms can be put into two groups: "+" and "-". In the "+" group, $P(Cov|+) = 0.083$, the proportion of Covid is 8.3%, and this group is given severe restrictions. In the "-" group, $P(Cov|-) = 0.0011$, the proportion of Covid is 0.11%, and this group can have less severe restrictions.

So, instead of all people living under restriction, you now get, $P(+) = 0.108$, 10.8% of the population that will have more severe restrictions, and $P(-) = 0.892$, 89.2% of the population that will have less severe restrictions. In the "+" group, $P(no Cov|+) = 0.917$, 91.7% of the people are living under severe restriction although they don't have Covid. This perhaps seems unnecessary, but the alternative is all of us living under restrictions, 100% of the population as opposed to 9.9% of the population.

The reality of Covid restrictions are of course complicated by a lot of things, among them that not the entire population is tested.

  • $\begingroup$ If a test is A: not at all reliable, and B: fuels behaviour/policy which may cause significant collateral harm, how is this better than no test at all? Isn't that just down to societal pressure to be seen to be "doing something" (or even to commercial incentives) when in fact not doing mass testing may be the less harmful path? $\endgroup$ Mar 28, 2021 at 9:40

You say that the high sensitivity, $P(+\vert D)$, means that the company should be happy with its covid test. I disagree. That’s part of the goal, but if all you want is a super sensitive test, just call everyone covid-positive and never miss a case.

The company blew it when it comes to specificity. The test will call most anything a covid case, so there’s no reason to have faith in the positive result, hence the low $P(D\vert+)$.

(Quick calculations, mostly in my head, say that your test also has $90\%$ specificity, but you get $99\%$ specificity just by knowing the proportion of patients with covid, so that specificity is awful.)

  • $\begingroup$ I now understand there is some kind of tradeoff between sensitivity and specificity. But returning to the issue of "explaining the result with logical and intuitive terms", wouldn't it be more sensible to give the result of P(D|+) rather than P(+|D) to individuals who must make their decisions after the test? The kit manufacturer may examine both results of sensitivity and specificity, but giving the result of P(+|D) would be futile to the ones who are holding their test results at their hand. $\endgroup$
    – HYL
    Mar 11, 2021 at 8:14
  • $\begingroup$ I get that the performance of the kit has two distinctive aspects thanks to your help, but what still bothers me is the difficulty of explaining this conundrum, which implies that seemingly reliable test kit can actually mean nothing when it comes to interpreting the result of an individual - without using mathematical terms, only using intuitive and logical words. In addition, in our real life, P(+|D) is a risky information since it is based on hypothetical assumption(D); while P(D|+) is based on the tangible event which actually happened with the test kit showing positive. $\endgroup$
    – HYL
    Mar 11, 2021 at 8:28
  • 1
    $\begingroup$ I do not think it is so hard to explain. Push it to the extreme: the test says everyone has covid. It misses no cases, but it also gives no insight into whether or not someone has covid. $\endgroup$
    – Dave
    Mar 12, 2021 at 11:26

I think the fact that we have no insight into how the test works makes it harder to form an intuition. It might help to consider a similar table for a case where we do understand the mechanics of the test, such as testing whether there is currently a solar eclipse by checking whether the sky is dark.

$$P(\text{Dark | Eclipse}) \approx 1 \\ P(\text{Eclipse | Dark}) \approx 0$$

False positives (cases where the sky is dark and there is no eclipse, e.g. night) are much more commmon than true positives (an actual eclipse), so even if you see a positive test result it is probably false.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.