Base Rate fallacy in Conditional Probability P(A|B) vs P(B|A)

Question

From Wikipedia:

P(A|B) (the conditional probability of A given B) is not equal to P(B|A). For example, if a person has dengue they might have a 90% chance of testing positive for dengue. In this case what is being measured is that if event B ("having dengue") has occurred, the probability of A (test is positive) given that B (having dengue) occurred is 90%: that is, P(A|B) = 90%. Alternatively, if a person tests positive for dengue they may have only a 15% chance of actually having dengue because most people do not have dengue and the false positive rate for the test may be high. In this case what is being measured is the probability of the event B (having dengue) given that the event A (test is positive) has occurred: P(B|A) = 15%. Falsely equating the two probabilities causes various errors of reasoning such as the base rate fallacy. Conditional probabilities can be correctly reversed using Bayes' theorem. Falsely equating the two probabilities causes various errors of reasoning such as the base rate fallacy.

$P(A|B)$ -- I understand what it means and it is correct. With common-sense it is easy to understand that if it is already known that you have dengue then testing positive for dengue will be 90% probable ( 10% false-negatives)

$P(B|A)$ -- While reverse is different, generally in real life, if someone is tested positive for dengue, we know he has dengue, for 100% sure and we put him in hospital immediately (we just do it, we don't take any chances). But we ignored the fact that, statistically, we are looking at within our own smaller number of experiences of family, friends and relatives (30 cases e.g.). If we take whole country's population (1.25 billion) into account then we will know that 85% is the false-positive rate for dengue and only 15% people tested positive really have it.

My question is how to learn to not to ignore such facts ? (like I remembered only friends and relatives and TV adverts for dengue) but when author brought facts: false positives and whole population, I understood why it is 15%, I would have laughed if someone would have said so without such statistical data. I think if one learns to think this way of not ignoring but looking for facts then one can learn Probability easily. How can I learn such way ?

Answer 1

You can use the Bayes' theorem:

Let's denote the test event by $T$(the outcome space is $\{0, 1\}$) and the event of getting struck by the disease by $D$(its outcome space: $\{0,1\}$). If we know the prior probability of someone having the disease is $\frac{1}{10,000}$(only one of the 10,000 people in the population get stricken by the disease), that's $P(D=1)=\frac{1}{10,000}$. And the the machine can test accurately 99 percent of those who have the disease, that is $P(T=1|D=1)=0.99$. We want to know how likely those who are tested as having the disease truely have the disease: $P(D=1|T=1)$.

We can form an equation according to the Bayes' theorem:
\begin{align} p(D=1 \mid T=1) &= \frac{p(T=1 \mid D=1)p(D=1)}{p(T=1)} \\ & = \frac{p(T=1 \mid D=1)p(D=1)}{p(T=1 \mid D=1)p(D=1) + p(T=1 \mid D=0)p(D=0)} \\ & = \frac{0.99 \times \frac{1}{10, 000}}{0.99 \times \frac{1}{10, 000} + 0.01 \times \frac{9, 999}{10, 000}} \\ & = 0.0098 \ll 0.99 = P(T=1|D=1) \end{align}

The intuition behind this is that the smaller the prior(the larger the population but the lower the disease incidence rate) the less likely the machine is right, because it can increase the false positive rate: $P(T=1, D=0)$ or $P(T=1|D=0)P(D=0)$ in the denominator.

Hope this would be of help.

Reference: An Intuitive (and Short) Explanation of Bayes’ Theorem