I have to teach the Bayes formula to some undergraduates, in the form:

$$ P(A|B) = \frac{P(A \cap B)}{P(B)} = \frac{P(B|A)P(A)}{P(B)} $$

I was wondering if anyone had any really thoughtful or alternative ways of teaching it instead of the standard approach.

  • 2
    $\begingroup$ What do you mean by "the standard approach"? I have seen several quite different ways of teaching it, I'm not sure I've seen one which is "standard" (nor necessarily that any of them struck me as particularly bad or worth avoiding, though I'm sure some put a different emphasis than others). $\endgroup$ – Silverfish Oct 31 '15 at 20:50
  • 1
    $\begingroup$ +1 to Silverfish, for you particular question maybe it will be worth checking with Mathematics Educators SE too. Also what level of Maths/Stats do you assume? Maths-y majors usually do their Bayesian statistics course around midway through their curriculum or later, so students are generally well-versed mathematically. Non-Maths-y majors on the other hand might do Bayesian statistics in their "Stats for Life/Social Sciences" course; usually one does not assume people know integration in that case. Different audiences... $\endgroup$ – usεr11852 Oct 31 '15 at 21:01
  • 2
    $\begingroup$ It depends on what you want to achieve and in what context. For an very elementary demonstration of the theorem, I usually start with writing P(AB) two ways: P(AB) = P(A|B)P(B) and P(AB) = P(B|A)P(A). Equate the two and divide by either P(A) or P(B) as desired. Some find that intuitive. Others require clear motivating examples (which I also try to provide). $\endgroup$ – Glen_b Nov 1 '15 at 2:50
  • 1
    $\begingroup$ This is equivalent to the definition of conditional probabilities, so if your students understand conditional probabilities, they should have no worries with Bayes' formula. In the French curriculum, it is taught in high school and kids do not seem to have a particularly hard time with it. $\endgroup$ – Xi'an Nov 19 '15 at 8:18
  • $\begingroup$ Because there are so many application of Bayes Theorem to other problems in statistics, its really important to make clear to student that what you present is one of several formulations. Alternatively you can present the theory in very general terms, i.e. relating the conditional probabilitiy of one event to the reverse conditional probability of a related event, and then connect that to more specific language around hypotheses and evidence or specificity and sensitivity (see answers below). $\endgroup$ – atirvine88 Apr 26 at 13:34

(I've been thinking of writing an academic paper on this topic for ages, so I'll give you the jist of what I was going to write in that paper.) Let $H$ refer to a hypothesis/theory and let $E$ refer to the observed evidence (i.e., the data). We will define the updating ratios:

$$\Delta(H|E) \equiv \frac{\mathbb{P}(H|E)}{\mathbb{P}(H)} \quad \quad \quad \quad \quad \Delta(E|H) \equiv \frac{\mathbb{P}(E|H)}{\mathbb{P}(E)}.$$

(These "updating ratios" are posterior-to-prior ratios for events, so they are "updating" our beliefs based on the conditioning information.) Bayes' rule can be easily rearranged to allow it to be presented as a symmetry principle concerning these updating ratios:

$$\Delta(H|E) = \frac{\mathbb{P}(H|E)}{\mathbb{P}(H)} = \frac{\mathbb{P}(E|H)}{\mathbb{P}(E)} = \Delta(E|H).$$

This symmetric version of Bayes' rule says that the updating ratio for the hypothesis given the evidence is the same as the updating ratio for the evidence given the hypothesis. Though this is a trivial mathematical result, its intuitive implications are substantial. It demonstrates that Bayesian reasoning consists of a simple inversion of the relationship between hypothesis and evidence through the mechanism of updating ratios. That is, Bayesian reasoning directs us to determine the updating ratio for the evidence, and then simply reverse the elements of this, with our interpretation being given mutatis mutandis for the hypothesis!

This principle can be illustrated graphically by a simple diagram below. As an applied example, suppose we have a document such as an academic paper, where the author name is not available. We have a hypothesis that the writer of a document is male, and our evidence for or against this hypothesis consists of whether the writer uses "he" as a gender-neutral third-person pronoun in the document. In this case the updating ratio $\Delta(E|H)$ can be thought of as our belief about the relative ratio of use of this pronoun by males, compared to its use by the whole population, and the updating ratio $\Delta(H|E)$ is the ratio of our posterior probability for the hypothesis that the writer is male, compared to our prior belief in this hypothesis. Bayes' rule tells us that these ratios must be the same ---i.e., if we think that a male writer is $x$% more likely to use "he" as a gender-neutral third-party pronoun, compared to a general writer in the population (who could be male or female),$^\dagger$ then if we observe this pronoun usage, our posterior probability that the writer is male should be $x$% higher than our prior probability.

enter image description here

The nice thing about this form of Bayes' rule is that it gives you a simple symmetry principle, which is philosophically appealing and intuitive. The standard form of Bayes' rule is of course more useful for analysis, since it is framed in terms of standard model objects. Thus, my presentation of the above rearrangement of Bayes' rule should not be regarded as a superior form in a practical sense. However, I think this gives you a version of the rule that has a simple intuitive meaning, insofar as it allows you to present Bayes' rule as a symmetry principle that reverses hypothesis and evidence.

$^\dagger$ I am being a bit rough here; technically the updating ratio for the evidence should be done on a document-weighted basis relative to the source of the document.

| cite | improve this answer | |

It helps to disambiguate the meaning of "accuracy" more precisely like this Reddit comment, in which there's a typo: "1485" (in "out of 1485 people who test positive") ought be "1495". I rewrote it with whole numbers (rather than 0.5% as the disease rate).

To understand the theorem, you need to understand the vocabulary. "99% accurate" doesn't really give us information about the disease. We ought use the following terms:

Sensitivity - the odds that the test will be positive if you have the disease.

Specificity - the odds that the test will be negative if you lack the disease.

Positive predictive value - the odds that the test will correctly predict you have the disease, if you test positive.

Negative predictive value - the odds that the test will correctly predict you lack the disease, if you test negative.

Our population of 10,000 people has a 1% disease rate. So $\color{springgreen}{1000}$ people have the disease, and $\color{forestgreen}{99,000}$ don't.

We introduce a test that is 98% sensitive and 99% specific. It will correctly identify $\color{deepskyblue}{980}$ of 1000 people with the disease and $\color{red}{98,010}$ of 99,000 without the disease. It will incorrectly claim $\color{red}{20}$ people ($ = 1000 - 980$) with the disease don't have it, and $\color{deepskyblue}{990}$ people ($= 99,000 - 98,010$) without the disease have it.

So out of $\color{deepskyblue}{1970 \; (= 980 + 990)}$ people who test positive, 980 have the disease. Thus, our positive predictive value is $\dfrac{\color{springgreen}{1000}}{\color{deepskyblue}{1970}} = 50.76\%$.

Out of $\color{red}{98,030 \; (= 98,010 + 20)}$ who test negative, $\color{forestgreen}{99,000}$ do not have the disease. Thus, our negative predictive value is $\dfrac{\color{red}{98,030}}{\color{forestgreen}{99,000}} = 99.02\%$.

In this case, this test is first-rate for determining who lacks the disease. The $\color{deepskyblue}{1970}$ who test positive can be tested to confirm they do have the disease, whereas those who tested negative need no further tests.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.