# Simple real world examples for teaching Bayesian statistics?

I would like to find some "real world examples" for teaching Bayesian statistics. Bayesian statistics allows one to formally incorporate prior knowledge into an analysis. I would like to give students some simple real world examples of researchers incorporating prior knowledge into their analysis so that students can better understand the motivation for why one might want to use Bayesian statistics in the first place.

Are you aware of any simple real world examples such as estimating a population mean, proportion, regression, etc where researchers formally incorporate prior information? I realize Bayesians can use "non-informative" priors too, but I am particularly interested in real examples where informative priors (i.e. real prior information) are used.

• I think IQ is a pretty good example. – hejseb Jan 16 '18 at 8:36
• Not strictly an answer but when you flip a coin three times and head comes up two times then no student would believe, that head was twice as likely as tails.That is pretty convincing although certainly not real research. – Bernhard Jan 16 '18 at 8:45
• You can check out this answer, written by yours truly: stats.stackexchange.com/a/134385/61496 – Yair Daon Jan 16 '18 at 16:03
• Are you perhaps conflating Bayes Rule, which can be applied in frequentist probability/estimation, and Bayesian statistics where "probability" is a summary of belief? – AdamO Jan 16 '18 at 16:33

Bayesian search theory is an interesting real-world application of Bayesian statistics which has been applied many times to search for lost vessels at sea. To begin, a map is divided into squares. Each square is assigned a prior probability of containing the lost vessel, based on last known position, heading, time missing, currents, etc. Additionally, each square is assigned a conditional probability of finding the vessel if it's actually in that square, based on things like water depth. These distributions are combined to prioritize map squares that have the highest likelihood of producing a positive result - it's not necessarily the most likely place for the ship to be, but the most likely place of actually finding the ship.

• Nice, these are the sort of applications described in the entertaining book The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from two centuries of controversy. Also, Turing used this kind of reasoning to crack enigma. – jpmuc Jan 18 '18 at 20:58

I think estimating production or population size from serial numbers is interesting if traditional explanatory example. Here you are trying the maximum of a discrete uniform distribution. Depending on your choice of prior then the maximum likelihood and Bayesian estimates will differ in a pretty transparent way.

Perhaps the most famous example is estimating the production rate of German tanks during the second World War from tank serial number bands and manufacturer codes done in the frequentist setting by (Ruggles and Brodie, 1947). An alternative analysis from a Bayesian point of view with informative priors has been done by (Downey, 2013), and with an improper uninformative priors by (Höhle and Held, 2004). The work by (Höhle and Held, 2004) also contains many more references to previous treatment in the literature and there is also more discussion of this problem on this site.

Sources:

Chapter 3, Downey, Allen. Think Bayes: Bayesian Statistics in Python. " O'Reilly Media, Inc.", 2013.

Ruggles, R.; Brodie, H. (1947). "An Empirical Approach to Economic Intelligence in World War II". Journal of the American Statistical Association. 42 (237): 72.

Höhle, Michael, and Leonhard Held. Bayesian estimation of the size of a population. No. 499. Discussion paper//Sonderforschungsbereich 386 der Ludwig-Maximilians-Universität München, 2006.

There is a nice story in Cressie & Wickle Statistics for Spatio-Temporal Data, Wiley, about the (bayesian) search of the USS Scorpion, a submarine that was lost in 1968. We tell this story to our students and have them perform a (simplified) search using a simulator.

Similar examples could be constructed around the story of the lost flight MH370; you might want to look at Davey et al., Bayesian Methods in the Search for MH370, Springer-Verlag.

An area of research where I believe the Bayesian methods are absolutely necessary is that of optimal design.

In the logistic regression setting, a researcher is trying to estimate a coefficient and is actively collecting data, sometimes one data point at a time. The researcher has the ability to choose the input values of $x$. The goal is to maximize the information learned for a given sample size (alternatively, minimize the sample size required to reach some level of certainty). One can show that for a given $\beta$ there is a set of $x$ values that optimize this problem.

The catch-22 here is that to choose the optimal $x$'s, you need to know $\beta$. Clearly, you don't know $\beta$ or you wouldn't need to collect data to learn about $\beta$. You could just use the MLE's to select $x$, but

• This doesn't give you a starting point; for $n = 0$, $\hat \beta$ is undefined

• Even after taking several samples, the Hauck-Donner effect means that $\hat \beta$ has a positive probability of being undefined (and this is very common for even samples of, say 10, in this problem)

• Even after the MLE is finite, its likely to be incredibly unstable, thus wasting many samples (i.e if $\beta = 1$ but $\hat \beta = 5$, you will pick values of $x$ that would have been optimal if $\beta = 5$, but it's not, resulting in very suboptimal $x$'s).

• This doesn't take into account the uncertainty of $\beta$

The (admittedly older) Frequentist literature deals with a lot of these issues in a very ad-hoc manner and offers sub-optimal solutions: "pick regions of $x$ that you think should lead to both 0's and 1's, take samples until the MLE is defined, and then use the MLE to choose $x$".

The Bayesian analysis is to start with a prior, find the $x$ that is most informative about $\beta$ given the current knowledge, repeat until the convergence.

Given that this is a problem that starts with no data and requires information about $\beta$ to choose $x$, I think it's undeniable that the Bayesian method is necessary; even the Frequentist methods instruct one to use prior information. The Bayesian method just does so in a much more efficient and logically justified manner.

Also, it's totally reasonable to analyze the data that comes in a Frequentist method (or ignoring the prior), but it's very hard to argue against using a Bayesian method to choose the next $x$.

Here is an example of estimating a mean, $\theta$, from Normal continuous data. Before delving directly into an example, though, I'd like to review some of the math for Normal-Normal Bayesian data models.

Consider a random sample of n continuous values denoted by $y_1, ..., y_n$. Here the vector $y = (y_1, ..., y_n)^T$ represents the data gathered. The probability model for Normal data with known variance and independent and identically distributed (i.i.d.) samples is

$$y_1, ..., y_n | \theta \sim N(\theta, \sigma^2)$$

Or as more typically written by Bayesian,

$$y_1, ..., y_n | \theta \sim N(\theta, \tau)$$

where $\tau = 1 / \sigma^2$; $\tau$ is known as the precision

With this notation, the density for $y_i$ is then

$$f(y_i | \theta, \tau) = \sqrt(\frac{\tau}{2 \pi}) \times exp\left( -\tau (y_i - \theta)^2 / 2 \right)$$

Classical statistics (i.e. maximum likelihood) gives us an estimate of $\hat{\theta} = \bar{y}$

In a Bayesian perspective, we append maximum likelihood with prior information. A choice of priors for this Normal data model is another Normal distribution for $\theta$. The Normal distribution is conjugate to the Normal distribution.

$$\theta \sim N(a,1/b)$$

The posterior distribution we obtain from this Normal-Normal (after a lot of algebra) data model is another Normal distribution.

$$\theta | y \sim N(\frac{b}{b + n\tau} a + \frac{n \tau}{b + n \tau} \bar{y}, \frac{1}{b + n\tau})$$

The posterior precision is $b + n\tau$ and mean is a weighted mean between $a$ and $\bar{y}$, $\frac{b}{b + n\tau} a + \frac{n \tau}{b + n \tau} \bar{y}$.

The usefulness of this Bayesian methodology comes from the fact that you obtain a distribution of $\theta | y$ rather than just an estimate since $\theta$ is viewed as a random variable rather than a fixed (unknown) value. In addition, your estimate of $\theta$ in this model is a weighted average between the empirical mean and prior information.

That said, you can now use any Normal-data textbook example to illustrate this. I'll use the data set airquality within R. Consider the problem of estimating average wind speeds (MPH).

> ## New York Air Quality Measurements
>
> help("airquality")
>
> ## Estimating average wind speeds
>
> wind = airquality$Wind > hist(wind, col = "gray", border = "white", xlab = "Wind Speed (MPH)") >  > n = length(wind) > ybar = mean(wind) > ybar [1] 9.957516 ## "frequentist" estimate > tau = 1/sd(wind) > > > ## but based on some research, you felt avgerage wind speeds were closer to 12 mph > ## but probably no greater than 15, > ## then a potential prior would be N(12, 2) > > a = 12 > b = 2 > > ## Your posterior would be N((1/)) > > postmean = 1/(1 + n*tau) * a + n*tau/(1 + n*tau) * ybar > postsd = 1/(1 + n*tau) > > set.seed(123) > posterior_sample = rnorm(n = 10000, mean = postmean, sd = postsd) > hist(posterior_sample, col = "gray", border = "white", xlab = "Wind Speed (MPH)") > abline(v = median(posterior_sample)) > abline(v = ybar, lty = 3) >  > median(posterior_sample) [1] 10.00324 > quantile(x = posterior_sample, probs = c(0.025, 0.975)) ## confidence intervals 2.5% 97.5% 9.958984 10.047404  In this analysis, the researcher (you) can say that given data + prior information, your estimate of average wind, using the 50th percentile, speeds should be 10.00324, greater than simply using the average from the data. You also obtain a full distribution, from which you can extract a 95% credible interval using the 2.5 and 97.5 quantiles. Below I include two references, I highly recommend reading Casella's short paper. It's specifically aimed at empirical Bayes methods, but explains the general Bayesian methodology for Normal models. References: 1. Casella, G. (1985). An Introduction to Empirical Bayes Data Analysis. The American Statistician, 39(2), 83-87. 2. Gelman, A. (2004). Bayesian data analysis (2nd ed., Texts in statistical science). Boca Raton, Fla.: Chapman & Hall/CRC. I was thinking of this question lately, and I think I have an example where bayesian make sense, with the use a prior probability: the likelyhood ratio of a clinical test. The example could be this one: the validity of the urine dipslide under daily practice conditions (Family Practice 2003;20:410-2). The idea is to see what a positive result of the urine dipslide imply on the diagnostic of urine infection. The likelyhood ratio of the positive result is: $$LR(+) = \frac{test+|H+}{test+|H-} = \frac{Sensibility}{1-specificity}$$ with$H+$the hypothesis of a urine infection, and$H-$no urine infection. What Bayes tells us is $$OR(+|test+) = LR(+) \times OR(+)$$ Where$OR$is the odds ratio.$OR(+|test+)$is the odd ratio of having a urine infection knowing that the test is positive, and$OR(+)$the prior odd ratio. The article gives that$LR(+) = 12.2$, and$LR(-) = 0.29$. Here the prior knowledge is the probability to have a urine infection based on the clinical analysis of the potentially sick person before making the test. if the physician estimate that this probability is$p_{+} = 2/3$based on observation, then a positive test leads the a post probability of$p_{+|test+} = 0.96$, and of$p_{+|test-} = 0.37\$ if the test is negative.

Here the test is good to detect the infection, but not that good to discard the infection.