# Unclear explanation of Bayes theorem

I am currently studying the textbook In All Likelihood -- Statistical Modelling and Inference Using Likelihood by Yudi Pawitan. Section Inverse probability: the Bayesians of chapter 1 says the following:

The first modern method to assimilate observed data for quantitative inductive reasoning was published (posthumously) in 1763 by Bayes with his Essay towards Solving a Problem in the Doctrine of Chances. He used an inverse probability, via the now-standard Bayes theorem, to estimate a binomial probability. The simplest form of the Bayes theorem for two events $$A$$ and $$B$$ is

$$P(A \vert B) = \dfrac{P(AB)}{P(B)} = \dfrac{P(B \vert A)P(A)}{P(B \vert A)P(A) + P(B \vert \overline{A})P(\overline{A})}. \tag{1.1}$$

Suppose the unknown binomial probability is $$\theta$$ and the observed number of successes in $$n$$ independent trials is $$x$$. Then, in modern notation, Bayes's solution is

$$f(\theta \vert x) = \dfrac{f(x, \theta)}{f(x)} = \dfrac{f(x \vert \theta) f(\theta)}{\int f(x \vert \theta) f(\theta) d \theta}, \tag{1.2}$$

where $$f(\theta \vert x)$$ is the conditional density of $$\theta$$ given $$x$$, $$f(\theta)$$ is the so-called prior density of $$\theta$$ and $$f(x)$$ is the marginal probability of $$x$$. (Note that we have used the symbol $$f(\cdot)$$ as a generic function, much like the way we use $$P(\cdot)$$ for probability. The named argument(s) of the function determines what the function is. Thus, $$f(\theta, x)$$ is the joint density of $$\theta$$ and $$x$$, $$f(x \vert \theta)$$ is the conditional density of $$x$$ given $$\theta$$, etc.)

Leaving aside the problem of specifying $$f(\theta)$$, Bayes had accomplished a giant step: he had put the problem of inductive inference (i.e. learning from data $$x$$) within the clean deductive steps of mathematics. Alas, 'the problem of specifying $$f(\theta)$$' a priori is an equally giant point of controversy up to the present day.

There is nothing controversial about the Bayes theorem (1.1), but (1.2) is a different matter. Both $$A$$ and $$B$$ in (1.1) are random events, while in the Bayesian use of (1.2) only $$x$$ needs to be a random outcome; in a typical binomial experiment $$\theta$$ is an unknown fixed parameter. Bayes was well aware of this problem, which he overcame by considering that $$\theta$$ was generated in an auxiliary physical experiment - throwing a ball on a level square table - such that $$\theta$$ is expected to be uniform in the interval $$(0, 1)$$. Specifically, in this case we have $$f(\theta) = 1$$ and

$$f(\theta \vert x) = \dfrac{\theta^x(1 - \theta)^{n - x}}{\int_0^1 u^x(1 - u)^{n - x} du} \tag{1.3}$$

I have no idea what was meant by "throwing a ball on a level square table - such that $$\theta$$ is expected to be uniform in the interval $$(0, 1)$$". What is the point of this, and why does throwing a ball in this way mean that $$\theta$$ is expected to be uniform on the interval $$(0, 1)$$? The author's explanation seems terribly unclear.

I would greatly appreciate it if people would please take the time to clarify this.

• Bayes himself describes his experiment as well as anyone else might: Bayes (1763), Philos. Trans. Royal Soc., 53, pp 370-418, "An essay towards solving a problem in the doctrine of chances". Feb 17, 2020 at 15:06
• @Scortchi-ReinstateMonica that’s 49 pages; exactly which pages are relevant? Feb 17, 2020 at 15:11
• The experiment's described on p. 385. In the Scholium on p. 392 he makes the argument that ignorance alone justifies the uniform prior. Feb 17, 2020 at 15:31
• A good modern reference, with a full and thoughtful explanation (that overcomes the anachronistic oversimplifications in the quotation) appears in Stephen M. Stigler, The History of Statistics (1986), in a section of Chapter 3 headed Bayes and the Binomial.
– whuber
Feb 17, 2020 at 15:42
• Another nice entry is Steve Fienberg's When did Bayesian inference become "Bayesian"?. Feb 17, 2020 at 15:45