1
$\begingroup$

I am currently studying the textbook In All Likelihood -- Statistical Modelling and Inference Using Likelihood by Yudi Pawitan. Section Inverse probability: the Bayesians of chapter 1 says the following:

The first modern method to assimilate observed data for quantitative inductive reasoning was published (posthumously) in 1763 by Bayes with his Essay towards Solving a Problem in the Doctrine of Chances. He used an inverse probability, via the now-standard Bayes theorem, to estimate a binomial probability. The simplest form of the Bayes theorem for two events $A$ and $B$ is

$$P(A \vert B) = \dfrac{P(AB)}{P(B)} = \dfrac{P(B \vert A)P(A)}{P(B \vert A)P(A) + P(B \vert \overline{A})P(\overline{A})}. \tag{1.1}$$

Suppose the unknown binomial probability is $\theta$ and the observed number of successes in $n$ independent trials is $x$. Then, in modern notation, Bayes's solution is

$$f(\theta \vert x) = \dfrac{f(x, \theta)}{f(x)} = \dfrac{f(x \vert \theta) f(\theta)}{\int f(x \vert \theta) f(\theta) d \theta}, \tag{1.2}$$

where $f(\theta \vert x)$ is the conditional density of $\theta$ given $x$, $f(\theta)$ is the so-called prior density of $\theta$ and $f(x)$ is the marginal probability of $x$. (Note that we have used the symbol $f(\cdot)$ as a generic function, much like the way we use $P(\cdot)$ for probability. The named argument(s) of the function determines what the function is. Thus, $f(\theta, x)$ is the joint density of $\theta$ and $x$, $f(x \vert \theta)$ is the conditional density of $x$ given $\theta$, etc.)

Leaving aside the problem of specifying $f(\theta)$, Bayes had accomplished a giant step: he had put the problem of inductive inference (i.e. learning from data $x$) within the clean deductive steps of mathematics. Alas, 'the problem of specifying $f(\theta)$' a priori is an equally giant point of controversy up to the present day.

There is nothing controversial about the Bayes theorem (1.1), but (1.2) is a different matter. Both $A$ and $B$ in (1.1) are random events, while in the Bayesian use of (1.2) only $x$ needs to be a random outcome; in a typical binomial experiment $\theta$ is an unknown fixed parameter. Bayes was well aware of this problem, which he overcame by considering that $\theta$ was generated in an auxiliary physical experiment - throwing a ball on a level square table - such that $\theta$ is expected to be uniform in the interval $(0, 1)$. Specifically, in this case we have $f(\theta) = 1$ and

$$f(\theta \vert x) = \dfrac{\theta^x(1 - \theta)^{n - x}}{\int_0^1 u^x(1 - u)^{n - x} du} \tag{1.3}$$

I have no idea what was meant by "throwing a ball on a level square table - such that $\theta$ is expected to be uniform in the interval $(0, 1)$". What is the point of this, and why does throwing a ball in this way mean that $\theta$ is expected to be uniform on the interval $(0, 1)$? The author's explanation seems terribly unclear.

I would greatly appreciate it if people would please take the time to clarify this.

$\endgroup$
5

1 Answer 1

2
$\begingroup$

I suggest reading this explanatory post on Bayes's experiment, but here's the main snippet:

He [Bayes] used a thought experiment to illustrate the process. Imagine that Bayes has his back turned to a table, and he asks his assistant to drop a ball on the table. The table is such that the ball has just as much chance of landing at any one place on the table as anywhere else. Now Bayes has to figure out where the ball is, without looking.

He asks his assistant to throw another ball on the table and report whether it is to the left or the right of the first ball. If the new ball landed to the left of the first ball, then the first ball is more likely to be on the right side of the table than the left side. He asks his assistant to throw the second ball again. If it again lands to the left of the first ball, then the first ball is even more likely than before to be on the right side of the table. And so on.

The general idea, however, it that the landing spot of the ball on the table is random, and therefore the position of the ball measured from the left side (or any, as long as it's chosen before throwing) of the table can be described through a Uniform distribution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.