Application of Bayes' theorem: $f(\theta \vert x) = \frac{\theta^x(1 - \theta)^{n - x}}{\int_0^1 u^x(1 - u)^{n - x} du}$

Question

I am currently studying the textbook In All Likelihood -- Statistical Modelling and Inference Using Likelihood by Yudi Pawitan. Section Inverse probability: the Bayesians of chapter 1 says the following:

The first modern method to assimilate observed data for quantitative inductive reasoning was published (posthumously) in 1763 by Bayes with his Essay towards Solving a Problem in the Doctrine of Chances. He used an inverse probability, via the now-standard Bayes theorem, to estimate a binomial probability. The simplest form of the Bayes theorem for two events $A$ and $B$ is

$$P(A \vert B) = \dfrac{P(AB)}{P(B)} = \dfrac{P(B \vert A)P(A)}{P(B \vert A)P(A) + P(B \vert \overline{A})P(\overline{A})}. \tag{1.1}$$

Suppose the unknown binomial probability is $\theta$ and the observed number of successes in $n$ independent trials is $x$. Then, in modern notation, Bayes's solution is

$$f(\theta \vert x) = \dfrac{f(x, \theta)}{f(x)} = \dfrac{f(x \vert \theta) f(\theta)}{\int f(x \vert \theta) f(\theta) d \theta}, \tag{1.2}$$

where $f(\theta \vert x)$ is the conditional density of $\theta$ given $x$, $f(\theta)$ is the so-called prior density of $\theta$ and $f(x)$ is the marginal probability of $x$. (Note that we have used the symbol $f(\cdot)$ as a generic function, much like the way we use $P(\cdot)$ for probability. The named argument(s) of the function determines what the function is. Thus, $f(\theta, x)$ is the joint density of $\theta$ and $x$, $f(x \vert \theta)$ is the conditional density of $x$ given $\theta$, etc.)

Leaving aside the problem of specifying $f(\theta)$, Bayes had accomplished a giant step: he had put the problem of inductive inference (i.e. learning from data $x$) within the clean deductive steps of mathematics. Alas, 'the problem of specifying $f(\theta)$' a priori is an equally giant point of controversy up to the present day.

There is nothing controversial about the Bayes theorem (1.1), but (1.2) is a different matter. Both $A$ and $B$ in (1.1) are random events, while in the Bayesian use of (1.2) only $x$ needs to be a random outcome; in a typical binomial experiment $\theta$ is an unknown fixed parameter. Bayes was well aware of this problem, which he overcame by considering that $\theta$ was generated in an auxiliary physical experiment - throwing a ball on a level square table - such that $\theta$ is expected to be uniform in the interval $(0, 1)$. Specifically, in this case we have $f(\theta) = 1$ and

$$f(\theta \vert x) = \dfrac{\theta^x(1 - \theta)^{n - x}}{\int_0^1 u^x(1 - u)^{n - x} du} \tag{1.3}$$

Equation (1.3) is obviously an application of Bayes' theorem. And the terms in the numerator and denominator seem to be the binomial PMF.

1. It's been a while since I dealt with conditional probabilities, but, if my understanding is correct, this means that the random variable $\theta$ in $f(\theta \vert x)$ is binomially distributed? Or does it mean that the entire conditional distribution $f(\theta \vert x)$ is binomially distributed? Sorry for the elementary question, but I'm trying to remember the correct way to think about this.

2. Given what I said in 1. regarding the binomial PMF, why is $u$ used for the probability values in the integral in the denominator, whereas $\theta$ is used for the same probability value in the numerator?

I would greatly appreciate it if people would please take the time to clarify this.

Answer 1

hint

$\int_0^{1} u^{a-1} (1-u)^{b-1} du=\beta(a,b)=\frac{\Gamma(a) \Gamma(b)}{\Gamma(a+b)}$

$$f(\theta |x)=\frac{\theta^{x} (1-\theta)^{n-x}}{\int_0^{1} u^{(x+1)-1} (1-u)^{(n-x+1)-1} du} =\frac{\theta^{(x+1)-1} (1-\theta)^{(n-x+1)-1}}{\beta(x+1,n-x+1)}$$ $$\theta \in (0,1)$$

For question 2)

In the part

$$\int_0^{1} u^{a-1} (1-u)^{b-1} du$$

It does not matter use $u$ or any symbol. On the other hand

$$\int_0^{1} u^{a-1} (1-u)^{b-1} du=\int_0^{1} t^{a-1} (1-t)^{b-1} dt$$