# Application of Bayes' theorem: $f(\theta \vert x) = \frac{\theta^x(1 - \theta)^{n - x}}{\int_0^1 u^x(1 - u)^{n - x} du}$

I am currently studying the textbook In All Likelihood -- Statistical Modelling and Inference Using Likelihood by Yudi Pawitan. Section Inverse probability: the Bayesians of chapter 1 says the following:

The first modern method to assimilate observed data for quantitative inductive reasoning was published (posthumously) in 1763 by Bayes with his Essay towards Solving a Problem in the Doctrine of Chances. He used an inverse probability, via the now-standard Bayes theorem, to estimate a binomial probability. The simplest form of the Bayes theorem for two events $$A$$ and $$B$$ is

$$P(A \vert B) = \dfrac{P(AB)}{P(B)} = \dfrac{P(B \vert A)P(A)}{P(B \vert A)P(A) + P(B \vert \overline{A})P(\overline{A})}. \tag{1.1}$$

Suppose the unknown binomial probability is $$\theta$$ and the observed number of successes in $$n$$ independent trials is $$x$$. Then, in modern notation, Bayes's solution is

$$f(\theta \vert x) = \dfrac{f(x, \theta)}{f(x)} = \dfrac{f(x \vert \theta) f(\theta)}{\int f(x \vert \theta) f(\theta) d \theta}, \tag{1.2}$$

where $$f(\theta \vert x)$$ is the conditional density of $$\theta$$ given $$x$$, $$f(\theta)$$ is the so-called prior density of $$\theta$$ and $$f(x)$$ is the marginal probability of $$x$$. (Note that we have used the symbol $$f(\cdot)$$ as a generic function, much like the way we use $$P(\cdot)$$ for probability. The named argument(s) of the function determines what the function is. Thus, $$f(\theta, x)$$ is the joint density of $$\theta$$ and $$x$$, $$f(x \vert \theta)$$ is the conditional density of $$x$$ given $$\theta$$, etc.)

Leaving aside the problem of specifying $$f(\theta)$$, Bayes had accomplished a giant step: he had put the problem of inductive inference (i.e. learning from data $$x$$) within the clean deductive steps of mathematics. Alas, 'the problem of specifying $$f(\theta)$$' a priori is an equally giant point of controversy up to the present day.

There is nothing controversial about the Bayes theorem (1.1), but (1.2) is a different matter. Both $$A$$ and $$B$$ in (1.1) are random events, while in the Bayesian use of (1.2) only $$x$$ needs to be a random outcome; in a typical binomial experiment $$\theta$$ is an unknown fixed parameter. Bayes was well aware of this problem, which he overcame by considering that $$\theta$$ was generated in an auxiliary physical experiment - throwing a ball on a level square table - such that $$\theta$$ is expected to be uniform in the interval $$(0, 1)$$. Specifically, in this case we have $$f(\theta) = 1$$ and

$$f(\theta \vert x) = \dfrac{\theta^x(1 - \theta)^{n - x}}{\int_0^1 u^x(1 - u)^{n - x} du} \tag{1.3}$$

Equation (1.3) is obviously an application of Bayes' theorem. And the terms in the numerator and denominator seem to be the binomial PMF.

1. It's been a while since I dealt with conditional probabilities, but, if my understanding is correct, this means that the random variable $$\theta$$ in $$f(\theta \vert x)$$ is binomially distributed? Or does it mean that the entire conditional distribution $$f(\theta \vert x)$$ is binomially distributed? Sorry for the elementary question, but I'm trying to remember the correct way to think about this.

2. Given what I said in 1. regarding the binomial PMF, why is $$u$$ used for the probability values in the integral in the denominator, whereas $$\theta$$ is used for the same probability value in the numerator?

I would greatly appreciate it if people would please take the time to clarify this.

• I think , $f(\theta |x)$ is $\beta (x+1,n-x+1)$ distribution,Beta, not binomial. Feb 18 '20 at 12:14
• @masoud Hmm, I don't see how what's in (1.3) matches the beta PDF? en.wikipedia.org/wiki/… Feb 18 '20 at 12:16
• note $\int_0^{1} u^{a-1} (1-u)^{b-1} du=\beta(a,b)=\frac{\Gamma(a) \Gamma(b)}{\Gamma(a+b)}$ Feb 18 '20 at 12:20
• dear @The Pointer, I see $\theta \in (0,1)$ so I think it can not be binomial. Feb 18 '20 at 12:34
• ,Yes. that is right Feb 18 '20 at 12:47

hint

$$\int_0^{1} u^{a-1} (1-u)^{b-1} du=\beta(a,b)=\frac{\Gamma(a) \Gamma(b)}{\Gamma(a+b)}$$

$$f(\theta |x)=\frac{\theta^{x} (1-\theta)^{n-x}}{\int_0^{1} u^{(x+1)-1} (1-u)^{(n-x+1)-1} du} =\frac{\theta^{(x+1)-1} (1-\theta)^{(n-x+1)-1}}{\beta(x+1,n-x+1)}$$ $$\theta \in (0,1)$$

For question 2)

In the part

$$\int_0^{1} u^{a-1} (1-u)^{b-1} du$$

It does not matter use $$u$$ or any symbol. On the other hand

$$\int_0^{1} u^{a-1} (1-u)^{b-1} du=\int_0^{1} t^{a-1} (1-t)^{b-1} dt$$

• the prior of $\theta$ is beta, the posterior also is beta. just updated the distribution based on observation.It mean $f(\theta|x)$ is beta,(the posterior distribution that updated by observation Feb 18 '20 at 12:43
• See my other comment. Doesn't the author say that $\theta$ is uniformly distributed on the interval $(0, 1)$? Feb 18 '20 at 12:46
• in this : $f(\theta)=1$ and note $\theta \in (0,1)$ Feb 18 '20 at 12:48
• Ok, thank you for the clarification. Feb 18 '20 at 12:49
• you are well come Feb 18 '20 at 12:50