1
$\begingroup$

I have a question about Bayesian statistics.

In a book called PML, I came across the following description of how to calculate the marginal likelihood of a beta-binomial distribution.

In the case of the betaBernoulli model, the marginal likelihood is proportional to the ratio of the posterior normalizer to the prior normalizer. To see this, recall that the posterior for the beta-binomial models is given by $p(\theta|D)=Beta(\theta|a', b')$, where $a'=a+N_1$ and $b'=b+N_0$. We know the normalization constant of the posterior is $B(a', b')$. Hence

$p(\theta|D)=\begin{pmatrix}N \\ N_1 \end{pmatrix}\frac{1}{p(D)}\frac{1}{B(a, b)}[\theta^{a+N_1-1}(1-\theta)^{b+N_0-1}]$

so

$\frac{1}{B(a+N_1, b+N_0)}=\begin{pmatrix}N \\ N_1 \end{pmatrix}\frac{1}{p(D)}\frac{1}{B(a, b)}$,

$p(D)=\begin{pmatrix}N \\ N_1 \end{pmatrix}\frac{B(a+N_1, b+N_0)}{B(a, b)}.$

  • suppose we toss a coin N times.
  • $N_1$: the number of heads.
  • $N_0$: the number of tails.

Why can the marginal likelihood be calculated in this way?

What does it mean that the marginal likelihood is proportional to the ratio of the posterior normalizer to the prior normalizer?

$\endgroup$
1

1 Answer 1

2
$\begingroup$

If $x|\theta\sim\mathcal B(N,\theta)$ and $\theta\sim\mathcal Be(a,b)$, the marginal density of $x$ is $$m(x)=\int_0^1 {N\choose x}\theta^x(1-\theta)^{N-x} \times B(a,b) \theta^{a-1}(1-\theta)^{b-1}\,\text d\theta$$ where $$B(a,b)=1{\Big/}\int_0^1 \theta^{a-1}(1-\theta)^{b-1}\,\text d\theta$$ normalizes the function $\theta\mapsto\theta^{a-1}(1-\theta)^{b-1}$ into a probability density. Hence, $$m(x)=B(a,b) \int_0^1 {N\choose x}\theta^{x+a-1}(1-\theta)^{N-1+b-1}\,\text d\theta=B(a,b) \Big/B(x+a,N-x+b)$$ the ratio of the two normalising constants.

$\endgroup$
1
  • $\begingroup$ Thank you very much! $\endgroup$
    – NakataKoo
    Jun 12, 2023 at 9:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.