Posterior Distribution for parameter with beta distribution given binomial data

Question

I am currently working on a problem but I'm having issues due to not having much experience with Bayesian statistics at the moment

I have a sequence of observations $X_1, X_2, \dots X_N$ the are distributed Binomially iid with parameter $\theta$. $\theta$ is distributed by a beta distribution with parameters $a,b$. I'm asked to find the likelihood function $L(\vec{x};\theta)$ and the posterior distribution for $\theta$ given my $N$ observations.

Here is my work \begin{align*} L( \vec{x} ; \theta) &= Pr(\vec{x}; \theta) \\ &= \prod_{i=1}^N \theta^{x_i}(1-\theta)^{1-x_i} \\ &= \theta^{\Sigma x_i} (1-\theta)^{N - \Sigma x_i} \end{align*} This one I think is correct

Now for the posterior distribution of $\theta$ I am having some issues: \begin{align*} Pr(\theta \vert \vec{x}) &= \frac{Pr(\vec{x} \mid \theta) Pr(\theta)}{Pr(\vec{x})} \\ &= \frac { \theta^{\Sigma x_i} (1-\theta)^{N - \Sigma x_i} \cdot Beta(a,b) } { \sum_{i=1}^N \theta_i^{\Sigma x_i} (1-\theta_i)^{N - \Sigma x_i} \cdot Beta(a,b) } \end{align*}

Mainly with the denominator, $Pr(\vec{x})$. Is it true that I have to calculate the total probability for all possible values of each $\theta_i$ to get $Pr(\vec{x})$ since the observations depend on $\theta?$ The question says that I must express the posterior distribution in terms of $a,b,$ and the data, but Im not sure how to get rid of the $\theta$.

Answer 1

You're not supposed to "get rid of" the $\theta$. Since it's a posterior distribution for $\theta$, you're seeking an expression in $\theta$. Once you have it, you will be able to recognize the density -- it is the parameters of that density will be in terms of $a,b$ and the data.

Firstly, your prior is a prior on $\theta$ so explicitly write down that density you've got as "Beta(a,b)" as a density function that would be a prior on $\theta$ ($p(\theta)$)

Secondly your denominator is wrong, for several reasons --

(i) you already used $i$ up in the powers of $\theta$ and $1-\theta$, so you couldn't index your sum by $i$

(ii) it shouldn't be a sum at all, $\theta$ is continuous.

You need to think more carefully about what you're doing with the denominator. It would involve an integral. You jumped some steps that would clarify what you should do with it. How would you get that marginal from a joint? How would you write the joint in terms of a conditional? (hint: look at your numerator)