Deriving posterior of Beta distribution

Question

You test a classifier on a test set consisting of 10 iid items. The classifier makes 2 mistakes. Assume the true error rate is $x$.

Let the prior be $ x \sim Beta(\alpha, \beta)$. Derive the posterior given the classifier's performance.

I know that to calculate the posterior, I use the definition $p(\theta|X) \propto p(\theta)p(X|\theta)$. I can plug in the definition of the beta distribution for $p(\theta)$ but I'm not sure what to plug in for the likelihood $p(X|\theta)$.

Is the likelihood function $p(X|\theta) = x^2(1-x)^8$?

Answer 1

Your formulas look a bit strange because $\theta=x$. $X$ and $x$ are usually used to denote the random variable and the data, not the parameter.

Anyway, if $X$ is the number of mistakes among 10 iid items then $X\sim Bin(10,x)$, i.e. $X$ follows the binomial distribution. Thus $p(X|x)=\binom{10}{X} x^2(1-x)^8$, and you can compute the posterior using your formula.

Answer 2

Using a simplified notation, you have $\textit{a priori}$ that $\theta\sim Beta(a,b)$, and $$ f(x\mid\theta)={n\choose x}\theta^x (1-\theta)^{n-x} \, , $$ for $x=0,1,\dots,n$ and $\theta\in[0,1]$.

As you already know, the posterior $\pi(\theta\mid x)$ is proportional to $$ Likelihood \times Prior \, , $$ where the $Likelihood$ is just $f(x\mid\theta)$ seen as function of $\theta$, for some fixed value of the observation $x$.

The important point here, which happens over and over in this kind of computation, is that you don't need to evaluate the integral $\int_0^1\pi(\theta\mid x)\,d\theta$ to find the normalization constant of the posterior density. For example, if the posterior is proportional to $$ \theta^5 (1-\theta)^7 = \theta^{6-1} (1-\theta)^{8-1} \, , $$ then you already know, by simple inspection of the formula of the beta density, that $\theta\mid x\sim Beta(6,8)$.

Can you see why?

This is a pattern that you should keep in your mind when solving similar problems.

Answer 3

To make things clear: you have a model like this
$X\sim Binomial\left ( 10, p \right )$
$p\sim Beta\left ( \alpha ,\beta \right )$
where $p$ is a probability of mistake. You want to find posterior. To do that you have to know likelihood, which is equal to
$L\left ( X|p \right )=\begin{pmatrix} 10\\2 \end{pmatrix}p^{2}\left ( 1-p \right )^{8}$.
As stated in wikipedia:
"the likelihood of a set of parameter values given some observed outcomes is equal to the probability of those observed outcomes given those parameter values".
But it is not generally a probability distribution and should not be though about as a probability measure.
As you already (correctly) stated, posterior is calculated as $p(\theta|X) \propto p(\theta)p(X|\theta)$.
(Perform neccesery calculations as an exercise problem and show that posterior is also Beta distribution with updated parameters).