2
$\begingroup$

You test a classifier on a test set consisting of 10 iid items. The classifier makes 2 mistakes. Assume the true error rate is $x$.

Let the prior be $ x \sim Beta(\alpha, \beta)$. Derive the posterior given the classifier's performance.

I know that to calculate the posterior, I use the definition $p(\theta|X) \propto p(\theta)p(X|\theta)$. I can plug in the definition of the beta distribution for $p(\theta)$ but I'm not sure what to plug in for the likelihood $p(X|\theta)$.

Is the likelihood function $p(X|\theta) = x^2(1-x)^8$?

$\endgroup$
3
$\begingroup$

Your formulas look a bit strange because $\theta=x$. $X$ and $x$ are usually used to denote the random variable and the data, not the parameter.

Anyway, if $X$ is the number of mistakes among 10 iid items then $X\sim Bin(10,x)$, i.e. $X$ follows the binomial distribution. Thus $p(X|x)=\binom{10}{X} x^2(1-x)^8$, and you can compute the posterior using your formula.

$\endgroup$
3
$\begingroup$

Using a simplified notation, you have $\textit{a priori}$ that $\theta\sim Beta(a,b)$, and $$ f(x\mid\theta)={n\choose x}\theta^x (1-\theta)^{n-x} \, , $$ for $x=0,1,\dots,n$ and $\theta\in[0,1]$.

As you already know, the posterior $\pi(\theta\mid x)$ is proportional to $$ Likelihood \times Prior \, , $$ where the $Likelihood$ is just $f(x\mid\theta)$ seen as function of $\theta$, for some fixed value of the observation $x$.

The important point here, which happens over and over in this kind of computation, is that you don't need to evaluate the integral $\int_0^1\pi(\theta\mid x)\,d\theta$ to find the normalization constant of the posterior density. For example, if the posterior is proportional to $$ \theta^5 (1-\theta)^7 = \theta^{6-1} (1-\theta)^{8-1} \, , $$ then you already know, by simple inspection of the formula of the beta density, that $\theta\mid x\sim Beta(6,8)$.

Can you see why?

This is a pattern that you should keep in your mind when solving similar problems.

$\endgroup$
0
$\begingroup$

To make things clear: you have a model like this
$X\sim Binomial\left ( 10, p \right )$
$p\sim Beta\left ( \alpha ,\beta \right )$
where $p$ is a probability of mistake. You want to find posterior. To do that you have to know likelihood, which is equal to
$L\left ( X|p \right )=\begin{pmatrix} 10\\2 \end{pmatrix}p^{2}\left ( 1-p \right )^{8}$.
As stated in wikipedia:
"the likelihood of a set of parameter values given some observed outcomes is equal to the probability of those observed outcomes given those parameter values".
But it is not generally a probability distribution and should not be though about as a probability measure.
As you already (correctly) stated, posterior is calculated as $p(\theta|X) \propto p(\theta)p(X|\theta)$.
(Perform neccesery calculations as an exercise problem and show that posterior is also Beta distribution with updated parameters).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.