Just learning Bayesian techniques through Insua et.al.'s "Bayesian Analysis of Stochastic Process Models." On page 18 they give an example of a gambler estimating the parameter $p$ in a binomial distribution after observing 9 heads in 12 coin tosses $\mathbf{x}$. Given her prior $f(p) = Beta(5,5)$, they state the posterior is \begin{align*} f(p|\mathbf{x}) & \propto f(\mathbf{x}|p)f(p) \\ & \propto {12 \choose 9} p^9 (1-p)^3 \frac{1}{B(5,5)}p^{5-1}(1-p)^{5-1} \\ & \propto p^{14-1} (1-p)^{8-1} \end{align*} and therefore $p|\mathbf{x} \sim Beta(14,8)$.

For the same problem they plot the likelihood function, but I guess they view it as function of $p$ instead of $\mathbf{x}$, and to do so they must normalize it so it becomes a pdf for $p$. Hence the likelihood function, which was a binomial(12,p) is now $$ \frac{l(p|\mathbf{x})}{\int_0^1 l(p|x)dp} = \frac{1}{B(10,4)}p^{10-1}(1-p)^{4-1}, $$ which is a $Beta(10,4)$ pdf. I.e., we have transformed the probability mass function $f(\mathbf{x}|p)$, a function of $\mathbf{x}$, into a probability density function $l(p|\mathbf{x})$, a function of $p$. Is this a proper way of understanding it?

  • 6
    $\begingroup$ The likelihood IS a function of your model parameters and not the observed data. There is no reason why this function should be a valid density (i.e. integrate to 1 wrt to the parameters). Hence one must normalize it to make it a valid PDF. There is a subtle point here that the likelihood may not be integratable at all in some cases. So, I am not sure if the second equation can be generalised to all cases. Wait for someone with more knowledge of this than me to clarify this! $\endgroup$ – Luca Jun 23 '15 at 21:07
  • $\begingroup$ There's usually no reason to normalize a likelihood (either as a Bayesian or as a frequentist); generally one proceeds to a posterior -- and where necessary (i.e. if it is not obtained in normalized form), to normalize that. $\endgroup$ – Glen_b -Reinstate Monica Mar 28 '18 at 2:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.