I am trying to estimate the posterior distribution using Bayes theorem. The following information is given:

\begin{align} \newcommand{\prior}{{\rm prior}} \newcommand{\likelihood }{{\rm likelihood }} \newcommand{\posterior}{{\rm posterior}} \posterior&∝\likelihood \times \prior \\ \likelihood(x|θ)&∝ θ^k\times(1-θ)^{n-k} \\ \prior(θ)&∝1/θ \end{align}

Here I interested in estimating the posterior distribution of θ, with Binomial likelihood(k successes of the n trials).

By multiplying the likelihood and prior I get the following:

\begin{align} \posterior&∝θ^k\times(1-θ)^{n-k}\times1/θ \quad = \quad θ^k\times(1-θ)^{n-k}\timesθ^{-1} \\ \posterior&∝θ^{k-1}\times(1-θ)^{n-k} \end{align}

Question: The posterior distribution looks like Beta distribution: $θ^{\alpha-1}(1-θ)^{\beta-1}$. It is quite clear that $\alpha=k$ , but I am struggling to figure out how to transform power $n-k$ to look like $\beta_{new}={\rm something} - 1$. Could anyone point me to the right direction?

  • $\begingroup$ Is your data discrete or continuous? Is the parameter a proportion. I am thinking you may be talking about a situation wher you have a beta posterior and a binomial likelihood. But why does a 1/theta prior enter into it? Is that even a proper prior? $\endgroup$ Jan 6, 2017 at 23:00
  • 1
    $\begingroup$ $\beta = n-k+1$. $\endgroup$
    – whuber
    Jan 6, 2017 at 23:11
  • $\begingroup$ This prior should not be used because the posterior is not defined when $X=n$, which has a positive probability to occur under the Binomial model. $\endgroup$
    – Xi'an
    Jan 7, 2017 at 8:36
  • $\begingroup$ @MichaelChernick, the data is discrete. The random variable is the numer of patience with no side effects. I am a bit comfused myself about the form of the prior, but that was given by the assignment: p(theta) ∝ 1/theta. $\endgroup$
    – Artur
    Jan 7, 2017 at 9:37
  • $\begingroup$ @Xi'an indeed it looks wierd, but that was given by the assignment: p(theta) ∝ 1/theta $\endgroup$
    – Artur
    Jan 7, 2017 at 9:39

1 Answer 1


Maybe I'm missing something, but isn't the prior, $p(\theta) \propto \theta^{-1}$ merely a $\mbox{beta}(a=0,b=1)$ distribution? (If $a=0$ is uncomfortable, then set $a=\epsilon$ for small $\epsilon$.) Therefore the usual beta-Bernoulli update rule should apply. That is, with a Bernoulli likelihood function (which this is, for $N$ independent trials), and a conjugate beta prior (for which the present problem is a special case), the posterior is again a beta distribution: $\mbox{beta}(a+k,b+N-k)$. Here we start with $a=\epsilon$ and $b=1$, so the posterior is $\mbox{beta}(\epsilon+k,1+N-k)$.

P.S. For a derivation of the general beta-Bernoulli update rule, see p. 132 of DBDA2E.

  • $\begingroup$ Thanks for the answer! I had the same in mind, but somehow Alpha=0 did not make sense as by definition of Beta distribution Alpha>0. Would that be a case of an improper prior? Also, would the posterior still be a proper Beta distribution? $\endgroup$
    – Artur
    Jan 7, 2017 at 14:38
  • $\begingroup$ @Artur check en.wikipedia.org/wiki/… $\endgroup$
    – Tim
    Jan 7, 2017 at 14:42
  • $\begingroup$ @Artur: as indicated in my earlier comment, you cannot use this improper prior because the posterior is not defined when $X=n$. $\endgroup$
    – Xi'an
    Jan 7, 2017 at 14:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.