1
$\begingroup$

I am trying to estimate the posterior distribution using Bayes theorem. The following information is given:

\begin{align} \newcommand{\prior}{{\rm prior}} \newcommand{\likelihood }{{\rm likelihood }} \newcommand{\posterior}{{\rm posterior}} \posterior&∝\likelihood \times \prior \\ \likelihood(x|θ)&∝ θ^k\times(1-θ)^{n-k} \\ \prior(θ)&∝1/θ \end{align}

Here I interested in estimating the posterior distribution of θ, with Binomial likelihood(k successes of the n trials).

By multiplying the likelihood and prior I get the following:

\begin{align} \posterior&∝θ^k\times(1-θ)^{n-k}\times1/θ \quad = \quad θ^k\times(1-θ)^{n-k}\timesθ^{-1} \\ \posterior&∝θ^{k-1}\times(1-θ)^{n-k} \end{align}

Question: The posterior distribution looks like Beta distribution: $θ^{\alpha-1}(1-θ)^{\beta-1}$. It is quite clear that $\alpha=k$ , but I am struggling to figure out how to transform power $n-k$ to look like $\beta_{new}={\rm something} - 1$. Could anyone point me to the right direction?

$\endgroup$
  • $\begingroup$ Is your data discrete or continuous? Is the parameter a proportion. I am thinking you may be talking about a situation wher you have a beta posterior and a binomial likelihood. But why does a 1/theta prior enter into it? Is that even a proper prior? $\endgroup$ – Michael R. Chernick Jan 6 '17 at 23:00
  • 1
    $\begingroup$ $\beta = n-k+1$. $\endgroup$ – whuber Jan 6 '17 at 23:11
  • $\begingroup$ This prior should not be used because the posterior is not defined when $X=n$, which has a positive probability to occur under the Binomial model. $\endgroup$ – Xi'an Jan 7 '17 at 8:36
  • $\begingroup$ @MichaelChernick, the data is discrete. The random variable is the numer of patience with no side effects. I am a bit comfused myself about the form of the prior, but that was given by the assignment: p(theta) ∝ 1/theta. $\endgroup$ – Artur Jan 7 '17 at 9:37
  • $\begingroup$ @Xi'an indeed it looks wierd, but that was given by the assignment: p(theta) ∝ 1/theta $\endgroup$ – Artur Jan 7 '17 at 9:39
2
$\begingroup$

Maybe I'm missing something, but isn't the prior, $p(\theta) \propto \theta^{-1}$ merely a $\mbox{beta}(a=0,b=1)$ distribution? (If $a=0$ is uncomfortable, then set $a=\epsilon$ for small $\epsilon$.) Therefore the usual beta-Bernoulli update rule should apply. That is, with a Bernoulli likelihood function (which this is, for $N$ independent trials), and a conjugate beta prior (for which the present problem is a special case), the posterior is again a beta distribution: $\mbox{beta}(a+k,b+N-k)$. Here we start with $a=\epsilon$ and $b=1$, so the posterior is $\mbox{beta}(\epsilon+k,1+N-k)$.

P.S. For a derivation of the general beta-Bernoulli update rule, see p. 132 of DBDA2E.

$\endgroup$
  • $\begingroup$ Thanks for the answer! I had the same in mind, but somehow Alpha=0 did not make sense as by definition of Beta distribution Alpha>0. Would that be a case of an improper prior? Also, would the posterior still be a proper Beta distribution? $\endgroup$ – Artur Jan 7 '17 at 14:38
  • $\begingroup$ @Artur check en.wikipedia.org/wiki/… $\endgroup$ – Tim Jan 7 '17 at 14:42
  • $\begingroup$ @Artur: as indicated in my earlier comment, you cannot use this improper prior because the posterior is not defined when $X=n$. $\endgroup$ – Xi'an Jan 7 '17 at 14:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.