6
$\begingroup$

I am working through an example on doing Bayesian inference on binomial distribution using a flat prior, and trying to understand the impact of choosing a prior. I know that if we work with a flat prior $\theta \in [0, 1]$, it is equivalent to $\theta \sim B(1,1)$, and we can use the conjugate pair of binomial distribution to conclude the posterior as $\theta \sim B(\alpha+1, \beta+1)$. Therefore our point estimate of $\theta$ equals to the mean of the distribution, $= \frac {\alpha+1} {\alpha + \beta + 2}$. However, I want to know what would happens if we choose the prior with a different uniform distribution, say $\theta \in [0.1,1]$. I don't know if the concluded posterior would be the same ($\theta \sim B(\alpha+1, \beta+1)$), but the point estimate would change; or if the concluded posterior would be different. I have thought either:

  • do $E[\frac{X+a}{b}] = \frac {E[X] + a}{b}$, with the assumption of the new $\theta = \frac{x-0.1}{1-0.1}$; or,
  • changing the calculation of the mean by changing the integration range

but both method seems wrong. So I'm stuck at this problem and would like some advice on this.

Thanks a lot for your time!

* Note: I know in general we should not use flat prior, and using normal prior gives a much better result, but this is kind of the point of the example I'm working on, on investigating the effect of selecting the prior.

Edit1: by changing the calculation of the mean, I mean normally, we use $\int_0^1 x \frac{x^{\alpha} (1-x)^{\beta}} {B(\alpha+1, \beta+1)} dx = \frac {\alpha+1} {\alpha + \beta + 2}$. To calculate the mean where $\theta \in [0,1]$, we do $\int_{0.1}^1 x \frac{x^{\alpha} (1-x)^{\beta}} {B(\alpha+1, \beta+1)} dx$

$\endgroup$
1
  • $\begingroup$ Your mean will not quite be correct as it has the wrong denominator, but $\dfrac{\int\limits_{0.1}^1 x^{\alpha+1}(1-x)^\beta\,dx}{\int\limits_{0.1}^1 x^{\alpha}(1-x)^\beta\,dx}$ would be for a truncated Beta distribution, or if you start with Beta(1,1) prior then $\dfrac{\int\limits_{0.1}^1 x^{\alpha+2}(1-x)^{\beta+1}\,dx}{\int\limits_{0.1}^1 x^{\alpha+1}(1-x)^{\beta+1}\,dx}$ $\endgroup$
    – Henry
    Aug 3 at 18:49
6
$\begingroup$

If you start with a uniform prior over the support of the parameter, you get the normalized likelihood back as the posterior (I'm going to restrict my attention to cases where the likelihood can be normalized to a density).

So if you start with some $\theta$ with support on $(0,1)$ and a likelihood that's proportional to a $B(\alpha,\beta)$ (for some $\alpha$ and $\beta$ say), then the posterior is that same beta distribution ($B(\alpha,\beta)$) back, naturally.

[Note that my $\alpha$ and $\beta$ are slightly different from yours. You didn't specify your binomial, doing so would probably simplify the discussion a little, but it's okay as is.]

If you replace that uniform prior with a "shorter" contiguous uniform one (i.e. whose support is wholly contained in the first one), you simply get a truncated version of the same $B(\alpha,\beta)$ posterior, where it's truncated to the support of the new prior.

The same idea applies to whatever other likelihood you might have.

(This is straightforward if you work with indicator notation.)

$\endgroup$
1
  • $\begingroup$ Thx. I end up running a simulation on the beta distribution, and then manually filtering the numbers I don't need (base on the indicator function), and calculate the mean afterwards. I'm not sure if this is statistically correct but I think the result make sense in demonstrating the effect on changing the base limit $\endgroup$
    – Ian
    Aug 4 at 11:09
5
$\begingroup$

Comments:

If the prior distribution has support $[.1,1],$ then the posterior distribution has support contained in or equal to $[.1,1],$ so the posterior distribution could not be any ordinary (two-parameter) beta distribution.

Also, if one tries to use a normal prior for a binomial success probability then it has to be truncated to have support $[0,1].$ Such a truncated normal prior is not conjugate to a binomial likelihood, so the posterior has to be computed by numerical methods.

Moreover, there are inferential applications relating to the prevalence of a disease (which must lie in $[0,1])$ where fragmentary information from medical screening tests in which traditional (non-Bayesian) inference can lead to point and interval estimates of prevalence with nonsensical values outside of $[0,1].$ However, a Bayesian approach with a beta prior must give point and interval estimates within $[0,1].$ One example is given here. [In this particular application, a Gibbs sampler is used to find the posterior distribution.]

Note: Undeniably, $\mathsf{Beta}(1,1) \equiv \mathsf{Unif}(0,1)$ is a "flat" prior distribution for the binomial success probability $\theta.$ However, technically there is controversy whether this distribution is noninformative. The Jeffries prior $\mathsf{Beta}(.5,.5)$ has been proposed as a more nearly noninformative prior distribution.

$\endgroup$
0
4
$\begingroup$

The standard proportionality result for the posterior still holds, but the posterior is now concentrated on the same restricted set as the prior. To see this, consider the general case where you restrict your prior to the set $\mathscr{D}$. If you use a prior proportionate to $\pi(\theta) \cdot \mathbb{I}(\theta \in \mathscr{D})$ then you get the posterior:

$$\pi(\theta|\mathbf{x}) \propto L_\mathbf{x}(\theta) \cdot \pi(\theta) \cdot \mathbb{I}(\theta \in \mathscr{D}),$$

which is proportionate to the unrestricted posterior, but restricted over the set $\mathscr{D}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.