What happens if I change the range of a flat prior for Bayesian inference?

Question

I am working through an example on doing Bayesian inference on binomial distribution using a flat prior, and trying to understand the impact of choosing a prior. I know that if we work with a flat prior $\theta \in [0, 1]$, it is equivalent to $\theta \sim B(1,1)$, and we can use the conjugate pair of binomial distribution to conclude the posterior as $\theta \sim B(\alpha+1, \beta+1)$. Therefore our point estimate of $\theta$ equals to the mean of the distribution, $= \frac {\alpha+1} {\alpha + \beta + 2}$. However, I want to know what would happens if we choose the prior with a different uniform distribution, say $\theta \in [0.1,1]$. I don't know if the concluded posterior would be the same ($\theta \sim B(\alpha+1, \beta+1)$), but the point estimate would change; or if the concluded posterior would be different. I have thought either:

• do $E[\frac{X+a}{b}] = \frac {E[X] + a}{b}$, with the assumption of the new $\theta = \frac{x-0.1}{1-0.1}$; or,
• changing the calculation of the mean by changing the integration range

but both method seems wrong. So I'm stuck at this problem and would like some advice on this.

Thanks a lot for your time!

* Note: I know in general we should not use flat prior, and using normal prior gives a much better result, but this is kind of the point of the example I'm working on, on investigating the effect of selecting the prior.

Edit1: by changing the calculation of the mean, I mean normally, we use $\int_0^1 x \frac{x^{\alpha} (1-x)^{\beta}} {B(\alpha+1, \beta+1)} dx = \frac {\alpha+1} {\alpha + \beta + 2}$. To calculate the mean where $\theta \in [0,1]$, we do $\int_{0.1}^1 x \frac{x^{\alpha} (1-x)^{\beta}} {B(\alpha+1, \beta+1)} dx$

Answer 1

Comments:

If the prior distribution has support $[.1,1],$ then the posterior distribution has support contained in or equal to $[.1,1],$ so the posterior distribution could not be any ordinary (two-parameter) beta distribution.

Also, if one tries to use a normal prior for a binomial success probability then it has to be truncated to have support $[0,1].$ Such a truncated normal prior is not conjugate to a binomial likelihood, so the posterior has to be computed by numerical methods.

Moreover, there are inferential applications relating to the prevalence of a disease (which must lie in $[0,1])$ where fragmentary information from medical screening tests in which traditional (non-Bayesian) inference can lead to point and interval estimates of prevalence with nonsensical values outside of $[0,1].$ However, a Bayesian approach with a beta prior must give point and interval estimates within $[0,1].$ One example is given here. [In this particular application, a Gibbs sampler is used to find the posterior distribution.]

Note: Undeniably, $\mathsf{Beta}(1,1) \equiv \mathsf{Unif}(0,1)$ is a "flat" prior distribution for the binomial success probability $\theta.$ However, technically there is controversy whether this distribution is noninformative. The Jeffries prior $\mathsf{Beta}(.5,.5)$ has been proposed as a more nearly noninformative prior distribution.

Answer 2

If you start with a uniform prior over the support of the parameter, you get the normalized likelihood back as the posterior (I'm going to restrict my attention to cases where the likelihood can be normalized to a density).

So if you start with some $\theta$ with support on $(0,1)$ and a likelihood that's proportional to a $B(\alpha,\beta)$ (for some $\alpha$ and $\beta$ say), then the posterior is that same beta distribution ($B(\alpha,\beta)$) back, naturally.

[Note that my $\alpha$ and $\beta$ are slightly different from yours. You didn't specify your binomial, doing so would probably simplify the discussion a little, but it's okay as is.]

If you replace that uniform prior with a "shorter" contiguous uniform one (i.e. whose support is wholly contained in the first one), you simply get a truncated version of the same $B(\alpha,\beta)$ posterior, where it's truncated to the support of the new prior.

The same idea applies to whatever other likelihood you might have.

(This is straightforward if you work with indicator notation.)

Answer 3

The standard proportionality result for the posterior still holds, but the posterior is now concentrated on the same restricted set as the prior. To see this, consider the general case where you restrict your prior to the set $\mathscr{D}$. If you use a prior proportionate to $\pi(\theta) \cdot \mathbb{I}(\theta \in \mathscr{D})$ then you get the posterior:

$$\pi(\theta|\mathbf{x}) \propto L_\mathbf{x}(\theta) \cdot \pi(\theta) \cdot \mathbb{I}(\theta \in \mathscr{D}),$$

which is proportionate to the unrestricted posterior, but restricted over the set $\mathscr{D}$.