6
$\begingroup$

Suppose we observe y successes in n trials where the probability of success in each trial is θ. If we choose a Beta(1, 1) (Uniform) prior then the posterior variance will be smaller than the prior variance. Show that the above isn’t necessarily the case if we choose a general Beta(a, b) prior. That is, find set of values for a, b, n, y where the above is not true.

I have tried many random examples and still have yet found a case where posterior variance is greater than prior variance. Please help me to find a better direction.

Improve this question
$\endgroup$
1
  • 4
    $\begingroup$ Hint: the variance of a Beta$(a,b)$ distribution is $ab/((a+b)^2(a+b+1)).$ Compare the variance after observing $y$ successes and $n-y$ failures to the variance before observing them. You could also use your statistical intuition: the variance might go up if the observations are surprising: that is, very different than what you might expect based on $a$ and $b.$ Look at values of $a$ and $b$ corresponding to an extreme case and suppose $y$ and $n-y$ correspond to the other extreme. $\endgroup$
    – whuber ♦
    Commented Sep 3, 2018 at 20:11

1 Answer 1

Reset to default
9
$\begingroup$

Gamma-Poisson: Suppose your prior for Poisson data is $\lambda \sim \mathsf{Gamma}(\text{shape}=4, \text{rate}=1/4).$ This distribution has mean 16 and variance 64. It's 95th percentile is about 31.

qgamma(.95, 4, .25)
[1] 31.01463

But your first Poisson observation is $x = 500.$ Then your posterior distribution is $\mathsf{Gamma}(\text{shape}=4+500, \text{rate}=1/4+1) = \mathsf{Gamma}(\text{shape}=504, \text{rate}=1.25),$ which has mean $504/1.25 = 403.2$ and variance $504/1.25^2 = 322.56 > 64.$

enter image description here

Beta-binomial: For a beta-binomial example (along lines suggested by @whuber), tossing a coin. Suppose your prior for $P(\text{Head})=\theta$ is $\mathsf{Beta}(10,1),$ for a coin heavily biased in favor of Heads. Then on four tosses you get two Heads and two tails. Find the variances of the prior and posterior distributions.

enter image description here

Note: In both examples, the idea is to have an informative prior and then a small amount of data that doesn't match the prior. With a large amount of data, the data can overwhelm the prior, yielding a posterior with small variance. Now I hope you can find examples of your own.

Improve this answer
$\endgroup$
1
  • 4
    $\begingroup$ +1. Also of interest would be to demonstrate that (1) given the prior variance, there is a finite upper bound to the ratio of the posterior variance and prior variance but (2) as one varies the prior, this upper bound can grow arbitrarily large. (We should leave these to the OP to show.) $\endgroup$
    – whuber ♦
    Commented Sep 4, 2018 at 14:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.