5
$\begingroup$

Suppose today I'm going to flip a coin. I believe that 9 of 10 flips will come up heads. I flip the coin and 8 of 10 are heads. Is my distribution of belief

  1. beta(9+8, 1+2)
  2. beta(1+9+8, 1+1+2)
  3. beta(m+8, n+2) where I choose m and n based on graphing beta(m,n) and deciding if it feels right. (This is not a facetious answer, it's a real suggestion.)
  4. Hidden option 4.

If I use #1, I believe that is an improper prior, but Wikipedia claims some statisticians use them.

This choice is not obvious to me.

$\endgroup$
  • 5
    $\begingroup$ If you want your prior to include the information of a hypothetical sample of 10 flips out of which 9 came out as heads, then your prior is $\mathrm{Beta}(9,1)$. The mean of the prior is $0.9$. After flipping the coin 10 times out of which 8 are heads, your posterior is $\mathrm{Beta}(17,3)$. The posterior mean is $0.85$. You can use an improper prior, that is no problem. The Beta distribution is the conjugate distribution of the binomial. Check out this post for a fully worked example using the Beta distribution. $\endgroup$ – COOLSerdash Jul 19 '13 at 15:54
  • 4
    $\begingroup$ Choice #1 is not an improper prior. $\endgroup$ – QuantIbex Jul 19 '13 at 15:58
  • $\begingroup$ @QuantIbex Your right, of course (+1). Sorry, my wording was inaccurate: I didn't want to imply that #1 was improper, I just wanted to say that an improper prior is not a problem automatically, at least not mathematically. $\endgroup$ – COOLSerdash Jul 19 '13 at 16:10
  • 2
    $\begingroup$ For the Beta distribution, the improper prior is $\lim_{a \to 0}\rm{Beta(a,a)}$. Improper means it isn't a probability density, e.g. doesn't integrate to 1. You might want to consider a non-informative prior, which is flat for all possible values of your parameter: the uniform distribution is suitable here, which is a special case of the Beta distribution, i.e. $\rm{Beta(1,1)}$. See online beta graphing. $\endgroup$ – TooTone Jul 19 '13 at 16:10
  • 1
    $\begingroup$ Improper priors are acceptable so long as they produce proper posteriors. For the beta prior with a binomial likelihood, any improper prior will produce a proper posterior provided you observe one (or more) heads and one (or more) tails. $\endgroup$ – Sycorax Jul 19 '13 at 18:27
6
$\begingroup$

If your prior belief is that 9 of the 10 coin flips will come up heads, then you want the expectation of your prior to be 0.9. Given $X \sim \mathrm{Beta}(\alpha,\beta)$ (for conjugacy in the beta-binomial model), then $E[X] = \alpha/(\alpha+\beta) = 0.9$, so you can use this as your first constraint. Obviously this leaves you with an infinite number of possibilities, so we'll need a second constraint which will represent your subjective confidence in this prior belief: the variance of your prior, $\operatorname{var}[X] = \frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}\ $

Decide how you want to set your variance and solve the system of equations for $\alpha$ and $\beta$ to define the parameters for your prior. Justifying your choice of variance here may be difficult: you can always err on the side of a wider (i.e. less informative) variance. The wider you set the variance, the closer your prior will approximate a uniform distribution.

If you want a truly uninformative prior, you should consider using Jeffrey's Prior.

$\endgroup$
  • 1
    $\begingroup$ There is no such thing as a trully uninformative prior. $\endgroup$ – COOLSerdash Jul 19 '13 at 16:54
  • $\begingroup$ Ok, that's fair. I consider it a 'very effective' uninformative prior, but you're right, I shouldn't be using such rigid/suggestive language. $\endgroup$ – David Marx Jul 19 '13 at 16:57
  • $\begingroup$ Thanks for your answer. The gist is: you have to say something about the variance to set the parameters. Seems sensible. I think the human-understandable way to approach it might be "I believe the mean is 0.9, and >0.8 with 95% confidence." It would be interesting to see the easiest way to do that. $\endgroup$ – dfrankow Jul 19 '13 at 20:13
  • $\begingroup$ By the way, I could not make heads or tails of Jeffrey's Prior. I'm sure it's very important and useful, but I won't worry about it. $\endgroup$ – dfrankow Jul 19 '13 at 20:14
  • $\begingroup$ @dfrankow here's a better explanation: cs.berkeley.edu/~jordan/courses/260-spring10/lectures/… $\endgroup$ – David Marx Jul 19 '13 at 23:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.