Suppose today I'm going to flip a coin. I believe that 9 of 10 flips will come up heads. I flip the coin and 8 of 10 are heads. Is my distribution of belief

  1. beta(9+8, 1+2)
  2. beta(1+9+8, 1+1+2)
  3. beta(m+8, n+2) where I choose m and n based on graphing beta(m,n) and deciding if it feels right. (This is not a facetious answer, it's a real suggestion.)
  4. Hidden option 4.

If I use #1, I believe that is an improper prior, but Wikipedia claims some statisticians use them.

This choice is not obvious to me.

  • 5
    $\begingroup$ If you want your prior to include the information of a hypothetical sample of 10 flips out of which 9 came out as heads, then your prior is $\mathrm{Beta}(9,1)$. The mean of the prior is $0.9$. After flipping the coin 10 times out of which 8 are heads, your posterior is $\mathrm{Beta}(17,3)$. The posterior mean is $0.85$. You can use an improper prior, that is no problem. The Beta distribution is the conjugate distribution of the binomial. Check out this post for a fully worked example using the Beta distribution. $\endgroup$ Jul 19, 2013 at 15:54
  • 4
    $\begingroup$ Choice #1 is not an improper prior. $\endgroup$
    – QuantIbex
    Jul 19, 2013 at 15:58
  • $\begingroup$ @QuantIbex Your right, of course (+1). Sorry, my wording was inaccurate: I didn't want to imply that #1 was improper, I just wanted to say that an improper prior is not a problem automatically, at least not mathematically. $\endgroup$ Jul 19, 2013 at 16:10
  • 2
    $\begingroup$ For the Beta distribution, the improper prior is $\lim_{a \to 0}\rm{Beta(a,a)}$. Improper means it isn't a probability density, e.g. doesn't integrate to 1. You might want to consider a non-informative prior, which is flat for all possible values of your parameter: the uniform distribution is suitable here, which is a special case of the Beta distribution, i.e. $\rm{Beta(1,1)}$. See online beta graphing. $\endgroup$
    – TooTone
    Jul 19, 2013 at 16:10
  • 1
    $\begingroup$ Improper priors are acceptable so long as they produce proper posteriors. For the beta prior with a binomial likelihood, any improper prior will produce a proper posterior provided you observe one (or more) heads and one (or more) tails. $\endgroup$
    – Sycorax
    Jul 19, 2013 at 18:27

1 Answer 1


If your prior belief is that 9 of the 10 coin flips will come up heads, then you want the expectation of your prior to be 0.9. Given $X \sim \mathrm{Beta}(\alpha,\beta)$ (for conjugacy in the beta-binomial model), then $E[X] = \alpha/(\alpha+\beta) = 0.9$, so you can use this as your first constraint. Obviously this leaves you with an infinite number of possibilities, so we'll need a second constraint which will represent your subjective confidence in this prior belief: the variance of your prior, $\operatorname{var}[X] = \frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}\ $

Decide how you want to set your variance and solve the system of equations for $\alpha$ and $\beta$ to define the parameters for your prior. Justifying your choice of variance here may be difficult: you can always err on the side of a wider (i.e. less informative) variance. The wider you set the variance, the closer your prior will approximate a uniform distribution.

If you want a truly uninformative prior, you should consider using Jeffrey's Prior.

  • 1
    $\begingroup$ There is no such thing as a trully uninformative prior. $\endgroup$ Jul 19, 2013 at 16:54
  • $\begingroup$ Ok, that's fair. I consider it a 'very effective' uninformative prior, but you're right, I shouldn't be using such rigid/suggestive language. $\endgroup$
    – David Marx
    Jul 19, 2013 at 16:57
  • $\begingroup$ Thanks for your answer. The gist is: you have to say something about the variance to set the parameters. Seems sensible. I think the human-understandable way to approach it might be "I believe the mean is 0.9, and >0.8 with 95% confidence." It would be interesting to see the easiest way to do that. $\endgroup$
    – dfrankow
    Jul 19, 2013 at 20:13
  • $\begingroup$ By the way, I could not make heads or tails of Jeffrey's Prior. I'm sure it's very important and useful, but I won't worry about it. $\endgroup$
    – dfrankow
    Jul 19, 2013 at 20:14
  • $\begingroup$ @dfrankow here's a better explanation: cs.berkeley.edu/~jordan/courses/260-spring10/lectures/… $\endgroup$
    – David Marx
    Jul 19, 2013 at 23:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.