Let's say I give you a 10-sided die and tell you that it is biased in some way. I then let you roll the die 5 times and you observe, let's say, 8, 8, 8, 2, and 8. Lastly, I ask you what your belief is regarding how the die is biased.

How would you use Bayes' Theorem in this situation to go from complete ignorance to an informed belief?

  • $\begingroup$ Do you mean literally you will say "the die is biased" or will you provide an explicit prior distribution for your belief? $\endgroup$ – whuber Aug 30 '13 at 20:05
  • $\begingroup$ The question is as written. An unknown bias; hence, no explicit prior distribution. $\endgroup$ – ac11ca Sep 4 '13 at 0:51
  • $\begingroup$ As far as I can tell, then, asserting there is an "unknown bias" has no information content, because infinitesimally small amounts of bias are indistinguishable from lack of bias. $\endgroup$ – whuber Sep 4 '13 at 16:35

For a Bayesian approach you need a prior distribution and a likelihood.

A reasonable prior here is a Dirichlet distribution with equal probability for the 10 sides (the 10 parameters all equal to 1).

A reasonable likelihood is the multinomial.

Now just multiply the prior and the likelihood (and normalize) and you have your posterior. Since the Dirichlet and multinomial are conjugate the posterior will be a Dirichlet with new parameters (in this case the parameter for 2 will be 2 and for 8 will be 5 and all the others will remain 1). The mode of this distribution would be that the probability of a 2 is 2/15, an 8 is 5/15, and all others are 1/15.

Of course other priors and likelihoods could be used that would lead to other posteriors.

  • $\begingroup$ Your "reasonable prior" contradicts the assumptions of the question, which is that the die is "biased in some way." $\endgroup$ – whuber Aug 30 '13 at 20:06
  • $\begingroup$ @whuber, But if it is "equally likely" to be biased towards each of the sides (since we don't know how it is biased) then we still get a prior with equal parameter values. However choosing a Dirichlet with equal parameters that are less than 1 would be consistent with a prior belief of some bias, values close to 1 (0.9999999) would indicate a prior belief of only slight bias whereas values close to 0 (0.0001) would be a belief of strong bias (we are not told how strong the bias is). But then I couldn't have done the last couple of computations in my head. $\endgroup$ – Greg Snow Aug 30 '13 at 20:23
  • 1
    $\begingroup$ It's unclear whether anybody can do any relevant computations until the O.P. clears up the meaning of "the die is biased." I don't see any reference to "equally likely" in the question, either. $\endgroup$ – whuber Aug 30 '13 at 20:32
  • $\begingroup$ @whuber: I assumed he meant it is biased, but the bias is unknown. Given no other information, why not just use a Jeffreys prior? ($\textrm{Dirichlet}(\frac12, \dotsc)$) $\endgroup$ – Neil G Aug 30 '13 at 20:41
  • $\begingroup$ Shouldn't the likelihood be over the belief, and so it should also be Dirichlet? Here, there is no choice — it's not about reasonable or unreasonable — it must be Dirichlet. (I don't agree with the last sentence; other likelihoods cannot be used.) $\endgroup$ – Neil G Aug 30 '13 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.