5
$\begingroup$

Let's say I give you a 10-sided die and tell you that it is biased in some way. I then let you roll the die 5 times and you observe, let's say, 8, 8, 8, 2, and 8. Lastly, I ask you what your belief is regarding how the die is biased.

How would you use Bayes' Theorem in this situation to go from complete ignorance to an informed belief?

Improve this question
$\endgroup$
3
  • $\begingroup$ Do you mean literally you will say "the die is biased" or will you provide an explicit prior distribution for your belief? $\endgroup$
    – whuber ♦
    Commented Aug 30, 2013 at 20:05
  • $\begingroup$ The question is as written. An unknown bias; hence, no explicit prior distribution. $\endgroup$
    – ac11ca
    Commented Sep 4, 2013 at 0:51
  • $\begingroup$ As far as I can tell, then, asserting there is an "unknown bias" has no information content, because infinitesimally small amounts of bias are indistinguishable from lack of bias. $\endgroup$
    – whuber ♦
    Commented Sep 4, 2013 at 16:35

1 Answer 1

Reset to default
3
$\begingroup$

For a Bayesian approach you need a prior distribution and a likelihood.

A reasonable prior here is a Dirichlet distribution with equal probability for the 10 sides (the 10 parameters all equal to 1).

A reasonable likelihood is the multinomial.

Now just multiply the prior and the likelihood (and normalize) and you have your posterior. Since the Dirichlet and multinomial are conjugate the posterior will be a Dirichlet with new parameters (in this case the parameter for 2 will be 2 and for 8 will be 5 and all the others will remain 1). The mode of this distribution would be that the probability of a 2 is 2/15, an 8 is 5/15, and all others are 1/15.

Of course other priors and likelihoods could be used that would lead to other posteriors.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Your "reasonable prior" contradicts the assumptions of the question, which is that the die is "biased in some way." $\endgroup$
    – whuber ♦
    Commented Aug 30, 2013 at 20:06
  • $\begingroup$ @whuber, But if it is "equally likely" to be biased towards each of the sides (since we don't know how it is biased) then we still get a prior with equal parameter values. However choosing a Dirichlet with equal parameters that are less than 1 would be consistent with a prior belief of some bias, values close to 1 (0.9999999) would indicate a prior belief of only slight bias whereas values close to 0 (0.0001) would be a belief of strong bias (we are not told how strong the bias is). But then I couldn't have done the last couple of computations in my head. $\endgroup$
    – Greg Snow
    Commented Aug 30, 2013 at 20:23
  • 1
    $\begingroup$ It's unclear whether anybody can do any relevant computations until the O.P. clears up the meaning of "the die is biased." I don't see any reference to "equally likely" in the question, either. $\endgroup$
    – whuber ♦
    Commented Aug 30, 2013 at 20:32
  • $\begingroup$ @whuber: I assumed he meant it is biased, but the bias is unknown. Given no other information, why not just use a Jeffreys prior? ($\textrm{Dirichlet}(\frac12, \dotsc)$) $\endgroup$
    – Neil G
    Commented Aug 30, 2013 at 20:41
  • $\begingroup$ Shouldn't the likelihood be over the belief, and so it should also be Dirichlet? Here, there is no choice — it's not about reasonable or unreasonable — it must be Dirichlet. (I don't agree with the last sentence; other likelihoods cannot be used.) $\endgroup$
    – Neil G
    Commented Aug 30, 2013 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.