4
$\begingroup$

On p38 of Lee and Wagenmakers (2012) "Bayesian Cognitive Modeling: A Practical Course" the following passage appears:

"One of the nice properties of using the θ ~ Beta (α,β) prior distribution for a rate is that it has a natural interpretation. The α and β values can be thought of as counts of, respectively, “prior successes” and “prior failures.” This means that using a θ ~ Beta (3,1) prior corresponds to having the prior information that 4 previous observations have been made, and 3 of them were successes. Or, more elaborately, starting with a θ ~ Beta (3,1) is the same as starting with a θ ~ Beta (1,1), and then seeing data giving two more successes (i.e., the posterior distribution in the second scenario will be same as the prior distribution in the first)."

I'm finding it hard to match that explanation with the following diagram, which I generated using MATLAB. In all instances in the figure the 'b' parameter is three times the size of 'a'.

Beta distributions

I understand why the distribution becomes more peaked around 0.25 in the Beta (100,300) condition than in the (10,30) condition - there's stronger evidence for θ being 0.25.

However, I don't understand what's going on in the (0.25,0.75) and (1,3) conditions. would have thought they'd both be fat-tailed distributions centred on 0.25. I don't understand why the mode of both distributions seems to be around 0.

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
10
$\begingroup$

Since the domain is bounded by 0 and 1, in order for the mean to be 0.25 and the variance to be large, most of the mass has to be pushed up against the boundaries. There's simply no other way for the variance to be large enough.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I understand that most of the mass has to get pushed closer to the boundaries, but (with reference to the blue 1,3 condition) I don't understand why the distribution doesn't have a small hump at 0.25, and then a lot of spread toward 0 and 1. Isn't 0.25 still the 'best guess' for what theta is? Intuitively I had expected a distribution with a very small peak at 0.25 and then really fat tails spreading out toward the edges. $\endgroup$
    – user1205901 - Слава Україні
    Commented Oct 30, 2014 at 0:53
  • $\begingroup$ You could make a distribution that looks the way that you are describing, but the Beta doesn't do that. Basically the mode does not coincide with the mean. $\endgroup$
    – Tom Minka
    Commented Oct 30, 2014 at 12:23
  • $\begingroup$ Does the Lee and Wagenmakers interpretation still hold in the 1,3 condition? I get that the mean value of theta is 0.25 there, but I can't understand how we can (coin toss example) observe one head and three tails, and then say that the modal value of theta is zero. $\endgroup$
    – user1205901 - Слава Україні
    Commented Oct 30, 2014 at 23:52
  • 3
    $\begingroup$ Their interpretation is aimed specifically at the mean of the distribution. If you want an interpretation that works for the mode, you should subtract 1 from both counts. In that interpretation, Beta(1,3) corresponds to observing 0 heads and 2 tails. $\endgroup$
    – Tom Minka
    Commented Oct 31, 2014 at 15:58
3
$\begingroup$

Beta(0.25, 0.75) corresponds to natural parameters that are negative. That is like making "negative" observations and so you're seeing the pointwise reciprocal of Beta(1.75, 1.25). Beta(1,1) is flat (and corresponds to zero natural parameters) — moving towards that flattens things out.

If you add 1,1 to each of the parameter pairs that you graphed, all of the modes will line up as you expect.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.