2
$\begingroup$

I am curious for myself, but also trying to explain this to others.

The beta distribution is often used as a Bayesian conjugate prior for a binomial likelihood. It is often explained with the example that $\left(\alpha-1\right)$ is analogous to the number of successes and $\left(\beta-1\right)$ is like the number of fails.

As expected, a beta distribution with $\alpha=\beta=1$ is equivalent to a uniform distribution.

But the beta distribution can have values less than 1 (any non-negative number). At the extreme case, $\alpha=\beta=0$ yields a bimodal PDF (probability density function) with values at only 0 and 1. I can still intuit this: it represents a case like flipping a coin - not the probability of heads or tails, but rather the outcomes: there are only 2 possibilities, 0 or 1 (or heads or tails).

But any $\alpha$ or $\beta$ value between 0 and 1 I cannot find a good way to explain or think about. I can calculate it, but not really grok it.

Bonus points for anyone who can help explain the difference between a conjugate prior using what to me seems it should provide no information, which would be a beta distribution with $\alpha=\beta=1$, and what is actually used as a prior with no information, the Jeffrey's Prior, which uses $\alpha=\beta=0.5$.

Addendum

Looks like I need to be clearer. I am looking to understand, conceptually what natural phenomenon might be represented by a beta distribution with $\alpha=\beta=\frac{1}{2}$.

For instance,

  • Binomial distribution with n=10 and k=4 "means": some phenomenon with a bimodal response experienced 4 "successes" in 10 attempts.
  • Poisson distribution with k=2 and $\lambda=4.5$ means: some phenomenon that "typically" happens 4.5 times per hour (or whatever unit of time) only happened twice in the interval.

Or even with positive integer beta distributions, I can say:

  • Beta distribution with $\alpha=4$ and $\beta=7$ means: some phenomenon with a bimodal response had 3 successes and 6 fails in 9 attempts.
    • (I know this one is a bit inaccurate, since beta distributions are continuous and provide a probability density instead of mass, but this is often how it is conceptually viewed or explained, and why it is used as a conjugate prior.)

What sort of similar construct or meaning could I create for the beta distribution with $\alpha=\beta=\frac{1}{2}$?

I am not looking for a plot. As I said earlier, I know how to work with a beta distribution mathematically (plot it, calculate it, etc.) I am just trying to get some natural intuition.

|cite|improve this question
$\endgroup$
  • 3
    $\begingroup$ Have you tried plotting it? $\endgroup$ – jbowman Aug 17 '18 at 19:53
  • 3
    $\begingroup$ What exactly do you mean by "grok it"? Would that mean seeing how such a distribution can arise in nature? (Contemplate the correlation coefficient of two normally-distributed 3-vectors, for instance.) Seeing its implications when used as a prior distribution in a Bayesian model? Seeing a picture of some mathematical representation of it, such as its pdf, cdf, mgf, cgf, or survival function? $\endgroup$ – whuber♦ Aug 17 '18 at 20:21
  • $\begingroup$ @whuber Yes, I mean "how it can arise in nature?" Or maybe "what does it conceptually represent?" I'll update the question to be more specific. $\endgroup$ – Mike Williamson Aug 17 '18 at 22:14
  • 1
    $\begingroup$ I’m not sure why you said $\alpha - 1$ is like the number of prior heads. It should be $\alpha$. The Bayes estimator after observing $Y$ heads in $N$ trials is $(Y + \alpha) / (N + \alpha + \beta)$ so clearly $\alpha$ is representing how many observed heads you think your prior is worth. Jeffreys prior then says your prior information is worth half a heads and half a tails. $\endgroup$ – guy Aug 18 '18 at 2:06
  • 1
    $\begingroup$ @MikeWilliamson that is not the generally accepted comparison. The generally accepted comparison is $\alpha$ for the number of heads and $\beta$ for the number of tails. The Wikipedia page is saying that $\alpha - 1$ and $\beta - 1$ comes from the posterior mode, which is not the usual Bayes estimator. It is standard to consider the posterior mean, not the posterior mode, when intuiting these parameters. $\endgroup$ – guy Aug 18 '18 at 21:44
3
$\begingroup$

Here is a frivolous example that may have some intuitive value.

In US Major League Baseball each team plays 162 games per season. Suppose a team is equally likely to win or lose each of its games. What proportion of the time will such a team have more wins than losses? (In order to have symmetry, if a team's wins and losses are tied at any point, we say it is ahead if it was ahead just before the tie occurred, otherwise behind.)

Suppose we look at a team's win-loss record as the season progresses. For our team with wins and losses are as if determined by tosses of a fair coin, you might think a team would most likely be ahead about half the time throughout a season. Actually, half the time is the least likely proportion of time for being ahead.

The "bathtub shaped" histogram below shows the approximate distribution of the proportion of time during a season that such a team is ahead. The curve is the PDF of $\mathsf{Beta}(.5,.5).$ The histogram is based on 20,000 simulated 162-game seasons for a team where wins and losses are like independent tosses of a fair coin, simulated in R as follows:

set.seed(1212);  m = 20000;  n = 162;  prop.ahead = numeric(m)
for (i in 1:m)
 {
 x = sample(c(-1,1), n, repl=T);  cum = cumsum(x)
 ahead = (c(0, cum) + c(cum,0))[1:n]  # Adjustment for ties
 prop.ahead[i] = mean(ahead >= 0)
 }

cut=seq(0, 1, by=.1); hdr="Proportion of 162-Game Season when Team Leads"  
hist(prop.ahead, breaks=cut, prob=T, col="skyblue2", xlab="Proportion", main=hdr)
curve(dbeta(x, .5, .5), add=T, col="blue", lwd=2)

enter image description here

Note: Feller (Vol. 1) discusses such a process. The CDF of $\mathsf{Beta}(.5,.5)$ is a constant multiple of an arcsine function, so Feller calls it an 'Arcsine Law'.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks, @BruceET ! I want to make sure I understand: you're describing a "zero sum" case (if I win, then you must lose), and the proportion you're showing is the proportion of time that I am ahead. So, this is analogous to a random walk or MCMC type of phenomenon, where even though over time my average will eventually settle back to 0.5 (equal # of wins & losses), the amount of time I am actually ahead or behind will follow this U-shaped distribution because once I'm better than 50/50, it's harder to fall under 50/50. This is a great example to help understand! Thanks! $\endgroup$ – Mike Williamson Aug 18 '18 at 18:24
  • $\begingroup$ Follow-up: then why is $Beta\left(0.5, 0.5\right)$ used as a non-informative distribution (Jeffrey's Prior)? This is information, IMHO... the knowledge that 0 and 1 are more likely than 0.5. $\endgroup$ – Mike Williamson Aug 18 '18 at 18:27
  • $\begingroup$ (a) I think you have the idea of my example just right. Of course, the distribution for 162 games is discrete and BETA(.5,.5) is an approximating distribution, but for the random walk you describe it is the limiting distribution, and that is closer to Feller's presentation. (b) I'm going to leave the Jeffrey's Prior issue to others. There are a couple of Q&A's on the site that give partial discussions of that. But maybe it is worthwhile to post a separate Question asking for a specific example to illustrate BETA(.5,.5) is a better 'noninformative prior' than is BETA(1,1) $\equiv$ UNIF(0,1). $\endgroup$ – BruceET Aug 18 '18 at 18:35
  • $\begingroup$ Thanks! Yeah, I'll post another question regarding Jeffrey's. I didn't really have the thought formulated well enough to post a separate question, and Cross Validated is notorious for being harshly critical when questions are poorly posed. So I was trying to sneak it in as a "rider". $\endgroup$ – Mike Williamson Aug 18 '18 at 18:37
  • 2
    $\begingroup$ Either 'harshly critical' or 'intensely interested' to get the application just right so our answers make sense. $\endgroup$ – BruceET Aug 18 '18 at 18:41
1
$\begingroup$

If you take for example $\alpha=\beta=0.5$, then the pdf looks like a horse-shoe, with high density near ends of the interval $(0,1)$ and low density near $0.5$. So as a prior, it puts a lot of density on the extremes, and that helps the posterior have a similar shape.

I understand it as a device to help the posterior move away from $50\%$ and towards $0$ or $1$, which can be helpful if you are trying to make a binary decision.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Right, I understand. But what natural phenomenon does that actually represent? And why does it carry less information than a true uniform distribution, at least when used as a Jeffrey's prior? $\endgroup$ – Mike Williamson Aug 17 '18 at 22:42
  • $\begingroup$ I can't understand the phenomenon you described above (with the 4 and the 7) so I cannot answer your question. Are you sure is just inaccurate or is it wrong? Also, can you quantify or prove the claim that it carries "less information" please? $\endgroup$ – papgeo Aug 17 '18 at 22:59
  • $\begingroup$ Often, the beta distribution is used as a form for a conjugate prior when the likelihood is generated from a binomial distribution so that the kernels can be multiplied together easily. When used in that way, the beta distribution is said to "mimic" a binomial distribution where $\alpha - 1$ is the number of successes and $\beta - 1$ is the number of fails. $\endgroup$ – Mike Williamson Aug 18 '18 at 0:08
  • $\begingroup$ Regarding the "quantify or prove my claim", I cannot. It's quite possible I am misunderstanding something, which is why I'm asking the question. However, I got that impression from reading about Jeffrey's Prior, where it explicitly states, "its functional dependence on the likelihood, L. L is invariant under reparameterization of the parameter vector" $\endgroup$ – Mike Williamson Aug 18 '18 at 0:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.