Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now
623

The short version is that the Beta distribution can be understood as representing a distribution of probabilities- that is, it represents all the possible values of a probability when we don't know what that probability is. Here is my favorite intuitive explanation of this: Anyone who follows baseball is familiar with batting averages- simply the number of ...


48

A Beta distribution is used to model things that have a limited range, like 0 to 1. Examples are the probability of success in an experiment having only two outcomes, like success and failure. If you do a limited number of experiments, and some are successful, you can represent what that tells you by a beta distribution. Another example is order statistics....


41

The Beta distribution also appears as an order statistic for a random sample of independent uniform distributions on $(0,1)$. Precisely, let $U_1$, $\ldots$, $U_n$ be $n$ independent random variables, each having the uniform distribution on $(0,1)$. Denote by $U_{(1)}$, $\ldots$, $U_{(n)}$ the order statistics of the random sample $(U_1, \ldots, U_n)$, ...


28

There are two principal motivations: First, the beta distribution is conjugate prior to the Bernoulli distribution. That means that if you have an unknown probability like the bias of a coin that you are estimating by repeated coin flips, then the likelihood induced on the unknown bias by a sequence of coin flips is beta-distributed. Second, a consequence ...


24

Let's assume a seller on some e-commerce web-site receives 500 ratings of which 400 are good and 100 are bad. We think of this as the result of a Bernoulli experiment of length 500 which led to 400 successes (1 = good) while the underlying probability $p$ is unknown. The naive quality in terms of ratings of the seller is 80% because 0.8 = 400 / 500. But ...


14

There are several problems with your approach. First, you want to use confidence intervals for something that they were not designed for. If $p$ varies, then confidence interval will not show you how does it vary. Check Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean? to learn more about confidence intervals. Moreover, ...


12

That is one useful interpretation of the Beta distribution when it is used as a conjugate prior distribution to the binomial distribution. It breaks down a bit when you consider the possibility that it is perfectly legitimate for $\alpha + \beta < 1$ or even for $\alpha + \beta < 1/2$, meaning that $\alpha + \beta$ being the prior sample size is also ...


11

Try this: $p_i\stackrel{iid}{\sim} Be(\alpha,\beta)$ and $p(\alpha,\beta)\propto (\alpha+\beta)^{-5/2}$. I believe the issue you are running into is that $ss\sim Ga(\gamma,\gamma)$ with $\gamma\to 0$ results in the improper $p(ss)\propto 1/ss$ prior. This prior, together with your uniform prior on mu, results in an improper posterior. Despite the fact that ...


10

I'm glad you mentioned this example, as one project I am working on is writing a whole chapter on Bayesian A/B testing. We are interested in two quantities: $P( p_A > p_B \;|\; data)$ and some measure of "increase". I'll discuss the $P( p_A > p_B \;|\; data)$ quantity first. There are no error bounds on $P( p_A > p_B \;|\; \text{data})$, it is a ...


9

This is a story about degrees of freedom and statistical parameters and why it is nice that the two have a direct simple connection. Historically, the "$-1$" terms appeared in Euler's studies of the Beta function. He was using that parameterization by 1763, and so was Adrien-Marie Legendre: their usage established the subsequent mathematical convention. ...


9

There are at least two ways of seeing this. The urn interpretation of the distribution can be shown to be The beta-binomial distribution can also be motivated via an urn model for positive integer values of $\alpha$ and $\beta$, known as the Polya urn model. Specifically, imagine an urn containing $\alpha$ red balls and $\beta$ black balls, where ...


8

The best --- and standard ways to handle underdispersed Poisson data is by using a generalized Poisson, or perhaps a hurdle model. Three parameter count models can also be used for underdispersed data; eg Faddy-Smith, Waring, Famoye, Conway-Maxwell and other generalized count models. The only drawback with these is interpretability. But for general ...


8

You're looking for the hypergeometric distribution. Incidentally, this link is one of the first two hits on Google for "beta binomial" "conjugate prior".


8

So far the preponderance of answers covered the rationale for Beta RVs being generated as the prior for a sample proportions, and one clever answer has related Beta RVs to order statistics. Beta distributions also arise from a simple relationship between two Gamma(k_i, 1) RVs, i=1,2 call them X and Y. X/(X+Y) has a Beta distribution. Gamma RVs already have ...


7

If you just want to be able to write down the probability mass function, you have a lot of flexibility, basically because you can repeatedly use integration by parts. As long as you can integrate the distribution of $X$ repeatedly to get closed form expressions, you get at worst a double sum of closed form expressions for the pmf of the compound distribution....


6

To quote Wikipedia: The beta-binomial distribution is the binomial distribution in which the probability of success at each trial is not fixed but random and follows the beta distribution. You can read more about the beta distribution here, the binomial distribution here, and the beta binomial distribution here


6

Have you seen this paper: Kadane, 2016, Sums of Possibly Associated Bernoulli Variables: The Conway-Maxwell-Binomial Distribution? In this paper, you can see that the conditions assumed in your question i.e. having $n$ marginally Binomial r.v. with the same probability of success, $p$, and the same pairwise correlation, $\rho$, between all pairs does not ...


5

Short summary: if the $p_i$s are independent, it's the binomial. If the $p_i$s are all equal, it's the beta-binomial. By $X_i \sim \textrm{Bernoulli}(p_i)$, you must mean the conditional distribution $X_i\mid p_i \sim \textrm{Bernoulli}(p_i)$. The marginal distribution of $X_i$ (that is, the distribution obtained by averaging over different values of $p_i$s)...


5

The hierarchical model You don't actually even need the marginal probability mass function $m()$, you actually only need the marginal moments of $Y$. In this tutorial, Casella (1992) is assuming the following hierarchical model for a response count $Y$: $$Y|p\sim\mbox{bin}(n,p)$$ and $$p \sim \mbox{Beta}(\lambda,\lambda)$$ with $n=50$. Moments of the prior ...


4

In linear random effects models, the additional source of variability due to the random effect results in an additive increase in the total variance. In the beta-binomial model the additional variability is accounted for by a multiplicative overdispersion factor $\phi$. The random effect here is modeled implicitly. \begin{align} & y_{ji}|\pi_{j} \...


4

My intuition says that it "weighs" both the current proportion of success "$x$" and current proportion of failure "$(1-x)$": $f(x;\alpha,\beta) = \text{constant}\cdot x^{\alpha-1}(1-x)^{\beta-1}$. Where the constant is $1/B(\alpha,\beta)$. The $\alpha$ is like a "weight" for success's contribution. The $\beta$ is like a "weight" for failure's contribution. ...


4

The beta distribution has two shape parameters, $\alpha$ and $\beta$. The parameters $\alpha$ and $\beta$ can be interpreted as the numbers of negative and positive outcomes, respectively. To fit a model with a beta-binomial distribution using vglm, you have to use the numbers of negative and positive outcomes as the dependent variable, i.e., cbind(x, y - x)...


4

Let $f(x;\alpha,\beta)$ denote the beta distribution parameterized by $\alpha$ and $\beta$. The aim is to express $f(x;\alpha,\beta)$ in terms of $f(x;\alpha-1,\beta)$ and $f(x;\alpha,\beta-1)$. $$f(x;\alpha, \beta) = \frac{\Gamma(\alpha + \beta)}{\Gamma(\alpha)\Gamma(\beta)} x^{\alpha - 1}x^{\beta - 1}$$ It turns out the Gamma function satisfies the ...


4

AFAIK, you cannot say that $p > \frac{1}{2}$ or even $\mathbb E[p]>\frac{1}{2}$ is an "observation" or "event", but rather a constraint on your model parameter(s). The term "observation" is usually reserved for specific realizations of a random variable (i.e. draws from a distribution). In your model, one could plausibly observe $p$ (numbers between 0 ...


3

This Blog entry here http://lingpipe-blog.com/2009/09/23/bayesian-estimators-for-the-beta-binomial-model-of-batting-ability/ shows another possibility for modelling the prior on the parameters of the Beta population distribution, namely $\frac{\alpha}{\alpha + \beta}$ ~ Uniform(0,1) = Beta(1,1) $(a+b) \sim $ Pareto(1.5, 1) Here is another resource ...


3

I encountered an under dispersed Poisson once that had to do with frequency at which people would play a social game. It turned out this was due to the extreme regularity with which people would play on Fridays. Removing Friday data gave me the expected overdispersed Poisson. Perhaps you have the option to similarly edit your data.


3

In the cited example the parameters are alpha = 81 and beta = 219 from the prior year [81 hits in 300 at bats or (81 and 300 - 81 = 219)] I don't know what they call the prior assumption of 81 hits and 219 outs but in English, that's the a priori assumption. Notice how as the season progresses the curve shifts left or right and the modal probability shifts ...


3

Disclosure: I know almost nothing about bandits. Still, my suggestion seems like a natuaral generalization of the case you presented. It does not consider the experimental design step in detail (since I don't know what people ususally consider as a loss function in this scenario), so might fail in this respect. Let me ignore the fact that we have two web ...


3

This problem is tackled by the Dearden paper on Bayesian Q-Learning. He considers a normal model for "returns" (future rewards) like @yair's solution. He also considers another term called the "value of information": "the expected improvement in future decision quality that might arise from the information acquired by exploration." Also, FYI Beta(0, 0) is ...


3

I don't think there is a closed-form solution to this question. (I'd be interested in being proven wrong.) I'd assume you will need to simulate. And hope that your predictive posterior is not misspecified too badly. In case it is interesting, we wrote a little paper (see also this presentation) once that explained how minimizing percentage errors can lead ...


Only top voted, non community-wiki answers of a minimum length are eligible