I have a Beta distribution $D$ with unknown parameters $\alpha$ and $\beta$ which I wish to estimate.

If I was given samples $p_1, \ldots, p_n$ from $D$, then it's relatively straightforward to fit $D$ (e.g. method of moments, or numerically maximizing the likelihood).

However, I'm only allowed to run the following experiment: sample a biased coin from $D$ (we're not told the bias), and flip it as many times as I want. Given I'm only allowed to run such an experiment (i.e. only allowed to observe the flips of coins generated by $D$), I want to estimate the $\alpha$ and $\beta$ parameters for $D$.

This leads to two questions:

  1. Given a list of flips from the different coins generated from $D$, how do I estimate $\alpha$ and $\beta$
  2. What is an optimal (or at least a decent) way I should run the experiments (do the flips) so as to ID $D$ as accurately as possible with as few coin flips as possible? And are there any bounds on what level of accuracy I can achieve if I allow myself to make $N$ coin flips?

One observation to illustrate the 2nd point: On one extreme, if, for every coin I generate from $D$, I choose to flip it 1 billion times, then the problem essentially boils down to fitting $D$ given exact values for the biases $p_1,\ldots, p_n$, which we know how to do. On the other extreme, if I only flip each coin generated from $D$ once, then I'll have no hope of fitting a general $D$ no matter how many times I do the experiment. The only thing I'll get from that experiment is the average of $D$, with no information on the shape. So, on 1 extreme, the problem is solvable, but I'm not being efficient with my coin tosses. On the other extreme, the problem just can't be solved. Is there some middle ground I should be gunning for?

As for the first question (how to actually fit the params given the coin flips), my guess at the solution is to maximize the log likelihood of the observed coin flips.

  • As a function of our estimates for $\alpha$ and $\beta$, the likelihood $L_i(\alpha, \beta)$ of flipping the $i$th biased coin we generated and getting $H_i$ heads and $T_i$ tails will be $$L_i(\alpha, \beta) = {H_i + T_i \choose H_i} \frac{B(H_i + \alpha, T_i + \beta)}{B(\alpha, \beta)}$$ according to the Beta-Binomial Distribution.
  • So the log liklihood of observing all our experiments will be $$LL(\alpha, \beta) = \sum_i \log L_i(\alpha, \beta)$$
  • We can maximize the above by gradient descent (or any other numerical method).
  • 1
    $\begingroup$ Re 1: This is a classic generalized linear model (GLM) with no predictors. Typically they are fit with MLE as you indicate. Question (2) is interesting. $\endgroup$
    – whuber
    Aug 30, 2019 at 12:15


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.