The model for the data is $X_{i}$ ~ $Bin(n_{i},\theta_{i})$ (iid, $i=1,...,k$). The prior distribution is $\theta_{i}$ ~ $Beta(\alpha,\beta)$.

How do we choose (and deduct) moment estimators for $\alpha$ and $\beta$ in this case ?


First, note that the combination of a Binomial distribution for $X_i | n_i, \theta_i$ and a Beta distribution for $\theta_i | \alpha, \beta$ leads directly to a Beta-Binomial distribution for $X_i | n_i, \alpha, \beta$.

The first two moments of the Beta-Binomial distribution are:

$$\mathbb{E}X_i = n_i {\alpha \over \alpha+\beta} $$ $$\sigma^2(X_i) = n_i{(\alpha+\beta+n_i)\alpha \beta \over (\alpha+\beta)^2(\alpha+\beta+1)}$$

The variance can be rewritten as follows: $$\sigma^2(X_i) = {n_i(\alpha+\beta)\alpha \beta \over (\alpha+\beta)^2(\alpha+\beta+1)} + {n_i^2 \alpha \beta \over (\alpha+\beta)^2(\alpha+\beta+1)}$$ $$ = {n_i\alpha \beta \over (\alpha+\beta)(\alpha+\beta+1)} + {n_i^2 \alpha \beta \over (\alpha+\beta)^2(\alpha+\beta+1)}$$

Let's define $X = \sum_{i=1}^kX_i$ and $n = \sum_{i=1}^k n_i$. Now, since the $X_i$ are independent, we know that the first two moments of the sum of the $X_i$ are just the sum of the first two moments of the individual $X_i$:

$$\mathbb{E}X = n {\alpha \over \alpha+\beta} $$ $$\sigma^2(X) = {n\alpha \beta \over (\alpha+\beta)(\alpha+\beta+1)} + {\alpha \beta \sum n_i^2 \over (\alpha+\beta)^2(\alpha+\beta+1)}$$

Equating sample moments to the two moments above results in one equation that solves for an estimate $\hat{\theta}$ of the ratio $\theta=\alpha / (\alpha+\beta)$ and another, messier, equation that can be partially written in terms of $\hat{\theta}$, $n$, and $\sum n_i^2$.

$$\hat{\theta} = {\sum X_i \over n}$$ $$s^2(X_i) = {n\hat{\theta}\hat{\beta} +\hat{\theta}(1-\hat{\theta})\sum n_i^2 \over \hat{\alpha}+\hat{\beta}+1}$$

Some more algebra, based upon the relationship $\hat{\beta} = (1-\hat{\theta})\hat{\alpha}/\hat{\theta}$ and substitution, gets us to:

$$s^2(X_i) = {n(1-\hat{\theta})\hat{\alpha} +\hat{\theta}(1-\hat{\theta})\sum n_i^2 \over \hat{\alpha}+(1-\hat{\theta})\hat{\alpha}/\hat{\theta}+1}={n(1-\hat{\theta})\hat{\alpha} +\hat{\theta}(1-\hat{\theta})\sum n_i^2 \over \hat{\alpha}/\hat{\theta}+1}$$ $$=\hat{\theta}(1-\hat{\theta}){n\hat{\alpha} +\hat{\theta}\sum n_i^2 \over \hat{\alpha} + \hat{\theta}}$$ $$=n\hat{\theta}(1-\hat{\theta}){\hat{\alpha} +\hat{\theta}\sum n_i^2/n \over \hat{\alpha} + \hat{\theta}}$$

And a few more steps leads to: $${s^2(X_i) \over n\hat{\theta}(1-\hat{\theta})}\hat{\alpha} + \hat{\theta}=\hat{\alpha} +\hat{\theta}\sum n_i^2/n$$

$$\hat{\alpha} = \hat{\theta}\left(\sum n_i^2/n-1\right)\left({n\hat{\theta}(1-\hat{\theta})\over s^2(X_i)-n\hat{\theta}(1-\hat{\theta})}\right)$$

with some possibility I've messed up the algebra, despite a couple of checks.

Once you've gotten this far, finding $\hat{\beta}$ is straightforward. Note that the MOM estimates won't exist if the sample variance is too small relative to the sample mean! This is because there is a limit on how small the population variance can be; consider all the $\theta_i = \theta$, then the variance of the $X_i$ is just $n_i\theta(1-\theta)$, but the sample variance might be smaller by chance, which would contradict the math that shows that the population variance is larger for any values of $\alpha, \beta$.

  • $\begingroup$ jbowman, am I mistaken, or is $s^2$ defined in terms of $\hat\alpha$, and vice versa? $\endgroup$ – Christoph Hanck Jan 19 '18 at 14:51
  • 1
    $\begingroup$ @ChristophHanck - should have clarified; $s^2(X_i)$ is the sample variance. Since $\hat{\theta}$ is just the sample mean, everything on the r.h.s. of the last equation is from the sample. $\endgroup$ – jbowman Jan 19 '18 at 15:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.