0
$\begingroup$

Consider an integer variable $k$ that follows a binomial distribution,

$$\binom{N}{k}p^{k}\left(1-p\right)^{N-k}$$

with total draws $N$ and probability of success $p$. I am interested in the fraction of successes, $f = k/N$. The distribution of $f$ can be approximated as a normal distribution with mean $p$ and variance $\sqrt Np(1-p)$. This approximation is good if both $Np$ and $N(1-p)$ are sufficiently large, but it has the problem that there is a finite (though small) probability that $f<0$ or $f>1$, because the normal distribution has infinite support.

Is there an approximation to the distribution of $f$ that has support $f\in[0,1]$? We can assume that $Np$ and $N(1-p)$ are both sufficiently large.

$\endgroup$
  • 1
    $\begingroup$ You can program small Monte Carlo simulation to see how well beta distribution approximates $Bin(N,p) / N$ in this case. In the beta distribution you could choose parameters $\alpha$ and $\beta$ by matching the mean and variance... To the best of my knowledge, there are no formula-based results on this. $\endgroup$ – stans - Reinstate Monica Jul 3 '18 at 10:54
  • 1
    $\begingroup$ Exactly how do you conceive of the Normal approximation for a binomial random variable $X$? In most cases it is used to approximate $\Pr(X=k)$ by the value $F(k+1/2) - F(k-1/2)$ when $k\in\{1,2,\ldots, N-1\},$ by $F(1/2)$ for $k=0,$ and $1-F(N-1/2)$ for $k=N$ (where $F$ is the appropriate Normal CDF). Those formulas never assign probabilities to negative fractions. $\endgroup$ – whuber Jul 3 '18 at 14:02
  • 2
    $\begingroup$ I believe you intend the variance to be $p(1-p)/N.$ $\endgroup$ – BruceET Jul 18 '18 at 1:44
  • $\begingroup$ Why not a Poisson? $\endgroup$ – David May 23 '19 at 15:13
6
$\begingroup$

An obvious candidate would be the beta distribution, since this is the conjugate to the binomial distribution and it is on the appropriate support. To allow for continuity correction and avoid poor approximation at the edges, it is desirable to approximate each discrete $x = 0, 1, ...., N$ by an equal-sized continuous interval. This means that we approximate the binomial mass function by an integral of the beta density over one of $N+1$ equal-sized intervals on its support:

$$\text{Bin}(x|N,p) \approx \int \limits_{B(x)}^{B(x+1)} \text{Beta}(\theta|\alpha, \beta)d\theta \quad \quad \quad B(x) \equiv \frac{x}{N+1}.$$

The approriate parameters $\alpha$ and $\beta$ can be found using the method-of-moments (MOM), which requires us to solve the following two moment equations:

$$\frac{\alpha}{\alpha+\beta} = p \quad \quad \quad \frac{\alpha \beta}{(\alpha+\beta)(\alpha+\beta+1)} = \frac{p(1-p)}{N+1}.$$

Solving for the required parameters yields the values $\alpha = pN$ and $\beta = (1-p)N$ so our approximation to the binomial is:

$$\text{Bin}(x|N,p) \approx \frac{\Gamma(N)}{\Gamma(pN)\Gamma((1-p)N)} \int \limits_{B(x)}^{B(x+1)} \theta^{pN-1} \theta^{(1-p)N-1} d\theta.$$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I was really excited to see that you'd done the math (and gotten such an intuitive answer). I was optimistic that this would be better than the normal approximation. Sadly, it seems that for sufficiently small value of $p$, the truncated normal approximation outperforms the beta approximation. $\endgroup$ – jjet Dec 3 '19 at 20:21
  • $\begingroup$ Bummer. Oh well, the normal approximation is a high bar to beat when $n$ is large. $\endgroup$ – Ben Dec 3 '19 at 21:26
1
$\begingroup$

Suppose you see $X$ Successes in $n$ trials. Then the estimate of the binomial success probability is $\hat p = X/n.$ You cannot know the exact distribution of $X$ or of $\hat p$ because you don't know $p.$ As I showed in my answer below, there are various methods to get a 'confidence interval' for $p,$ using the fact that $X \sim \mathsf{Norm}(\mu = np, \sigma = \sqrt{np(1-p)}).$ Two Answers have also shown that $X$ is approximately distributed according to a beta distribution. But you say:

No. I am interested in the distribution of the number of successes. In the limit of large number of trials, the fraction of successes should follow an approximately continuous distribution.

This response is puzzling. And you have not responded to @whuber's Comments. It seems this matter will remain unresolved until you explain what you are doing and exactly why answers and comments to date are not what you are looking for.


It seems as if you might be trying to get a confidence interval (CI) for $p$ based on seeing $X$ Successes in $n$ trials.

Wald interval: Then a point estimate for $p$ is $\hat p = X/n.$ A traditional 'Wald' 95% confidence interval is of the form $$\hat p \pm 1.96 \sqrt{\frac{\hat p(1-\hat p)}{n}}.$$ It uses two approximations: (a) that $\hat p$ is approximately normally distributed and (b) that the standard error of $\hat p.$

$$SD(\hat p) = \sqrt{\frac{p(1-p)}{n}} \approx \sqrt{\frac{\hat p(1-\hat p)}{n}}.$$

Because these two approximations are best for large $n$ and $p$ relatively near $1/2,$ this style of CI does not provide the nominal 95% coverage probability in many practical situations. You mentioned one difficulty: the CI has length $0$ if $X = 0$ or $X = n.$

Example: If $X = 30$ and $n = 50,$ a 95% Wald CI is $(0.464, 0.736).$

n = 50;  p.hat = 30/50; pm = c(-1,1); wald.ci = p.hat + pm*1.96*sqrt(p.hat*(1-p.hat)/n)
wald.ci
[1] 0.4642072 0.7357928

Agresti-Coull CI: The improved Agresti-Coull style of CI is of the form $$\tilde p \pm 1.96 \sqrt{\frac{\tilde p(1-\tilde p)}{\tilde n}},$$ where $\tilde n =n+4$ and $\tilde p = (X+2)/\tilde n).$

Example: For the data above, this CI is $(0.456, 0.729).$

n.tilde=54; p.hat=32/54; pm=c(-1,1); agresti.ci=p.hat+pm*1.96*sqrt(p.hat*(1-p.hat)/n)
agresti.ci
[1] 0.4563968 0.7287884

The procedure @Ben (+1) describes can be adapted to give a 95% Bayesian probability interval ('credible interval'), using $\mathsf{Beta}(1,1) \equiv \mathsf{Unif}(0,1)$ as a non-informative prior distribution and a binomial likelihood function proportional to $p^X(1-p)^{n-X}.$

Then the posterior distribution is $\mathsf{Beta}(X+1, n-X+1)$ and the interval estimate has quantiles .025 and .975 of that distribution as its endpoints: $(0.461, 0.724)$ in our example.

qbeta(c(.025, .975), 31, 21)
[1] 0.461141 0.724157

Note: If you are interested in interval estimation for binomial data, you may want to look at a similar Q & A and the link at the end.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ No. I am interested in the distribution of the number of successes. In the limit of large number of trials, the fraction of successes should follow an approximately continuous distribution. $\endgroup$ – becko Jul 18 '18 at 8:34
  • 1
    $\begingroup$ Yes, but as I pointed out in a comment to your question, there is no need for that continuous distribution to have positive support. Imposing an unnecessary constraint can only degrade the approximation, never improve it. Your reason for seeking to worsen your statistical analysis remains obscure. $\endgroup$ – whuber Jul 18 '18 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.