4
$\begingroup$

Consider an integer variable $k$ that follows a binomial distribution,

$$\binom{N}{k}p^{k}\left(1-p\right)^{N-k}$$

with total draws $N$ and probability of success $p$. I am interested in the fraction of successes, $f = k/N$. The distribution of $f$ can be approximated as a normal distribution with mean $p$ and variance $\sqrt Np(1-p)$. This approximation is good if both $Np$ and $N(1-p)$ are sufficiently large, but it has the problem that there is a finite (though small) probability that $f<0$ or $f>1$, because the normal distribution has infinite support.

Is there an approximation to the distribution of $f$ that has support $f\in[0,1]$? We can assume that $Np$ and $N(1-p)$ are both sufficiently large.

$\endgroup$
4
  • 1
    $\begingroup$ You can program small Monte Carlo simulation to see how well beta distribution approximates $Bin(N,p) / N$ in this case. In the beta distribution you could choose parameters $\alpha$ and $\beta$ by matching the mean and variance... To the best of my knowledge, there are no formula-based results on this. $\endgroup$
    – stans
    Jul 3, 2018 at 10:54
  • 3
    $\begingroup$ Exactly how do you conceive of the Normal approximation for a binomial random variable $X$? In most cases it is used to approximate $\Pr(X=k)$ by the value $F(k+1/2) - F(k-1/2)$ when $k\in\{1,2,\ldots, N-1\},$ by $F(1/2)$ for $k=0,$ and $1-F(N-1/2)$ for $k=N$ (where $F$ is the appropriate Normal CDF). Those formulas never assign probabilities to negative fractions. $\endgroup$
    – whuber
    Jul 3, 2018 at 14:02
  • 2
    $\begingroup$ I believe you intend the variance to be $p(1-p)/N.$ $\endgroup$
    – BruceET
    Jul 18, 2018 at 1:44
  • $\begingroup$ Why not a Poisson? $\endgroup$
    – David
    May 23, 2019 at 15:13

4 Answers 4

11
$\begingroup$

An obvious candidate would be the beta distribution, since this is the conjugate to the binomial distribution and it is on the appropriate support. To allow for continuity correction and avoid poor approximation at the edges, it is desirable to approximate each discrete $x = 0, 1, ...., N$ by an equal-sized continuous interval. This means that we approximate the binomial mass function by an integral of the beta density over one of $N+1$ equal-sized intervals on its support:

$$\text{Bin}(x|N,p) \approx \int \limits_{B(x)}^{B(x+1)} \text{Beta}(\theta|\alpha, \beta)d\theta \quad \quad \quad B(x) \equiv \frac{x}{N+1}.$$

The approriate parameters $\alpha$ and $\beta$ can be found using the method-of-moments (MOM), which requires us to solve the following two moment equations:

$$\frac{\alpha}{\alpha+\beta} = p \quad \quad \quad \frac{\alpha \beta}{(\alpha+\beta)(\alpha+\beta+1)} = \frac{p(1-p)}{N+1}.$$

Solving for the required parameters yields the values $\alpha = pN$ and $\beta = (1-p)N$ so our approximation to the binomial is:

$$\text{Bin}(x|N,p) \approx \frac{\Gamma(N)}{\Gamma(pN)\Gamma((1-p)N)} \int \limits_{B(x)}^{B(x+1)} \theta^{pN-1} \theta^{(1-p)N-1} d\theta.$$

$\endgroup$
2
  • 1
    $\begingroup$ I was really excited to see that you'd done the math (and gotten such an intuitive answer). I was optimistic that this would be better than the normal approximation. Sadly, it seems that for sufficiently small value of $p$, the truncated normal approximation outperforms the beta approximation. $\endgroup$
    – jjet
    Dec 3, 2019 at 20:21
  • $\begingroup$ Bummer. Oh well, the normal approximation is a high bar to beat when $n$ is large. $\endgroup$
    – Ben
    Dec 3, 2019 at 21:26
2
$\begingroup$

Suppose you see $X$ Successes in $n$ trials. Then the estimate of the binomial success probability is $\hat p = X/n.$ You cannot know the exact distribution of $X$ or of $\hat p$ because you don't know $p.$ As I showed in my answer below, there are various methods to get a 'confidence interval' for $p,$ using the fact that $X \sim \mathsf{Norm}(\mu = np, \sigma = \sqrt{np(1-p)}).$ Two Answers have also shown that $X$ is approximately distributed according to a beta distribution. But you say:

No. I am interested in the distribution of the number of successes. In the limit of large number of trials, the fraction of successes should follow an approximately continuous distribution.

This response is puzzling. And you have not responded to @whuber's Comments. It seems this matter will remain unresolved until you explain what you are doing and exactly why answers and comments to date are not what you are looking for.


It seems as if you might be trying to get a confidence interval (CI) for $p$ based on seeing $X$ Successes in $n$ trials.

Wald interval: Then a point estimate for $p$ is $\hat p = X/n.$ A traditional 'Wald' 95% confidence interval is of the form $$\hat p \pm 1.96 \sqrt{\frac{\hat p(1-\hat p)}{n}}.$$ It uses two approximations: (a) that $\hat p$ is approximately normally distributed and (b) that the standard error of $\hat p.$

$$SD(\hat p) = \sqrt{\frac{p(1-p)}{n}} \approx \sqrt{\frac{\hat p(1-\hat p)}{n}}.$$

Because these two approximations are best for large $n$ and $p$ relatively near $1/2,$ this style of CI does not provide the nominal 95% coverage probability in many practical situations. You mentioned one difficulty: the CI has length $0$ if $X = 0$ or $X = n.$

Example: If $X = 30$ and $n = 50,$ a 95% Wald CI is $(0.464, 0.736).$

n = 50;  p.hat = 30/50; pm = c(-1,1); wald.ci = p.hat + pm*1.96*sqrt(p.hat*(1-p.hat)/n)
wald.ci
[1] 0.4642072 0.7357928

Agresti-Coull CI: The improved Agresti-Coull style of CI is of the form $$\tilde p \pm 1.96 \sqrt{\frac{\tilde p(1-\tilde p)}{\tilde n}},$$ where $\tilde n =n+4$ and $\tilde p = (X+2)/\tilde n).$

Example: For the data above, this CI is $(0.456, 0.729).$

n.tilde=54; p.hat=32/54; pm=c(-1,1); agresti.ci=p.hat+pm*1.96*sqrt(p.hat*(1-p.hat)/n)
agresti.ci
[1] 0.4563968 0.7287884

The procedure @Ben (+1) describes can be adapted to give a 95% Bayesian probability interval ('credible interval'), using $\mathsf{Beta}(1,1) \equiv \mathsf{Unif}(0,1)$ as a non-informative prior distribution and a binomial likelihood function proportional to $p^X(1-p)^{n-X}.$

Then the posterior distribution is $\mathsf{Beta}(X+1, n-X+1)$ and the interval estimate has quantiles .025 and .975 of that distribution as its endpoints: $(0.461, 0.724)$ in our example.

qbeta(c(.025, .975), 31, 21)
[1] 0.461141 0.724157

Note: If you are interested in interval estimation for binomial data, you may want to look at a similar Q & A and the link at the end.

$\endgroup$
2
  • $\begingroup$ No. I am interested in the distribution of the number of successes. In the limit of large number of trials, the fraction of successes should follow an approximately continuous distribution. $\endgroup$
    – a06e
    Jul 18, 2018 at 8:34
  • 1
    $\begingroup$ Yes, but as I pointed out in a comment to your question, there is no need for that continuous distribution to have positive support. Imposing an unnecessary constraint can only degrade the approximation, never improve it. Your reason for seeking to worsen your statistical analysis remains obscure. $\endgroup$
    – whuber
    Jul 18, 2018 at 14:29
1
$\begingroup$

Express factorial functions on the Binomial coefficient as Gamma functions: $\Gamma(x + 1)$.

$${K\choose k} \equiv \frac{K!}{k!(K-k)!} = \frac{\Gamma(K + 1)}{\Gamma(k + 1)\Gamma(K - k + 1)},\ \forall\ \ \mathbb{N} \in \{0, 1, ..., K\},$$ where $K$ is the total number of trials, and $k$ the number of successful trials.

The Binomial distribution can as such be expressed as:

$$\frac{\Gamma(K + 1)}{\Gamma(k + 1)\Gamma(K - k + 1)}p^k(1-p)^{K-k},$$ where $p$ is the probability of success per trial.

This extends the Binomial distribution to the real-number plane. However, the extension allows for non-zero frequencies of occurences less than $0$ and greater than $K$. Hence, it needs to be truncated to the interval $[0, K]$. This range-restriction makes necessary a normalizing constant ensuring that the density on this interval equal $1$. The constant can be obtained by:

$$C = \int_0^K \frac{\Gamma(K + 1)}{\Gamma(k + 1)\Gamma(K - k + 1)}p^k(1-p)^{K-k}\ dk$$

A continuous analogue to the Binomial distribution can as such be expressed as:

$$ P(X; K, k, p) = \begin{cases} \frac{1}{C}\left[\frac{\Gamma(K+1)}{\Gamma(k + 1)\Gamma(K - k + 1)}p^k(1-p)^{K-k}\right],& \text{if}\ 0 \leq k \leq K,\\ 0,& \text{otherwise.} \end{cases} $$

$\endgroup$
1
  • 1
    $\begingroup$ The normalizing constant can be obtained explicitly: $\int_0^K p^k (1 - p)^k \mathrm{d}k = \frac{(1 - p)^K - p^K}{2 \mathrm{atanh} (1 - 2 p)}$ $\endgroup$
    – a06e
    Apr 18, 2021 at 7:49
0
$\begingroup$

The cbinom R package implements a "continuous Binomial" distribution, based on a paper by A. Ilienko. The CDF of this "continuous Binomial" is essentially the ratio of an incomplete and a complete Beta function. For the gory details please refer to the paper linked above, it is freely downloadable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.