5
$\begingroup$

Suppose I want to estimate the proportion parameter for a binomial distribution, and my method for constructing a 95% confidence interval (e.g. MLE +- 1.96 SDs) gives an interval of let's say (-.1, .3). Then what is the standard practice? I mean it's not wrong by my understanding of what a confidence interval is, but on the other hand would it be better to truncate the interval? So if the normal approximation gives (a,b), the CI should be (max{a,0},min{b,1})

$\endgroup$
0

4 Answers 4

2
$\begingroup$

There are several ways to construct confidence intervals for proportion parameters which have better properties than the normal approximation ("Wald-type" confidence interval) which you have mentioned.

A relatively simple choice that avoids "overshoot", is to compute a Wald-type confidence interval for the log-transformed proportion and to transform it then back to the original scale.

Let $X \sim \text{Bin}(n, \pi)$ and $x$ be a realization of $X$. The maximum likelihood estimator (MLE) of $\pi$ is $$\hat{\pi} = \frac{x}{n}$$ with standard error $$\text{se}(\hat{\pi}) = \sqrt{\frac{\hat{\pi}(1 - \hat{\pi})}{n}}.$$

By the invariance of the MLE, the MLE of $\log\pi$ is $$\widehat{\log\pi} = \log\frac{x}{n}.$$ Application of the delta method leads to $$\text{se}(\widehat{\log\pi}) = \sqrt{\frac{1 - \hat{\pi}}{\hat{\pi}n}} = \sqrt{\frac{1}{x} - \frac{1}{n}}.$$

So a 95% confidence interval for $\log\pi$ is given by $$\widehat{\log\pi} \pm z_{0.975} \cdot \text{se}(\widehat{\log\pi}),$$ with $z_{0.975}$ the 0.975 quantile of the standard normal distribution. Transforming this confidence interval with the exponential function yields an improved confidence interval for the proportion $\pi$.

x <- 3
n <- 20
p <- x/n

## standard CI
sep <- sqrt(p*(1 - p)/n)
(CI1 <- p + c(-1, 1)*qnorm(0.975)*sep)
[1] -0.006490575  0.306490575

## improved CI
logp <- log(p)
selogp <- sqrt(1/x - 1/n)
(CI2 <- exp(logp + c(-1, 1)*qnorm(0.975)*selogp))
[1] 0.05284509 0.42577277
$\endgroup$
2
$\begingroup$

You can look at the Wikipedia article on confidence intervals for binomial proportions. About half a dozen methods are in common use.

In elementary US statistics books the 95% Wald interval (based on a limiting argument) was used for some years before it was shown to have poor coverage probability for small $n$ (and also to give interval endpoints outside $(0,1)$ for very small or very large observed proportions $\hat p = x/n).$

For 1 success in 10 trials the 95% Wald interval is $\hat p \pm 1.96\sqrt{\frac{\hat p(1-\hat p)}{n}},$ which computes to $(-0.086, 0.286)$ and would usually get truncated to $(0, 0.286),$ perhaps with a footnote. [Computation in R.]

n = 10;  x = 1;  hat.p = x/n
hat.p + qnorm(c(.025,.975))*sqrt(hat.p*(1-hat.p)/n)
[1] -0.08593851  0.28593851

One kind of interval is easy compute if you have access to computation of quantiles of beta distributions. For this interval, one uses quantiles 0.025 and 0.975 of $\mathsf{Beta}(x+.5, n-x+.5).$ This is called a 95% Jeffreys CI because it is derived by a Bayesian argument from a noninformative Jeffreys prior distribution. Because beta distributions have support $(0,1),$ endpoints of these intervals are always between $0$ and $1.$ For my example above, the 95% Jeffreys confidence interval is $(0.011, 0.381).$

qbeta(c(.025,.975), 1.5, 9.5)
[1] 0.01101167 0.38131477

Notes: (1) Some of the types of CIs mentioned in the Wikipedia article are tedious to compute or not widely implemented in statistical software. One easily computed improvement on the Wald interval, mentioned there is due to Agresti and Coull, although it can also occasionally give endpoints outside of $(0,1),$ as below:

n = 10;  x = 1;  hat.p = (x+2)/(n+4)
hat.p + qnorm(c(.025,.975))*sqrt(hat.p*(1-hat.p)/(n+4))
[1] -0.0006521887  0.4292236173

(2) Reference 4 (by Brown et al.) in the Wikipedia article has an accessible discussion of coverage probabilities of various styles of binomial confidence intervals.

$\endgroup$
1
  • 1
    $\begingroup$ As your last reference documents the Jeffreys interval also has frequentist interpretations. $\endgroup$
    – Nick Cox
    Commented Jan 31, 2021 at 10:28
1
$\begingroup$

The problem is that for you are trying to apply the CLT, but you have no control over the rate of convergence. Your assumption that the CLT sufficiently applies so that the confidence interval is MLE +/- 1.96 * SD's is not true if you are getting a CI of (-0.1, 0.3). In fact, it is well known that CLT convergence in the binomial case is quite slow for p close to 0 and 1.

Truncating the CI like you want will not result in a 95% CI. Either get more data so the CI is contained in [0,1] (for p close to 0 or 1, you'll need a lot of data, rates of convergence to the CLT are quite slow) or use a Bayesian technique.

See https://math.unm.edu/~james/Agresti1998.pdf

See https://towardsdatascience.com/five-confidence-intervals-for-proportions-that-you-should-know-about-7ff5484c024f

$\endgroup$
0
$\begingroup$

Definitely, you can and should truncate the CI. It is standard practice.

If the un-truncated CI contains the true parameter 95% of the time, then the truncated CI will also contain the true parameter 95% of the time (because you are only removing values that are impossible). Better than truncating the asymptotic CI is to use one of the methods for small sample sizes. A comparison of different methods can be seen here.

$\endgroup$
3
  • 3
    $\begingroup$ "standard practice" I would call an exaggeration as it's been very widely known for much more than 20 years that truncation is a poor ad hoc method and there are better methods which are widely publicised and implemented. That truncation is still common practice wouldn't surprise me. This was the first answer, but it has been superseded by better answers. Suggesting a different method for small samples raises the question of when samples are no longer small. $\endgroup$
    – Nick Cox
    Commented Jan 31, 2021 at 10:32
  • $\begingroup$ "the truncated CI will also contain the true parameter 95% of the time (because you are only removing values that are impossible)" --intuitively this feels untrue, do you have a proof? $\endgroup$ Commented Jan 31, 2021 at 12:38
  • $\begingroup$ @SimonN Yes. Let LL be the lower limit and UL be the upper limit. A is the event $LL<\theta<UL$. Suppose the event occurs. Since it is known that $0\le \theta \le 1$ always, it follows that $B=max(LL,0) \le \theta \le min(UL,1)$ also occurs. The event B is the intersection of the event A and an event that always occurs. Conversely, if B occurs, then so does A. So, the expected value of the indicator function of the event A is equal to the expected value of the indictor function of the event B. That's equivalent to the statement that the probabilities are equal. $\endgroup$
    – John L
    Commented Jan 31, 2021 at 14:12

Not the answer you're looking for? Browse other questions tagged or ask your own question.