Why we don’t make use of the t-distribution for constructing a confidence interval for a proportion?

Question

To calculate the confidence-interval (CI) for mean with unknown population standard deviation (sd) we estimate the population standard deviation by employing the t-distribution. Notably, $CI=\bar{X} \pm Z_{95\% }\sigma_{\bar X}$ where $\sigma_{\bar X} = \frac{\sigma}{\sqrt n}$. But because, we do not have point estimate of the standard deviation of the population, we estimate through the approximation $CI=\bar{X} \pm t_{95\% }(se)$ where $se = \frac{s}{\sqrt n}$

Contrastingly, for population proportion, to calculate the CI, we approximate as $CI = \hat{p} \pm Z_{95\% }(se)$ where $se = \sqrt\frac{\hat{p}(1-\hat{p})}{n}$ provided $n \hat{p} \ge 15$ and $n(1-\hat{p}) \ge 15$

My question is, why are we complacent with standard distribution for population proportion?

Answer 1

Both the standard Normal and Student t distributions are rather poor approximations to the distribution of

$$Z = \frac{\hat p - p}{\sqrt{\hat p(1-\hat p)/n}}$$

for small $n,$ so poor that the error dwarfs the differences between these two distributions.

Here is a comparison of all three distributions (omitting the cases where $\hat p$ or $1-\hat p$ are zero, where the ratio is undefined) for $n=10, p=1/2:$

The "empirical" distribution is that of $Z,$ which must be discrete because the estimates $\hat p$ are limited to the finite set $\{0, 1/n, 2/n, \ldots, n/n\}.$

The $t$ distribution appears to do a better job of approximation.

For $n=30$ and $p=1/2,$ you can see the difference between the standard Normal and Student t distributions is completely negligible:

Because the Student t distribution is more complicated than the standard Normal (it's really an entire family of distributions indexed by the "degrees of freedom," formerly requiring entire chapters of tables rather than a single page), the standard Normal is used for almost all approximations.

Answer 2

The justification for using the t distribution in the confidence interval for a mean relies on the assumption that the underlying data follows a normal distribution, which leads to a chi-squared distribution when estimating the standard deviation, and thus $\frac{\bar{x}-\mu}{s/ \sqrt{n}} \sim t_{n-1}$. This is an exact result under the assumption that the data are exactly normal that leads to confidence intervals with exactly 95% coverage when using $t$, and less than 95% coverage if using $z$.

In the case of Wald intervals for proportions, you only get asymptotic normality for $\frac{\hat{p}- p}{\sqrt{ \hat{p}(1-\hat{p} )/n}}$ when n is large enough, which depends on p. The actual coverage probability of the procedure, since the underlying counts of successes are discrete, is sometimes below and sometimes above the nominal coverage probability of 95% depending on the unknown $p$. So, there is no theoretical justification for using $t$, and there is no guarantee that from a practical perspective that using $t$ just to make the intervals wider would actually help achieve nominal coverage of 95%.

The coverage probability can be calculated exactly, though it's fairly straightforward to simulate it. The following example shows the simulated coverage probability when n=35. It demonstrates that the coverage probability for using the z-interval is generally slightly smaller than .95, while the coverage probability for the t-interval may generally be slighter closer to .95 on average depending on your prior beliefs on the plausible values of p.

Answer 3

Both AdamO and jsk give a great answer.

I would try to repeat their points with plain English:

When the underlying distribution is normal, you know there are two parameters: mean and variance. T distribution offers a way to do inference on the mean without knowing the exact value of the variances. Instead of using actual variances, only sample means and sample variances are needed. Because it is an exact distribution, you know exactly what you are getting. In other words, the coverage probability is correct. The usage of t simply reflects the desire to get around the unknown populuation variance.

When we do inference on proportion, however, the underlying distribution is binomial. To get the exact distribution, you need to look at Clopper-Pearson confidence intervals. The formula you provide is the formula for Wald confidence interval. It use the normal distribution to approximate the binomial distribution, because normal distribution is the limiting distribution of the binomial distribution. In this case, because you are only approximating, the extra level of precision from using t statistics becomes unnecessary, it all comes down to empirical performance. As suggested in BruceET's answer, the Agresti-Coull is simple and standard formula nowadays for such approximation.

My professor Dr Longnecker of Texas A&M has done a simple simulation to illustrate how the different approximation works compared to the binomial based CI.

Further information can be found in the article Interval Estimation for a Binomial Proportion in Statistical Science, Vol. 16, pp.101-133, by L. Brown, T. Cai and A. DasGupta. Basically, A-C CI is recommended for n >= 40.

Answer 4

Confidence interval for normal mean. Suppose we have a random sample $X_1, X_2, \dots X_n$ from a normal population. Let's look at the confidence interval for normal mean $\mu$ in terms of hypothesis testing. If $\sigma$ is known, then a two-sided test of $H_0:\mu = \mu_0$ against $H_a: \mu \ne \mu_0$ is based on the statistic $Z = \frac{\bar X - \mu_0}{\sigma/\sqrt{n}}.$ When $H_0$ is true, $Z \sim \mathsf{Norm}(0,1),$ so we reject $H_0$ at the 5% level if $|Z| \ge 1.96.$

Then 'inverting the test', we say that a 95% CI for $\mu$ consists of the values $\mu_0$ that do not lead to rejection--the 'believable' values of $\mu.$ The CI is of the form $\bar X \pm 1.96\sigma/\sqrt{n},$ where $\pm 1.96$ cut probability 0.025 from the upper and lower tails, respectively, of the standard normal distribution.

If the population standard deviation $\sigma$ is unknown and estimated by by the sample standard deviation $S,$ then we use the statistic $T=\frac{\bar X - \mu_0}{S/\sqrt{n}}.$ Before the early 1900's people supposed that $T$ is approximately standard normal for $n$ large enough and used $S$ as a substitute for unknown $\sigma.$ There was debate about how large counts as large enough.

Eventually, it was known that $T \sim \mathsf{T}(\nu = n-1),$ Student's t distribution with $n-1$ degrees of freedom. Accordingly, when $\sigma$ is not known, we use $\bar X \pm t^*S/\sqrt{n},$ where $\pm t^*$ cut probability 0.025 from the upper and lower tails, respectively, of $\mathsf{T}(n-1).$

[Note: For $n > 30,$ people have noticed that for 95% CIs $t^* \approx 2 \approx 1.96.$ Thus the century-old idea that you can "get by" just substituting $S$ for $\sigma$ when $\sigma$ is unknown and $n > 30,$ has persisted even in some recently-published books.]

Confidence interval for binomial proportion. In the binomial case, suppose we have observed $X$ successes in a binomial experiment with $n$ independent trials. Then we use $\hat p =X/n$ as an estimate of the binomial success probability $p.$ In order to test $H_0:p = p_0$ vs $H_a: p \ne p>0,$ we use the statitic $Z = \frac{\hat p - p_0}{\sqrt{p_0(1-p_0)/n}}.$ Under $H_0,$ we know that $Z \stackrel{aprx}{\sim} \mathsf{Norm}(0,1).$ So we reject $H_0$ if $|Z| \ge 1.96.$

If we seek to invert this test to get a 95% CI for $p,$ we run into some difficulties. The 'easy' way to invert the test is to start by writing $\hat p \pm 1.96\sqrt{\frac{p(1-p)}{n}}.$ But his is useless because the value of $p$ under the square root is unknown. The traditional Wald CI assumes that, for sufficiently large $n,$ it is OK to substitute $\hat p$ for unknown $p.$ Thus the Wald CI is of the form $\hat p \pm 1.96\sqrt{\frac{\hat p(1-\hat p)}{n}}.$ [Unfortunately, the Wald interval works well only if the number of trials $n$ is at least several hundred.]

More carefully, one can solve a somewhat messy quadratic inequality to 'invert the test'. The result is the Wilson interval. (See Wikipedia.) For a 95% confidence interval a somewhat simplified version of this result comes from defining $\check n = n+4$ and $\check p = (X+2)/\check n$ and then computing the interval as $\check p \pm 1.96\sqrt{\frac{\check p(1-\check p)}{\check n}}.$ This style of binomial confidence interval is widely known as the Agresti-Coull interval; it has been widely advocated in elementary textbooks for about the last 20 years.

In summary, one way to look at your question is that CIs for normal $\mu$ and binomial $p$ can be viewed as inversions of tests.

(a) The t distribution provides an exact solution to the problem of needing to use $S$ for $\sigma$ when $\sigma$ is unknown.

(b) Using $\hat p$ for $p$ requires some care because the mean and variance of $\hat p$ both depend on $p.$ The Agresti-Coull CI provides one serviceable way to get CIs for binomial $p$ that are reasonably accurate even for moderately small $n.$

Answer 5

Note your use of the $\sigma$ notation which means the (known) population standard deviation.

The T-distribution arose as an answer to the question: what happens when you don't know $\sigma$?

He noted that, when you cheat by estimating $\sigma$ from the sample as a plug-in estimator, your CIs are on average too narrow. This necessitated the T-distribution.

Conversely, if you use the T distribution when you actually do know $\sigma$, your confidence intervals will on average be too wide.

Also, it should be noted that this question mirrors the answer solicited by this question.