Is there any assumption between confidence intervals and data normalty?

Question

I read this discussion on whether confidence intervals are useful and I am trying to figure out if there is any underlying assumption about the data that I used to construct a confidence interval.

1. Does my data need to come from a normal distribution?
2. What does happen if the data used to construct confidence intervals is not normally distributed?
3. Bonus question: why should I use confidence intervals and not just report mean/median and standard deviation?

Answer 1

1. No, although a lot of confidence intervals you may encounter are motivated by either transforming some statistic onto a scale in which the sampling distribution is something close to normal or by using asymptotic arguments (e.g. with enough data, the distribution is close enough to normal).

   The sample mean for an exponential random variable is a good example of this. If $X_1, \dots, X_n$ are iid samples from an exponential distribution with rate parameter $\lambda$, their sample mean distributed according to a gamma distribution with shape and rate $n$ and $n\lambda$ respectively. The data are not normal, and yet we can obtain a confidence interval as explained here

2. The consequence of non-normality is that it may be harder to compute a confidence interval pen and paper. You might have to resort to using computational methods (like the bootstrap or profile likelihood methods).

3. Confidence intervals communicate uncertainty in the estimate, whereas the standard deviation as a measure of uncertainty on the level of the observation. Let's say I report a mean of 10 and a standard deviation of 2. I am more certain that the estimate of 10 is closer to the truth if I used 10,000 data points as opposed to 100 data points (assuming I used a random sample and there is no bias in my data collection). Confidence intervals account for this. As you get more data, confidence intervals will get narrower, meaning there less uncertainty in our estimate.

Answer 2

There are various styles of confidence intervals for different kinds of populations (normal or not) and different parameters.

Examples:

Parametric confidence intervals assume that you know the distribution type of the population.

If you have data from a normal population in which the mean $\mu$ and standard deviation are unknown a 95% confidence interval for $\mu$ is $\bar X \pm t^* \frac{S}{\sqrt{n}},$ where $bar X$ estimates $\mu,$ $S$ estimates $\sigma$ and $t^*$ cuts probability $0.025$ from the upper tail of the (symmetrical) Student t distribution with $\nu = n-1$ degrees of freedom.

set.seed(2021)
x = rnorm(100, 50, 15)
mean(x);  sd(x)
[1] 47.38273
[1] 15.42939

CI = mean(x) + qt(c(.025,.975), 99)*sd(x)/sqrt(100);  CI
[1] 44.32121 50.44426  # CI (55.32, 50,44) includes 50--just barely

In the same circumstances, a 95% CI for $\sigma$ has endpoints $\sqrt{\frac{(n-1)S^2}{U}}$ and $\sqrt{\frac{(n-1)S^2}{L}},$ where $L$ and $U$ cut probabilities $0.025$ from the lower and upper tails, respectively, of the distribution $\mathsf{Chisq}(\nu = n-1).$

CI = sqrt(99*var(x)/qchisq(c(.975,.025), 99));  CI
[1] 13.54711 17.92394 # CI (13.44, 17.92) includes 15

If you have data from an exponential population with rate $\lambda$ and mean $\theta = 1/\lambda,$ then a 95% CI for $\theta$ has endpoints $\frac{\bar X}{U}$ and $\frac{\bar X}{L},$ where $\bar X = \frac 1n\sum_{i=1}^n X_i$ estimates $\theta$ and where $L$ and $U$ cut probabilities $0.025$ from the lower and upper tails, respectively, of the distribution $\mathsf{Gamma}(\mathrm{shape}=n,\mathrm{rate}=n).$

set.seed(525)
X = rexp(100, 1/50)  # 'X' here is distinct from 'x' above
mean(X)
[1] 49.80193


CI = mean(X)/qgamma(c(.975,.025), 100, 100);  CI
[1]  41.31948 61.20881 # CI (41.32, 61.21) includes 50

If you do not know the distribution family of the population (but you do know that the population mean $\mu$ exists), then you may be able to find a useful nonparametric confidence interval for $\mu$ by bootstrapping. One nonparametric bootstrap 95% CI can be make by taking $B = 3000$ re-samples of the data x. Re-samples are of size $n$ taken with replacement. We find the mean a.re of each resample, to obtain the bootstrap distribution; then the CI has endpoints at quantiles $0.025$ and $0.975$ of the bootstrap distribution. [There are many styles of bootstrap CIs; this is a simple one, but not always the best.]

set.seed(1234)  # bootstrap normal sample `x`
a.re = replicate(3000, mean(sample(x, 100, rep=T)))
CI = quantile(a.re, c(.025, .975));  CI
    2.5%     7.5% 
44.45223 50.41978    # CI (44.45, 50.42) includes 50

set.seed(1235) # bootstrap exponential sample 'X'
A.re = replicate(3000, mean(sample(X, 100, rep=T)))
CI = quantile(A.re, c(.025, .975));  CI
    2.5%    97.5% 
39.70519 61.21473   # CI (39.71. 61.21) includes 50 

The figure below shows the two bootstrap distributions and corresponding CIs.

Note: All of the 95% CI's shown above happen to include the target parameter value. We know this because we are using fictitious simulated data. In about 5% of cases real data will not produce a CI that covers the parameter; of course we will not know which samples produce CIs that don't happen to cover the parameter value.