5
$\begingroup$

I've seen a few resources online that construct confidence intervals (CI) in different ways.

  • One is through bootstrapping which essentially samples with replacement from a group of measurements and takes the mean of each of these samples. A 95% CI would then be the range that includes 95% of these sample means.

  • Another is by taking a sample mean and adding a t-statistic multiplied by the sample standard error:

$$\bar{x} \pm t*\frac{s}{\sqrt{n}}$$

How are these related? (Are they identical?) How are these different?

$\endgroup$
1
  • 2
    $\begingroup$ In general, they are not identical. The confidence interval constructed using the standard error of the mean assumes that the sampling distribution of the mean is normal (hence the use of the standard error). However, if the population distribution is long tailed, this assumption is dubious. Hence, bootstrapping can be used (which basically approximates the sampling distribution of the statistic). $\endgroup$ Feb 15, 2022 at 20:51

1 Answer 1

2
$\begingroup$

If you know that data are sampled from a normal distribution, but with unknown mean and variance, you can use the displayed formula in the second part of your question to get an exact 95% CI. That is, over the long run such CI's will 'cover' (include) the population mean $\mu$ 95% of the time. Roughly speaking, to the extent that data are not normal such CIs (especially ones based on small samples) will not be accurate.

The R procedure t.test makes 95% CIs, using your formula. A normal sample of size $n = 20$ from a distribution with mean $\mu = 50$ and the resulting 95% CI $(45.522, 51.140)$ are shown below:

set.seed(2022)
y = rnorm(20, 50, 6)
summary(y);  sd(y)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  32.60   44.25   49.20   48.33   52.38   56.12 
[1] 6.00178   # sample SD
t.test(y)$conf.int
[1] 45.52181 51.13965
attr(,"conf.level")
[1] 0.95

If you know the distribution type of a non-normal population, you may be able to find an exact 95% CI using statistical theory. For example, if data are exponential, then you can 'pivot' the relationship $$\frac{\bar X}{\mu} \sim \mathsf{Gamma}(\mathrm{shape}=n,\, \mathrm{rate}=n)$$ to get an exact 95% CI of the form $(\bar X/U,\, \bar X/L),$ where $L, U$ cut probabilities $0.025$ from the lower and upper tails, respectively, for the gamma distribution given above.

For example, consider the following data and 95% CI $(7.02,\, 17.04).$

set.seed(1234)
x = rexp(20, 1/10)  # population mean 10.
summary(x)
    Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
 0.06582  2.35724  8.31061 10.40792 17.44730 30.52458 
mean(x)/qgamma(c(.975,.025), 20, 20)
[1]  7.015584 17.039088

If you had mistakenly assumed that sample x was sampled from a normal population, you would get a somewhat inaccurate "95% CI" $(6.18, 14.63).$ from your formula.

t.test(x)$conf.int
[1]  6.184321 14.631515
attr(,"conf.level")
[1] 0.95

If you did not know the distribution families from which samples x and y were drawn, then you might use nonparametric bootstrap CIs for the population means.

There are many styles of bootstraps. Unfortunately, the quantile method you describe in your question does not always give the best results. I will use a slight modification of it based on distances between re-sample means and the means of the original samples.

The sample size $n = 20$ is barely adequate for bootstrapping. But let's see what CIs we get for the population means of normal sample y and exponential sample y, respectively.

First, the normal sample for which the t CI is $(45.522, 51.140)$ The 95% nonparametric bootstrap CI is $(45.903, 50.878),$ which is not the same, but also not far off.

set.seed(216);  a = mean(y)
d = replicate(2000, mean(sample(y,20,rep=T))-a)
UL = quantile(d, c(.975,.025))
a - UL
  97.5%    2.5% 
45.9033 50.8783 

Second, for the exponential sample 'x' the exact 95% CI is $(7.02, 17.04)$ and the 95% bootstrap CI is $(6.67, 14.15),$ which is not bad for such a small sample from a skewed distribution.

set.seed(217);  a = mean(x)
d = replicate(2000, mean(sample(x,20,rep=T))-a)
UL = quantile(d, c(.975,.025))
a - UL
97.5%      2.5% 
6.670051 14.148429  

Note: The exact 95% CIs above for the exponential mean are not symmetrical about $\bar X.$ Instead, they are 'probability symmetric' because they cut probability $0.025$ from each tail of the relevant gamma distribution.

Equally valid 95% CIs might split the tail probabilities, totaling 0.05, as 0.02:0.03 or as 0.03:0.02. Thus, it one usually speaks of a 95% CI instead of the 95% CI for a particular parameter. Three equally valid exact 95% CIs for the exponential mean $\mu = 1/\lambda$ above might be as follows:

set.seed(1234)
x = rexp(20, 1/10)
mean(x)/qgamma(c(.97,.02), 20, 20)
[1]  7.125314 17.464727

mean(x)/qgamma(c(.98,.03), 20, 20)
[1]  6.88854 16.69026

mean(x)/qgamma(c(.975,.025), 20, 20)
[1]  7.015584 17.039088
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.