4
$\begingroup$

My question is based on the notes from the relevant lectures.

When deriving confidence intervals (CI) from the null hypothesis significance test, we invoke the assumption that the data comes from the distribution parametrized according to the null hypothesis, and arrive at a statement of the following form:

The 95% CI is $[x_{avg}-1.96*\sigma/n^{0.5},x_{avg}+1.96*\sigma/n^{0.5}]$, where $x_{avg}$ is the empirical mean computed from samples drawn from a normal distribution with some unknown mean $\mu$ and known standard deviation $\sigma$, conditioned on/ASSUMING the data is drawn from the distribution defined by the null hypothesis. The assumption was used to compute the desired probability of 95%.

In non-parametric bootstrapping, used to compute the CI for data derived from unknown distributions, we estimate the variation in the statistic e.g., empirical mean from the population mean using variations between the mean estimated on samples drawn from the resampled distribution, and the empirical mean, and then derive CI according to the distribution of the deviations. But in this calculation, we don't use any information from the null hypothesis, so is it correct to state that in non-parametric bootstrapping, the CI is the probability that the quantity of interest e.g., population mean lies in the computed interval, conditioned on nothing?

$\endgroup$
1
  • 2
    $\begingroup$ Condition on nothing is too strong here. More justified view could read: Bootstrap principle is based on the assumption the set of resampled empirical distributions can approximate "true" data distribution. $\endgroup$
    – msuzen
    Sep 10, 2021 at 1:21

1 Answer 1

3
$\begingroup$

I think it is a mistake to assume that a CI must depend on a null distribution. Consider the fictitious normal sample of size $n = 100$ below:

Confidence interval based on data and t distribution. The 95% t confidence interval is based on the sample. Specifically, it is $\bar X \pm t^*S/\sqrt{n},$ where $t^*$ cuts probability 0.025 from the upper tail of $\mathsf{T}(\nu=99),$ which computes to $(47.35,\, 50.21).$

set.seed(2021)
x = rnorm(100, 50, 7)
mean(x);  sd(x)
[1] 48.77861
[1] 7.200381

stripchart(x, pch="|")

enter image description here

ci = mean(x) + qt(c(.025,.975),99)*sd(x)/sqrt(100); ci
[1] 47.34990 50.20732

Two-sided t tests at 5% level of significance. We can use this sample to test $H_0: \mu = 55$ against $H_a: \mu \ne 55$ (rejecting $H_0$ at the 5% level with P-value near $0)$ or to test $H_0: \mu = 50.01$ against $H_a: \mu \ne 50.01$ (failing to reject with P-value $0.07 > 0.05).$ Either way, the CI reported by the t.test procedure in R is the same as above.

t.test(x, mu=55)

        One Sample t-test

data:  x
t = -8.6404, df = 99, p-value = 1e-13
alternative hypothesis: 
 true mean is not equal to 55
95 percent confidence interval:
 47.34990 50.20732  # same as above
sample estimates:
mean of x 
 48.77861 

t.test(x, mu=50.1)

        One Sample t-test

data:  x
t = -1.8352, df = 99, p-value = 0.06948
alternative hypothesis: 
 true mean is not equal to 50.1
95 percent confidence interval:
 47.34990 50.20732  # again, same as above
sample estimates:
mean of x 
 48.77861 

However, there is a connection between the 95% CI and the two-sided t test at the 5% level of significance. Any hypothetical value within the CI will not be rejected (as for 50.0); and any hypothetical value outside the CI will be rejected (as was 55).

Nonparametric bootstrap CIs. Now suppose we do not know that the data x were sampled from a normal distribution. Thus, we are not sure that a valid 95% CI for the population mean $\mu$ can be obtained by using Student's t distribution.

A 95% nonparametric CI can be obtained by re-sampling the data. There are many styles of bootstrap CIs. They are based on data without reference to the null hypothesis of a test.

You may have a particular style of bootstrap CI in mind. One of the simplest, which may be OK because our sample seems roughly symmetrical, is illustrated below. The result is $(47.4,\, 50.16),$ which is not very different from the t CI for $\mu$ above.

a.re = replicate(2000, mean(sample(x,100,rep=T)))
quantile(a.re, c(.025,.975))
    2.5%    97.5% 
 47.38079 50.16279 

Addendum: The one-sample Wilcoxon signed rank procedure in R gives a 95% nonparametric confidence interval $(47.20, 50.23)$ for the "center" of the population. (The median of the sample x is $49.06.)$

wilcox.test(x, conf.int=T)$conf.int
[1] 47.20100 50.22682
attr(,"conf.level")
[1] 0.95
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.