Why isn't bootstrapping done in the following manner?

Question

I'm under the impression that when you bootstrap, your final results are the original statistic from your sample data, and the standard errors from the bootstrapped trials. However, it seems more intuitive to take the mean statistic from all your trials, rather than just the statistic from the original trial. Is there some statistical intuition why it is one and not the other?

Also, I came across a use case where someone uses bootstrapping using the mean as the statistic. They did their sampling, took the mean of each trial, and used that to calculate the confidence interval around the mean. Is this ok? It seems like you could draw confidence intervals using the original data itself, and bootstrapping would artificially lower the standard errors. Again, is there some intuition I could use to understand why this is ok/not ok?

Answer 1

The idea of the bootstrap is to estimate the sampling distribution of your estimate without making actual assumptions about the distribution of your data.

You usually go for the sampling distribution when you are after the estimates of the standard error and/or confidence intervals. However, your point estimate is fine. Given your data set and without knowing the distribution, the sample mean is still a very good guess about the central tendency of your data. Now, what about the standard error? The bootstrap is a good way getting that estimate without imposing a probabilistic distribution for data.

More technically, when building a standard error for a generic statistic, if you knew the sampling distribution of your estimate $\hat \theta$ is $F$, and you wanted to see how far you can be from it's mean $\mu$, the quantity $\hat \theta$ estimates, you could look at the differences from the mean of the sampling distribution $\mu$, namely $\delta$, and make that the focus of your analysis, not $\hat \theta$

$$ \delta = \hat \theta - \mu $$

Now, since we know that $\hat \theta \sim F$, when know that $\delta$ should be related with $F$ minus the constant $\mu$. A type of "standardization" as we do with the normal distribution. And with that in mind, just compute the 80% confidence interval such that

$$ P_F(\delta_{.9} \le \hat \theta - \mu \le \delta_{.1} | \mu) = 0.8 \leftrightarrow P_F(\hat \theta - \delta_{.9} \ge \mu \ge \ \hat \theta - delta_{.1} | \mu) = 0.8 $$

So we just build the CI as $\left[\hat \theta - \delta_{.1}, \hat \theta - \delta_{.9} \right]$. Keep in mind that we don't know $F$ so we cant know $\delta_{.1}$ or $\delta_{.9}$. And we don't want to assume that it is normal and just look at the percentiles of a standard normal distribution either.

The bootstrap principle helps us estimate the sampling distribution $F$ by resampling our data. Our point estimate will be forever $\hat \theta$. There isn't anything wrong with it. But if I take another resample I can built $\hat \theta^*_1 $. And then another resmple $\hat \theta^*_2 $. And then another $\hat \theta^*_3 $. I think you get the idea.

With a set of estimates $\hat \theta^*_1 ... \hat \theta^*_n$ has a distribution $F^*$ which approximates $F$. We can then compute $$ \delta^*_i = \hat \theta^*_i - \hat \theta $$

Notice that the point estimate for the $\mu$ is replaced by our best guess $\hat \theta$. And look at the empirical distribution of $\theta^*$ to compute $\left[\hat \theta - \delta^*_{.1}, \hat \theta - \delta^*_{.9} \right]$.

Now, this explanation is heavily based on this MIT class on the bootstrap. I highly recommend you give it a read.

Answer 2

That's not OK. You would need to use the double bootstrap to get a correct confidence interval from a new estimator that is a function of many bootstrap estimates. The bootstrap was not created to provide new estimators, except in rare cases such as the Harrell-Davis quantile estimator. The main function of the bootstrap is to study the performance of an existing estimator, or to tell how bad the estimator is (e.g., in terms of variance or bias). The bootstrap can also provide confidence intervals for strange quantities such as the number of modes in a continuous distribution.

Answer 3

The reason you typically take the statistic calculated from all data as your point estimate is that (at least for a mean) with the number of bootstrap samples going to infinity, you will get that same answer. I.e. any deviation is just due to the number of bootstrap samples and you might just as well use the known exact answer.

In the second part of your question, what do you mean by calculating the confidence around the mean "using the original data"? The main reason you use boostrapping is usually that there's no simple formula for just getting a CI from the original data. If you mean taking the variation in the original data (e.g. take 1.96 $\times$ SD of original data), then that's not a confidence interval for the mean, but rather an interval that also describes the variation in the outcome.

Answer 4

On the first question: if the statistic you are interested in is not the mean, then there are cases where taking the mean statistic from all the resampling trials is arguably better than taking the single statistic from the original trial.

For example, suppose you are interested in the median of a distribution. The distribution turns out to be bimodal with narrow peaks at 0 and 1. You have 99 points in your sample, of which 50 are near 0 and 49 are near 1. It's too close to call whether the population median is nearer 0 or 1. Your sample median is close to 0, but if you wanted to minimise the mean squared error of your estimate of the population median, you would want your estimate to be something close to 0.5.