Understanding different approaches to construct confidence intervals with bootstrap

Question

During my statistics course, we studied bootstrap and its operational principles. We explored the potential application of bootstrap in cases where we have limited knowledge about the distribution. Specifically, we utilized this technique to approximate certain statistics such as the variance of the mean.

However, I encountered some difficulties in comprehending the process of constructing confidence intervals (CI) through the utilization of bootstrap. My professor mentioned three methods, and I would greatly appreciate some guidance in comprehending each approach:

1 - Normal approximation: if I'm not mistaken, this works only when our statistic is normally distributed (asymptotically at least). If so, we can somehow use bootstrap to approximate the standard error $se$. Perhaps this works like any classical bootstrap where we generate many subsets of the dataset and just calculate the statistic for each one, like the variance in this case, and then find the $se$. I just didn't get why would it only work when the statistic is normally distributed?

2 - Pivot (pivotal intervals): This one confuses me the most. It has been defined like this: "A function $Q(X_1,...X_n;\theta)$ is a pivot if the distribution of Q does not depend on \theta". However, I am uncertain as to how this method helps in determining the CI, and whether it implies that the statistic itself is a pivot.

3 - Percentile intervala: as I understand it, involves generating multiple subsets of the data with $n$ samples in each one, from the original data distribution (or the empirical data distribution if the original is unknown). For each subset, we compute the statistic of interest. Next, we sort the computed statistics in ascending order, and the lower (left) bound of the confidence interval (CI) is determined by the $\alpha/2$ percentile, such as the 0.025 percentile, while the upper (right) bound is defined by the $1-\alpha/2$ percentile, such as the 0.975 percentile.

Although I have not fully understood the proof behind this approach, I believe that sorting the statistics generated using the bootstrap is the fundamental step in constructing the percentile intervals.

3.5 - Parametric: This is not a new approach. The parametric approach involves generating subsets of data from a known underlying distribution. We can generate the data subsets from this distribution. If it's a Normal distribution, for example, we can calculate the mean for each subset. And then, what do we do? If we only want the mean of the normal distribution we take the mean of the means?

It's crucial that I fully understand those three approaches and how to use them. The pivotal intervals I think is the one that is causing me the most trouble. It's not very intuitive, unlike the percentile approach which makes a lot of sense to me. The Normal approximation is also weird to me, I would appreciate any help I can get. If you could also provide some examples for the usage of those approaches that would also be a very big help.

Answer 1

As @dipetkov says in a comment, a full answer requires chapters if not a book. If you want to understand the bootstrap, originally developed by Efron, then you could do worse than to consult the relevant parts of Computer Age Statistical Inference (CASI) by Efron and Hastie.

What follows is a bit of guidance to point you in better directions as you undertake that study. I sense that there are some misunderstandings of what's involved in some of the bootstrap flavors that you note.

Overall, remember that what you are trying to get with any confidence interval is a range, calculated from the data sample, that would cover the true value of the statistic of interest in the specified fraction of repeated experiments (often 95%), sampling from the same population, when the confidence interval is calculated the same way. What's of interest is the distribution of a statistic.

In reverse, starting with the parametric bootstrap:

Parametric... If it's a Normal distribution, for example, we can calculate the mean for each subset. And then, what do we do? If we only want the mean of the normal distribution we take the mean of the means?

You want the distribution of estimates of the means. Or the distribution of differences between estimates for different groups. For 95% CI, you put those estimates in order and choose a range that covers 95% of the estimates, similar to how you describe the percentile bootstrap.

In some circumstances you can learn something about bias in an estimate by comparing the mean of the means of the bootstrapped samples against the original mean. If you're willing to assume a parametric distribution then that's less likely to be helpful than it is in non-parametric bootstrapping.

Percentile interval: as I understand it, involves generating multiple subsets of the data with n samples in each one, from the original data distribution (or the empirical data distribution if the original is unknown). For each subset, we compute the statistic of interest...

The thing to recognize is that the "statistic of interest" is often a difference between an estimate from a sample and the true population value. The distribution of the estimates themselves among bootstrap samples isn't always the same thing, particularly when there is bias or skew in the estimates.

Pivot (pivotal intervals): This one confuses me the most... I am uncertain as to how this method helps in determining the CI, and whether it implies that the statistic itself is a pivot.

I think there's some confusion here between the concept of a pivot and what's sometimes called the empirical/basic bootstrap. I suspect that this section of your course of study was on the empirical/basic bootstrap.

Frequentist statistical inference in general is based on analysis of pivotal quantities. See page 16 of CASI.

The reliability of the bootstrap also depends on having a pivot to analyze. That's not usually possible in practice, so the issue becomes how close to pivotal a quantity is.

The importance of a pivot might be discussed in the context of the empirical/basic bootstrap, the flavor that comes closest to following the bootstrap principle that a bootstrap re-sample is to the sample as the sample is to the population. In that method, you evaluate the distribution of the differences between the values calculated from the boostrapped samples and the value from the original sample.

This answer and its links covers the distinction between the percentile and the empirical/basic bootstrap. The empirical/basic bootstrap handles bias and skew more reliably than the percentile method. The "BCa" method can be even better.

Normal approximation... I just didn't get why would it only work when the statistic is normally distributed?

If the statistic of interest doesn't have a normal distribution, then there's no assurance that a range covering 95% of an approximating normal distribution would cover 95% of the distribution of the statistic of interest. Think in particular about an asymmetric distribution versus the symmetric normal distribution.