During my statistics course, we studied bootstrap and its operational principles. We explored the potential application of bootstrap in cases where we have limited knowledge about the distribution. Specifically, we utilized this technique to approximate certain statistics such as the variance of the mean.
However, I encountered some difficulties in comprehending the process of constructing confidence intervals (CI) through the utilization of bootstrap. My professor mentioned three methods, and I would greatly appreciate some guidance in comprehending each approach:
1 - Normal approximation: if I'm not mistaken, this works only when our statistic is normally distributed (asymptotically at least). If so, we can somehow use bootstrap to approximate the standard error $se$. Perhaps this works like any classical bootstrap where we generate many subsets of the dataset and just calculate the statistic for each one, like the variance in this case, and then find the $se$. I just didn't get why would it only work when the statistic is normally distributed?
2 - Pivot (pivotal intervals): This one confuses me the most. It has been defined like this: "A function $Q(X_1,...X_n;\theta)$ is a pivot if the distribution of Q does not depend on \theta". However, I am uncertain as to how this method helps in determining the CI, and whether it implies that the statistic itself is a pivot.
3 - Percentile intervala: as I understand it, involves generating multiple subsets of the data with $n$ samples in each one, from the original data distribution (or the empirical data distribution if the original is unknown). For each subset, we compute the statistic of interest. Next, we sort the computed statistics in ascending order, and the lower (left) bound of the confidence interval (CI) is determined by the $\alpha/2$ percentile, such as the 0.025 percentile, while the upper (right) bound is defined by the $1-\alpha/2$ percentile, such as the 0.975 percentile.
Although I have not fully understood the proof behind this approach, I believe that sorting the statistics generated using the bootstrap is the fundamental step in constructing the percentile intervals.
3.5 - Parametric: This is not a new approach. The parametric approach involves generating subsets of data from a known underlying distribution. We can generate the data subsets from this distribution. If it's a Normal distribution, for example, we can calculate the mean for each subset. And then, what do we do? If we only want the mean of the normal distribution we take the mean of the means?
It's crucial that I fully understand those three approaches and how to use them. The pivotal intervals I think is the one that is causing me the most trouble. It's not very intuitive, unlike the percentile approach which makes a lot of sense to me. The Normal approximation is also weird to me, I would appreciate any help I can get. If you could also provide some examples for the usage of those approaches that would also be a very big help.