2
$\begingroup$

I just learned Bootstrap Method from my Statistics course. The teacher says that the whole population is unknown, however we have some sample set $\mathcal{D}$ with sample size $N$. Then we use this sample set $\mathcal{D}$ as population and sample from $\mathcal{D}$ with replacement to compute all statistics we want (like mean, variance, median, or train a ML model from resampled set, etc.). We will have a fantatic result from this method (average result from all resampled sets).

However, I totally cannot understand the spirit of resampling from $\mathcal{D}$. For example, the actual distribution is Bernoulli distribution with $p = 0.5$. I get a sample set $\mathcal{D}$ with sample size $100$ from this distribution with $52$ of one and $48$ of zero. If I resample with replacement from $\mathcal{D}$, it just means that I have a Bernoulli distribution with $p = 0.52$.

My Questions:

  1. Why shouldn't I directly compute all statistics I want from $\mathcal{D}$? No matter how many times of resampling, I can only get converged $\text{mean} = 0.52$ and $\text{var} = 0.52 * 0.48$. I don't think I can get any improvement from resampling or I can even recover the underlying distribution. Am I right?

  2. What's the loophole of my arguments? What's the advantage of Bootstrap? For example, is there anything I cannot get from sample set $\mathcal{D}$ but from $Bootstrap$?

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
3
$\begingroup$

The main application of the bootstrap is to learn something about the sampling distribution of the statistic, not the statistic itself. While it is easy to calculate an estimate $\hat D$ of the ratio $D$ of two medians from data, there might not be a reliable formula for its standard error $\sqrt{\text{Var}(\hat D)}$. You would need that to construct simple confidence intervals. The bootstrap provides you with methods to estimate measures like $\sqrt{\text{Var}(\hat D)}$.

Improve this answer
$\endgroup$
2
$\begingroup$

Please read the whole referenced Wikipedia article. There is a lot of background. My recommendation is to distinguish assessment of central moments from "bayesian" analysis of the distribution as a whole. The one of main usage of bootstraping nowadays is assessment of confidence intervals, where the assumptions of normality (or some other expected properties) are not valid. Be careful as the bootstrap method is not recommended for fat tailed distribution. Easy to imagine that if you do not have extreme values from such distribution in the sample then the assessment will be broken. Now one of the most important that for very complex use cases we could obtained the confidence intervals without any mathematical understanding of the process. We could simple repeat n times the model with resampled data and get the distribution of results.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.