I'm still wrapping my head around bootstrapping but am struggling to think of how it is applied. I have looked at: Explaining to laypeople why bootstrapping works
So far, my understanding is that the key implicit assumption behind bootstrapping is that we assume our observed sample is a proxy for the population. In other words, the frequencies of our sample observations is proportionate to the population density.
We then sample from our observed data (treating it as the population) with replacement, and generate basically a bunch of smaller samples. From these smaller samples, we calculate statistics of interest like mean, SD, etc...
The second key point is that this allows us to produce sampling distributions from which we can then determine the precision of our sample statistics (mean, SD, etc...).
Maybe it's just when this topic has been introduced in class but I still don't get why we would use bootstrapping and when would we apply it. It seems to be redundant or add no value to what we are doing. Don't for most statistics (such as mean, regression coefficients B0, B1, etc...) have sampling distributions that are approximately normal? What would bootstrapping add?
I guess in class I have never thought of when I would ever use bootstrapping as it's just kinda thrown in as a side note (here's bootstrapping and look what it does). Is bootstrapping just another way to numerically solve something that we cannot analytically do? And if so, what's like a good example of this?
Like most explanations I have seen for bootstrapping tell you how to implement it or what it is but never tell you when you would actually apply it and the value it brings as an additional tool.