Why do we need to resample from an initial set of samples when using bootstrapping? Why don't we just take fresh sets of samples from the original distribution? What is the justification behind resampling? Or is it just a computational trick?
EDIT: I understand that it is usually very expensive to get fresh samples, but assuming we have access to a generator of the original distribution, does it still make sense to resample due to some theoretical reasons?
In real life, sample points have real costs, so sampling from the true distribution is, at best, expensive, and at worst, impossible. So, it's the money. The bootstrap is mostly free.
As @NickCox puts it with exceptional visual aplomb:
You wouldn't relaunch the Titanic and crash it into another iceberg
Because, you know, that would be expensive (in many, many ways).
I understand that it is usually very expensive to get fresh samples, but assuming we have access to a generator of the original distribution, does it still make sense to re-sample due to some theoretical reasons?
No. If it's free to sample from the population distribution, you should do so.
