If all you are doing is re-sampling from the empirical distribution, why not just study the empirical distribution? For example instead of studying the variability by repeated sampling, why not just quantify the variability from the empirical distribution?
Bootstrapping (or other resampling) is an experimental method to estimate the distribution of a statistic.
It is a very straightforward and easy method (it just means you compute with many random variants of the sample data in order to obtain, an estimate of, the desired distribution of the statistic).
You most likely use it when the 'theoretical/analytical' expression is too difficult to obtain/calculate (or like aksakal says sometimes they are unknown).
- Example 1: If you do a pca analysis and wish to compare the results with 'estimates of the deviation of the eigenvalues' given the hypothesis that there is no correlation in the variables.
You could, scramble the data many times and re-computing the pca eigenvalues such that you get a distribution (based on random tests with the sample data) for the eigenvalues.
Note that the current practices are gazing at a scree plot and apply rules of thumb in order to 'decide' whether a certain eigenvalue is significant/important or not.
- Example 2: You did a non-linear regression y ~ f(x) providing you with some estimate of bunch of parameters for the function f. Now you wish to know the standard error for those parameters.
Some simple look at the residuals and linear algebra, like in OLS, is not possible here. However, an easy way is to compute the same regression many times with the residuals/errors re-scrambled in order to get an idea how the parameters would vary (given the distribution for the error term can be modeled by the observed residuals).
The key thing is that the bootstrap isn't really about figuring out features of the distribution of the data, but rather figuring out features of an estimator applied to the data.
Something like empirical distribution function will tell you a fairly good estimate of the CDF from which the data came from...but by in isolating, it tells you essentially nothing about how reliable the estimators we build from that data will be. This is the question answered by using bootstrap.
IF you know exactly what is the underlying distribution, then you don't need to study it. Sometimes, in natural sciences you know exactly the distribution.
IF you know the type of the distribution, then you only need to estimate its parameters, and study it in the sense you meant. For instance, sometime you know a priori that the underlying distribution is normal. In some cases you even know what is its mean. So, for normal the only thing that is left to find out is the standard deviation. You get the sample standard deviation from the sample, and voila, you get the distribution to study.
IF you don't know what is the distribution, but think that it is one of the several in the list, then you could try to fit those distribution to data, and pick the one that fits best. THEN you study that distribution.
FINALLY, often you don't know type of distribution you're dealing with. And you do't have a reason to believe that it belongs to one of 20 distributions that R can fit your data to. What are you going to do? Ok, you look at mean and standard deviations, nice. But what if it's very skewed? What if its kurtosis is very large? and so on. You really need to know all the moments of distribution to know, and study it. So, in this case non parametric bootstrapping comes handy. You don't assume much, and simple sample from it, then study its moments and other properties.
Though non-parametric bootstrapping is not a magical tool, it has issues. For instance, it can be biased. I think parametric bootstrapping is unbiased