Pros and cons of bootstrapping I have just learned about the concept of bootstrapping, and a naive question came to mind: If we can always generate numerous bootstrap samples of our data, why bother to obtain more "real" data at all? 
I do think I have an explanation, please tell me if I'm correct: I think the bootstrapping process reduces variance, BUT if my original dataset is BIASED, than I'm stuck with low variance and high bias, no matter how many replicas I'm taking.  
 A: A bootstrap sample can only tell you things about the original sample, and won't give you any new information about the real population. It is simply a nonparametric method for constructing confidence intervals and similar.
If you want to gain more information about the population, you have to gather more data from the population.
A: The bootstrap is a method of doing inference in a way that does not require assuming a parametric form for the population distribution.  It does not treat the original sample as if it is the population even those it involves sampling with replacement from the original sample. It assumes that sampling with replacement from the original sample of size n mimics taking a sample of size n from a larger population.  It also has many variants such as the m out of n bootstrap which re-samples m time from a sample of size n where m < n. The nice properties of the bootstrap depend on asymptotic theory. As others have mentioned the bootstrap does not contain more information about the population than what is given in the original sample. For that reason it sometimes doesn't work well in small samples.
In my book "Bootstrap Methods: A Practitioners Guide" second edition published by Wiley in 2007, I point out situations where the bootstrap can fail.  This includes distributions that do not have finite moments, small sample sizes, estimating extreme values from the distribution and estimating variance in survey sampling where the population size is N and a large sample n is taken.  In some cases variants of the bootstrap can work better than the original approach.  This happens with the m out of n bootstrap in some applications In the case of estimating error rates in discriminant analysis, the 632 bootstrap is an improvement over other methods including other bootstrap methods..
A reason for using it is that sometimes you can't rely on parametric assumptions and in some situations the bootstrap works better than other non-parametric methods. It can be applied to a wide variety of problems including nonlinear regression, classification, confidence interval estimation, bias estimation, adjustment of p-values and time series analysis to name a few.
