Connection between bootstrap (both parametric and non-parametric) and maximum likelihood Can anyone show me what the connection between bootstrap methods and maximum likelihood is?  
I was told that as the number of bootstraps goes to infinity, the statistics like confidence intervals, mean, variance and stability are same values that maximum likelihood would estimate.
 A: From page 267 of Elements of Statistical Learning:
8.2.3 Bootstrap versus Maximum Likelihood
In essence the bootstrap is a computer implementation of nonparametric or parametric maximum likelihood. The advantage of the bootstrap over the maximum likelihood formula is that it allows us to compute maximum likelihood estimates of standard errors and other quantities in settings where no formulas are available. In our example, suppose that we adaptively choose by cross-validation the number and position of the knots that define the B-splines, rather than fix them in advance. Denote by λ the collection of knots and their positions. Then the standard errors and confidence bands should account for the adaptive choice of λ, but there is no way to do this analytically. With the bootstrap, we compute the B-spline smooth with an adaptive choice of knots for each bootstrap sample. The percentiles of the resulting curves capture the variability from both the noise in the targets as well as that from ˆλ. In this particular example the confidence bands (not shown) don’t look much different than the fixed λ bands. But in other problems, where more adaptation is used, this can be an important effect to capture.
