Why do we need Bootstrapping? I'm currently reading Larry Wasserman's "All of Statistics" and puzzled by something he wrote in the chapter about estimating statistical functions of nonparametric models.
He wrote 

"Sometimes we can find the estimated standard error of a statistical
  function by doing some calculations. However in other cases it's not
  obvious how to estimate the standard error".

I'd like to point out that in the next chapter he talks about bootstrap to address this issue, but since I don't really understand this statement I don't fully get the incentive behind Bootstrapping? 
What example is there for when it's not obvious how to estimate the standard error?
All the examples I've seen so far have been "obvious" such as $X_1,...X_n ~Ber(p)$ then $ \hat{se}(\hat{p}_n )=\sqrt{\hat{p}\cdot(1-\hat{p})/n}$
 A: An example might help to illustrate. Suppose, in a causal modeling framework, you're interested in determining whether the relation between $X$ (an exposure of interest) an $Y$ (an outcome of interest) is mediated by a variable $W$. This means that in the two regression models:
$$\begin{eqnarray}
E[Y|X] &=& \beta_0 + \beta_1 X \\
E[Y|X, W] &=& \gamma_0 + \gamma_1 X + \gamma_2 W \\
\end{eqnarray}$$
The effect $\beta_1$ is different than the effect $\gamma_1$. 
As an example, consider the relationship between smoking and cardiovascular (CV) risk. Smoking obviously increases CV risk (for events like heart attack and stroke) by causing veins to become brittle and calcified. However, smoking is also an appetite suppressant. So we would be curious whether the estimated relationship between smoking and CV risk is mediated by BMI, which independently is a risk factor for CV risk. Here $Y$ could be a binary event (myocardial or neurological infarction) in a logistic regression model or a continuous variable like coronary arterial calcification (CAC), left ventricular ejection fraction (LVEF), or left ventricular mass (LVM). 
We would fit two models 1: adjusting for smoking and the outcome along with other confounders like age, sex, income, and family history of heart disease then 2: all the previous covariates as well as body mass index. The difference in the smoking effect between models 1 and 2 is where we base our inference.
We are interested in testing the hypotheses 
$$\begin{eqnarray}
\mathcal{H} &:& \beta_1 = \gamma_1\\
\mathcal{K} &:& \beta_1 \ne \gamma_1\\
\end{eqnarray}$$
One possible effect measurement could be: $T = \beta_1 - \gamma_1$ or $S = \beta_1 / \gamma_1$ or any number of measurements. You can use the usual estimators for $T$ and $S$. The standard error of these estimators is very complicated to derive. Bootstrapping the distribution of them, however, is a commonly applied technique, and it is easy to calculate the $p$-value directly from that.
A: Having parametric solutions for each statistical measure would be desirable but, at the same time, quite unrealistic. Bootstrap comes in handy in those instances. The example that springs to my mind concerns the difference between two means of highly skewed cost distributions. In that case, the classic two-sample t-test fails to meet its theoretical requirements (the distributions from which the samples under investigation were drawn surely depart from normality, due to their long right-tail) and non-parametric tests lack to convey useful infromation to decision-makers (who are usually not interested in ranks). A possible solution to avoid being stalled on that issue is a two-sample bootstrap t-test.
A: Two answers. 


*

*What's the standard error of the ratio of two means? What's the standard error of the median? What's the standard error of any complex statistic? Maybe there's a closed form equation, but it's possible that no one has worked it out yet. 

*In order to use the formula for (say) the standard error of the mean, we must make some assumptions. If those assumptions are violated, we can't necessarily use the method. As @Whuber points out in the comments, bootstrapping allows us to relax some of these assumptions and hence might provide more appropriate standard errors (although it may also make additional assumptions). 

