I'm currently reading Larry Wasserman's "All of Statistics" and puzzled by something he wrote in the chapter about estimating statistical functions of nonparametric models.

He wrote

"Sometimes we can find the estimated standard error of a statistical function by doing some calculations. However in other cases it's not obvious how to estimate the standard error".

I'd like to point out that in the next chapter he talks about bootstrap to address this issue, but since I don't really understand this statement I don't fully get the incentive behind Bootstrapping?

What example is there for when it's not obvious how to estimate the standard error?

All the examples I've seen so far have been "obvious" such as $X_1,...X_n ~Ber(p)$ then $ \hat{se}(\hat{p}_n )=\sqrt{\hat{p}\cdot(1-\hat{p})/n}$


3 Answers 3


Two answers.

  1. What's the standard error of the ratio of two means? What's the standard error of the median? What's the standard error of any complex statistic? Maybe there's a closed form equation, but it's possible that no one has worked it out yet.
  2. In order to use the formula for (say) the standard error of the mean, we must make some assumptions. If those assumptions are violated, we can't necessarily use the method. As @Whuber points out in the comments, bootstrapping allows us to relax some of these assumptions and hence might provide more appropriate standard errors (although it may also make additional assumptions).
  • 2
    $\begingroup$ Answer 1 is fine, but answer 2 seems to beg the question, because bootstrapping makes assumptions, too. I suppose the point might be that it typically makes different assumptions than other popular procedures, but that's just my guess about what you're trying to say and I could be mistaken. $\endgroup$
    – whuber
    Commented May 29, 2015 at 19:04
  • $\begingroup$ @Whuber - thanks, I've added a bit of clarification. $\endgroup$ Commented May 29, 2015 at 19:07
  • 5
    $\begingroup$ Thank you for the edits. But isn't it the case that bootstrapping typically makes different assumptions, rather than actually relaxing some? For instance, the assumptions needed to estimate an SE of a sample mean are that the data are iid and the underlying distribution has a finite variance. The bootstrap actually has to add assumptions in this case: it doesn't work unless the sample size is "sufficiently large." Although this might seem like quibbling over technicalities, what I am trying to address is the big picture: bootstrapping is neither a panacea nor is it always applicable. $\endgroup$
    – whuber
    Commented May 29, 2015 at 19:18
  • 3
    $\begingroup$ @JeremyMiles the bootstrap is not free of assumptions. You need to verify that the distribution is pivotal for most bootstrap error calculations which can often be more complicated than obtaining a consistent estimator for a standard error. Additionally, the ratio of means has a very easy error approximation obtained from the δ-method. So I don't think that example defies the OP's point. $\endgroup$
    – AdamO
    Commented May 29, 2015 at 20:38

An example might help to illustrate. Suppose, in a causal modeling framework, you're interested in determining whether the relation between $X$ (an exposure of interest) an $Y$ (an outcome of interest) is mediated by a variable $W$. This means that in the two regression models:

$$\begin{eqnarray} E[Y|X] &=& \beta_0 + \beta_1 X \\ E[Y|X, W] &=& \gamma_0 + \gamma_1 X + \gamma_2 W \\ \end{eqnarray}$$

The effect $\beta_1$ is different than the effect $\gamma_1$.

As an example, consider the relationship between smoking and cardiovascular (CV) risk. Smoking obviously increases CV risk (for events like heart attack and stroke) by causing veins to become brittle and calcified. However, smoking is also an appetite suppressant. So we would be curious whether the estimated relationship between smoking and CV risk is mediated by BMI, which independently is a risk factor for CV risk. Here $Y$ could be a binary event (myocardial or neurological infarction) in a logistic regression model or a continuous variable like coronary arterial calcification (CAC), left ventricular ejection fraction (LVEF), or left ventricular mass (LVM).

We would fit two models 1: adjusting for smoking and the outcome along with other confounders like age, sex, income, and family history of heart disease then 2: all the previous covariates as well as body mass index. The difference in the smoking effect between models 1 and 2 is where we base our inference.

We are interested in testing the hypotheses $$\begin{eqnarray} \mathcal{H} &:& \beta_1 = \gamma_1\\ \mathcal{K} &:& \beta_1 \ne \gamma_1\\ \end{eqnarray}$$

One possible effect measurement could be: $T = \beta_1 - \gamma_1$ or $S = \beta_1 / \gamma_1$ or any number of measurements. You can use the usual estimators for $T$ and $S$. The standard error of these estimators is very complicated to derive. Bootstrapping the distribution of them, however, is a commonly applied technique, and it is easy to calculate the $p$-value directly from that.

  • $\begingroup$ I think I understand where you're going with this answer, but I am puzzled by the details. Did you intend to put hats over the parameters in your descriptions of $T$ and $S$? The text sounds like these should be properties of a model rather than estimators. What sense does it make to mix properties of two different models like this? If you really did mean hats, then $T$ and $S$ are statistics, apparently to be used as estimators, but what are they intended to estimate? $\endgroup$
    – whuber
    Commented May 29, 2015 at 19:01
  • $\begingroup$ @whuber I think you're right that in conventional notation they don't use hats. I will make the edit. Perhaps I was not clear enough... there are two parameters for the same variable fit in two different models on the same dataset. It is very difficult to directly calculate the standard error of the statistics $T$ and $S$. $\endgroup$
    – AdamO
    Commented May 29, 2015 at 19:14
  • $\begingroup$ The only way I have been able to make sense of this is to understand the second model to be nested in the first, so that the hypothesis you are testing is $\gamma_2 = 0$. I do not even know of a valid definition of "hypothesis" that involves two separate models. $\endgroup$
    – whuber
    Commented May 29, 2015 at 19:20
  • $\begingroup$ @whuber Ah I see the confusion. Please see a recommended article from MacKinnon here. $\endgroup$
    – AdamO
    Commented May 29, 2015 at 19:27
  • $\begingroup$ Thank you: that reference helps me understand your example much better. Although I have reservations about the many theoretical solecisms involved in that approach, they are irrelevant to the aptness of your example: it suffices that people have actually tried to understand data in this way and have seen a need to estimate standard errors for estimators of $T$ or $S$. I notice, though, that your last paragraph still does not distinguish between $T$ and its estimator: $T$ is a model property and as such has no distribution and no SE. An estimator of $T$ does have a distribution. $\endgroup$
    – whuber
    Commented May 29, 2015 at 19:37

Having parametric solutions for each statistical measure would be desirable but, at the same time, quite unrealistic. Bootstrap comes in handy in those instances. The example that springs to my mind concerns the difference between two means of highly skewed cost distributions. In that case, the classic two-sample t-test fails to meet its theoretical requirements (the distributions from which the samples under investigation were drawn surely depart from normality, due to their long right-tail) and non-parametric tests lack to convey useful infromation to decision-makers (who are usually not interested in ranks). A possible solution to avoid being stalled on that issue is a two-sample bootstrap t-test.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.