# Bootstrap variance of squared sample mean

The following is question 8 of chapter 8 in Wasserman's All of Statistics:

Let $T_n = \overline{X}_n^2$, $\mu = \mathbb{E}(X_1)$, $\alpha_k = \int|x - \mu|^kdF(x)$, and $\hat{\alpha}_k = n^{-1}\sum_{i=1}^n|X_i - \overline{X}_n|^k$.

Show that $$v_{\mathrm{boot}} = \frac{4\overline{X}_n^2\hat{\alpha}_2}{n} + \frac{4\overline{X}_n\hat{\alpha}_3}{n^2} + \frac{\hat{\alpha}_4}{n^3} \>.$$

He previously defines $v_{\mathrm{boot}} = \frac{1}{B}\sum_{b=1}^B(T_{n,b}^* - \frac{1}{B}\sum_{r=1}^BT_{n,r}^*)^2$, where $T_{n,i}^*$ is the desired statistic from the $i$th bootstrap replication of the sample $X_1,...,X_n$.

It seems that the question as stated does not make sense: how can there be a formula for the bootstrap variance if the quantity requires simulation? Perhaps he meant to ask for the variance of the sampling distribution, but I get $\frac{\sigma^2}{n}$ for that. Any hints on how to intepret or solve this?

• Hi Alex, welcome to the site. Here is what I think the question is asking: Find $v_{\mathrm{boot}}$ which is the variance under the (empirical) measure $\hat F_n$. The "previously defined" version is simply the Monte Carlo estimate of $v_{\mathrm{boot}}$ rather than the quantity itself. Even so, I think what you'll find is that there are at least two other typos lurking: (1) The definition of $\hat \alpha_k$ should probably not include the modulus and (2) I believe the last term on the right-hand size is not quite correct. :) – cardinal Apr 8 '12 at 19:20
• (When I looked on that page of the book, I saw what I think was at least one other typo in the question before that as well. Also, this same problem, with the same [conjectured] errors appears to be reproduced in Wasserman's All of Nonparametric Statistics as well, on page 39. Once you've completed the exercise, you might consider sending a note to the author so that he can add it to the errata.) – cardinal Apr 8 '12 at 19:23
• Alex, it is important to understand that the bootstrap does not require simulation. The two concepts are separable: the bootstrap is a well-defined statistic, equal to a complicated function of the sample. In some cases (as in this situation), the complicated function simplifies greatly and has a closed form in terms of easily computed quantities: no simulation or actual resampling are needed. However, in many practical cases it is easier to approximate the bootstrap with a simulation rather than work out a computationally simple expression (if one even exists). – whuber Apr 9 '12 at 15:08
• @whuber and cardinal, thank you each for the explanations, I understand much better now. I believe I've worked out the question (the last term is indeed different) and will notify the author once I type it up. Is it accepted to post the solution here as well? Finally, how can I credit you for the help? – AlexK Apr 10 '12 at 18:39
• Hi AlexK. Yes, please do post your solution as an answer! I was hoping this would be the outcome. I do not want to speak for @whuber too much (though I doubt he'll mind in this instance), but do not be concerned with "crediting" us. I, for one, am happy to see you've arrived at a positive result and have benefitted from the site. I hope you'll continue to frequent it and participate. Cheers. – cardinal Apr 15 '12 at 2:27

A little late, but anyways... First, to simplify later calculations, rewrite the sample mean in terms of an expression containing the central moments under the empirical measure. Let $S_n = \frac{1}{n}\sum(X_i - \bar{X}_n) = 0$. Then $$\bar{X}_n = S_n +\bar{X}_n = \frac{1}{n}\sum (X_i - \bar{X}_n) + \bar{X}_n$$ Now, Var$(\bar{X}_n^2) = E(\bar{X}_n^4) - (E\bar{X}_n^2)^2$. We'll tackle the first term. Note that $\bar{X}_n$ is the mean under the empirical measure, so we treat it as a constant when taking expectations. \begin{align} E(\bar{X}_n^4) &= E(S_n + \bar{X}_n)^4 \\ &= E(S_n^4 + 4\bar{X}_nS_n^3 + 6\bar{X}_n^2S_n^2 + 4\bar{X}_n^3S_n + \bar{X}_n^4) \\ &= E(S_n^4) + 4\bar{X}_nE(S_n^3) + 6\bar{X}_n^2E(S_n^2) + \bar{X}_n^4 \end{align} where we used that $S_n = 0$ to drop the second-to-last term. In the following expansions, terms involving a product with $nS_n$ will not be written. \begin{align} E(S_n^4) &= E\left(\frac{1}{n^4}\left[\sum(X_i - \bar{X}_n)^4 + \sum\sum(X_i - \bar{X}_n)^2(X_j - \bar{X}_n)^2\right]\right) \\ &= \frac{\hat{a}_4}{n^3} + \frac{3(n-1)\hat{a}_2^2}{n^3}\\ E(S_n^3) &= \left(\frac{1}{n^3}\sum(X_i - \bar{X}_n)^3\right) = \frac{\hat{a}_3}{n^2}\\ E(S_n^2) &= \left(\frac{1}{n^2}\sum(X_i - \bar{X}_n)^2\right) = \frac{\hat{a}_2}{n} \end{align} These are straightforward sums of products with some combinatorics to count the number of terms. Doing similar calculations for the second term of the variance and putting it all together: $$Var(\bar{X}_n^2) = \frac{4\bar{X}_n^2\hat{a}_2}{n} + \frac{4\bar{X}_n\hat{a}_3}{n^2} + \frac{\hat{a}_4 + (2n - 3)\hat{a}_2^2}{n^3}$$

• Shouldn't it rather be $$E(S_n^4)=E(\frac{1}{n^4}[\sum_{i=1}^n(X_i^*-\bar X_n)^4 + \sum_{i\neq j}(X_i^*-\bar X_n)^2(X_j^*-\bar X_n)^2])=\frac{\hat \alpha_4}{n^3} + \frac{(n-1)\hat\alpha_2^2}{n^3},$$ which leads to the corrected term $\frac{\hat\alpha_4 - \hat\alpha_2^2}{n^3}$ in the end? – Jonas Dahlbæk Jun 29 '17 at 23:49

I actually think that the book may in fact have the right answer.

My suggested change to the process:

$E(S_n^4) = E(\frac{1}{n^4}[\Sigma(X_i-\bar{X}_n)^4+\frac{1}{n^2}\frac{1}{n^4}\Sigma\Sigma(X_i-\bar{X}_n)^2(X_j-\bar{X_n})^2)$ actually simplifies to $\frac{\hat{\alpha_4}}{n^3} + \frac{\hat{\alpha}_2^2}{n^3}$ which subsequently gets canceled when you find that $E(\bar{X}_n^2)^2 = E((\bar{X}_n+S_n)^2)^2 = \bar{X}_n^4 + 2\bar{X}_n^2\frac{\hat{\alpha}_2}{n} + \frac{\hat{\alpha}_2^2}{n^2}$.

Thus the final result is as the book gives: $$v_{boot} = E(\bar{X}_n^4) - E(\bar{X}_n^2)^2 = \frac{4\bar{X}_n^2\hat{\alpha}_2}{n} + \frac{4\bar{X}_n\hat{\alpha}_3}{n^2} + \frac{\hat{\alpha}_4}{n^3}$$

Hope this clarifies the process. Please submit corrections or suggestions if somehow my reasoning was flawed.

I think AlexK is correct. The term $$\displaystyle \sum_{i\neq j}(X_i^*-\bar X_n)^2(X_j^*-\bar X_n)^2$$ has $$3n(n-1)$$ parts in total (counting duplicates): From 4 factors you can chose 2 in $$4\choose2$$ ways and the remaining 2 in $$2\choose 2$$ ways so the coefficients of the distinct terms are 6. From $$n$$ indices you can choose 2 indices in $$n\choose 2$$ ways. Hence there are $$3n(n-1)$$ terms of this form.