# Bootstrap variance of squared sample mean

The following is question 8 of chapter 8 in Wasserman's All of Statistics:

Let $T_n = \overline{X}_n^2$, $\mu = \mathbb{E}(X_1)$, $\alpha_k = \int|x - \mu|^kdF(x)$, and $\hat{\alpha}_k = n^{-1}\sum_{i=1}^n|X_i - \overline{X}_n|^k$.

Show that $$v_{\mathrm{boot}} = \frac{4\overline{X}_n^2\hat{\alpha}_2}{n} + \frac{4\overline{X}_n\hat{\alpha}_3}{n^2} + \frac{\hat{\alpha}_4}{n^3} \>.$$

He previously defines $v_{\mathrm{boot}} = \frac{1}{B}\sum_{b=1}^B(T_{n,b}^* - \frac{1}{B}\sum_{r=1}^BT_{n,r}^*)^2$, where $T_{n,i}^*$ is the desired statistic from the $i$th bootstrap replication of the sample $X_1,...,X_n$.

It seems that the question as stated does not make sense: how can there be a formula for the bootstrap variance if the quantity requires simulation? Perhaps he meant to ask for the variance of the sampling distribution, but I get $\frac{\sigma^2}{n}$ for that. Any hints on how to intepret or solve this?

• Hi Alex, welcome to the site. Here is what I think the question is asking: Find $v_{\mathrm{boot}}$ which is the variance under the (empirical) measure $\hat F_n$. The "previously defined" version is simply the Monte Carlo estimate of $v_{\mathrm{boot}}$ rather than the quantity itself. Even so, I think what you'll find is that there are at least two other typos lurking: (1) The definition of $\hat \alpha_k$ should probably not include the modulus and (2) I believe the last term on the right-hand size is not quite correct. :) Commented Apr 8, 2012 at 19:20
• (When I looked on that page of the book, I saw what I think was at least one other typo in the question before that as well. Also, this same problem, with the same [conjectured] errors appears to be reproduced in Wasserman's All of Nonparametric Statistics as well, on page 39. Once you've completed the exercise, you might consider sending a note to the author so that he can add it to the errata.) Commented Apr 8, 2012 at 19:23
• Alex, it is important to understand that the bootstrap does not require simulation. The two concepts are separable: the bootstrap is a well-defined statistic, equal to a complicated function of the sample. In some cases (as in this situation), the complicated function simplifies greatly and has a closed form in terms of easily computed quantities: no simulation or actual resampling are needed. However, in many practical cases it is easier to approximate the bootstrap with a simulation rather than work out a computationally simple expression (if one even exists).
– whuber
Commented Apr 9, 2012 at 15:08
• @whuber and cardinal, thank you each for the explanations, I understand much better now. I believe I've worked out the question (the last term is indeed different) and will notify the author once I type it up. Is it accepted to post the solution here as well? Finally, how can I credit you for the help? Commented Apr 10, 2012 at 18:39
• Hi AlexK. Yes, please do post your solution as an answer! I was hoping this would be the outcome. I do not want to speak for @whuber too much (though I doubt he'll mind in this instance), but do not be concerned with "crediting" us. I, for one, am happy to see you've arrived at a positive result and have benefitted from the site. I hope you'll continue to frequent it and participate. Cheers. Commented Apr 15, 2012 at 2:27

A little late, but anyways... First, to simplify later calculations, rewrite the sample mean in terms of an expression containing the central moments under the empirical measure. Let $S_n = \frac{1}{n}\sum(X_i - \bar{X}_n) = 0$. Then $$\bar{X}_n = S_n +\bar{X}_n = \frac{1}{n}\sum (X_i - \bar{X}_n) + \bar{X}_n$$ Now, Var$(\bar{X}_n^2) = E(\bar{X}_n^4) - (E\bar{X}_n^2)^2$. We'll tackle the first term. Note that $\bar{X}_n$ is the mean under the empirical measure, so we treat it as a constant when taking expectations. \begin{align} E(\bar{X}_n^4) &= E(S_n + \bar{X}_n)^4 \\ &= E(S_n^4 + 4\bar{X}_nS_n^3 + 6\bar{X}_n^2S_n^2 + 4\bar{X}_n^3S_n + \bar{X}_n^4) \\ &= E(S_n^4) + 4\bar{X}_nE(S_n^3) + 6\bar{X}_n^2E(S_n^2) + \bar{X}_n^4 \end{align} where we used that $S_n = 0$ to drop the second-to-last term. In the following expansions, terms involving a product with $nS_n$ will not be written. \begin{align} E(S_n^4) &= E\left(\frac{1}{n^4}\left[\sum(X_i - \bar{X}_n)^4 + \sum\sum(X_i - \bar{X}_n)^2(X_j - \bar{X}_n)^2\right]\right) \\ &= \frac{\hat{a}_4}{n^3} + \frac{3(n-1)\hat{a}_2^2}{n^3}\\ E(S_n^3) &= \left(\frac{1}{n^3}\sum(X_i - \bar{X}_n)^3\right) = \frac{\hat{a}_3}{n^2}\\ E(S_n^2) &= \left(\frac{1}{n^2}\sum(X_i - \bar{X}_n)^2\right) = \frac{\hat{a}_2}{n} \end{align} These are straightforward sums of products with some combinatorics to count the number of terms. Doing similar calculations for the second term of the variance and putting it all together: $$Var(\bar{X}_n^2) = \frac{4\bar{X}_n^2\hat{a}_2}{n} + \frac{4\bar{X}_n\hat{a}_3}{n^2} + \frac{\hat{a}_4 + (2n - 3)\hat{a}_2^2}{n^3}$$

• Shouldn't it rather be $$E(S_n^4)=E(\frac{1}{n^4}[\sum_{i=1}^n(X_i^*-\bar X_n)^4 + \sum_{i\neq j}(X_i^*-\bar X_n)^2(X_j^*-\bar X_n)^2])=\frac{\hat \alpha_4}{n^3} + \frac{(n-1)\hat\alpha_2^2}{n^3},$$ which leads to the corrected term $\frac{\hat\alpha_4 - \hat\alpha_2^2}{n^3}$ in the end? Commented Jun 29, 2017 at 23:49

I actually think that the book may in fact have the right answer.

My suggested change to the process:

$E(S_n^4) = E(\frac{1}{n^4}[\Sigma(X_i-\bar{X}_n)^4+\frac{1}{n^2}\frac{1}{n^4}\Sigma\Sigma(X_i-\bar{X}_n)^2(X_j-\bar{X_n})^2)$ actually simplifies to $\frac{\hat{\alpha_4}}{n^3} + \frac{\hat{\alpha}_2^2}{n^3}$ which subsequently gets canceled when you find that $E(\bar{X}_n^2)^2 = E((\bar{X}_n+S_n)^2)^2 = \bar{X}_n^4 + 2\bar{X}_n^2\frac{\hat{\alpha}_2}{n} + \frac{\hat{\alpha}_2^2}{n^2}$.

Thus the final result is as the book gives: $$v_{boot} = E(\bar{X}_n^4) - E(\bar{X}_n^2)^2 = \frac{4\bar{X}_n^2\hat{\alpha}_2}{n} + \frac{4\bar{X}_n\hat{\alpha}_3}{n^2} + \frac{\hat{\alpha}_4}{n^3}$$

Hope this clarifies the process. Please submit corrections or suggestions if somehow my reasoning was flawed.

I think AlexK is correct. The term $$\displaystyle \sum_{i\neq j}(X_i^*-\bar X_n)^2(X_j^*-\bar X_n)^2$$ has $$3n(n-1)$$ parts in total (counting duplicates): From 4 factors you can chose 2 in $$4\choose2$$ ways and the remaining 2 in $$2\choose 2$$ ways so the coefficients of the distinct terms are 6. From $$n$$ indices you can choose 2 indices in $$n\choose 2$$ ways. Hence there are $$3n(n-1)$$ terms of this form.

Although the idea behind the procedure proposed by AlexK and others is in the good direction, there are some key points that should be considered if you want to be strictly correct in this.

• You cannot arbitrarily decide over which statistic can or can't be considered to be "treated it as a constant when taking expectations" (As it happens with $$\overline{X_n}$$ respect to $$S_n$$).
• In the process you are also taking an expectation over an statistic that you know is defined as zero and still expect get something from it (in this case $$S_n$$, $$S_n^2$$,...).
• In general, the process of getting an estimator consist in first getting an expression for the statistic, as for example $$\mathbb{V}(\overline{X_n})=\sigma^2/n$$, and then transform it into an estimator, $$\rightarrow \mathbb{\hat{V}}(\overline{X_n})=\hat{\sigma}^2/n$$.

You can obtain an expression of this kind (with the desired form requested from the exercise) using

$$\mathbb{V}(\overline{X_n}^2)=\mathbb{E}\left(\overline{X_n} - \mu + \mu\right)^4-\mathbb{E}^2\left(\overline{X_n} - \mu + \mu\right)^2$$

From this point, the procedure is quite similar to the proposed by AlexK, which requires taking $$\mu$$ instead of $$\overline{X_n}$$ and $$\overline{X_n} - \mu$$ instead of $$S_n$$. In this case $$\mu$$ is allowed to be taken out from expectation. Then

\begin{align} \mathbb{E}(\overline{X_n}^4) =& \mathbb{E}\left(\overline{X_n} - \mu + \mu\right)^4\\ =& \mathbb{E}\left(\overline{X_n} - \mu\right)^4 + 4 \mu \mathbb{E}\left(\overline{X_n} - \mu\right)^3 + 6\mu^2 \mathbb{E}\left(\overline{X_n} - \mu\right)^2 + 4\mu^3 \underbrace{\mathbb{E}\left(\overline{X_n} - \mu\right)}_{=0} + \mu^4 \end{align}

and

\begin{align} \mathbb{E}^2(\overline{X_n}^2) =& \mathbb{E}^2\left(\overline{X_n} - \mu + \mu\right)^2\\ =& \mathbb{E}^2\left(\overline{X_n} - \mu\right)^2+ 2\mu^2 \mathbb{E}\left(\overline{X_n} - \mu\right)^2 + \mu^4 \end{align} where $$\mathbb{E}\left(\overline{X_n} - \mu\right)^2 = \frac{1}{n} \alpha_2$$ $$\mathbb{E}\left(\overline{X_n} - \mu\right)^3 = \frac{1}{n^2} \alpha_3$$ $$\mathbb{E}\left(\overline{X_n} - \mu\right)^4 = \frac{1}{n^3} \left[\alpha_4 + 3 (n-1) \alpha_2^2\right]$$ All this gives as result $$\mathbb{V}(\overline{X_n}^2)= \frac{1}{n^3} \left[\alpha_4 + \left(2n-3\right) \alpha_2^2\right] + 4 \mu \frac{1}{n^2} \alpha_3+ 4\mu^2 \frac{1}{n} \alpha_2$$ which then can be used as ansatz for the estimator $$\mathbb{\hat{V}}(\overline{X_n}^2)= \frac{1}{n^3} \left[\hat{\alpha}_4 + \left(2n-3\right) \hat{\alpha_2}^2\right] + 4 \overline{X_n} \frac{1}{n^2} \hat{\alpha_3}+ 4\overline{X_n}^2 \frac{1}{n} \hat{\alpha_2}$$

Notice that in this solution I skipped the summation and combinatorial calculations, as it has been already discussed here. Also notice that I used the definition $$\mathbb{E}\left(X_i - \mu\right)^k = \alpha_k$$.