5
$\begingroup$

The following is question 8 of chapter 8 in Wasserman's All of Statistics:

Let $T_n = \overline{X}_n^2$, $\mu = \mathbb{E}(X_1)$, $\alpha_k = \int|x - \mu|^kdF(x)$, and $\hat{\alpha}_k = n^{-1}\sum_{i=1}^n|X_i - \overline{X}_n|^k$.

Show that $$v_{\mathrm{boot}} = \frac{4\overline{X}_n^2\hat{\alpha}_2}{n} + \frac{4\overline{X}_n\hat{\alpha}_3}{n^2} + \frac{\hat{\alpha}_4}{n^3} \>.$$

He previously defines $v_{\mathrm{boot}} = \frac{1}{B}\sum_{b=1}^B(T_{n,b}^* - \frac{1}{B}\sum_{r=1}^BT_{n,r}^*)^2$, where $T_{n,i}^*$ is the desired statistic from the $i$th bootstrap replication of the sample $X_1,...,X_n$.

It seems that the question as stated does not make sense: how can there be a formula for the bootstrap variance if the quantity requires simulation? Perhaps he meant to ask for the variance of the sampling distribution, but I get $\frac{\sigma^2}{n}$ for that. Any hints on how to intepret or solve this?

$\endgroup$
  • 1
    $\begingroup$ Hi Alex, welcome to the site. Here is what I think the question is asking: Find $v_{\mathrm{boot}}$ which is the variance under the (empirical) measure $\hat F_n$. The "previously defined" version is simply the Monte Carlo estimate of $v_{\mathrm{boot}}$ rather than the quantity itself. Even so, I think what you'll find is that there are at least two other typos lurking: (1) The definition of $\hat \alpha_k$ should probably not include the modulus and (2) I believe the last term on the right-hand size is not quite correct. :) $\endgroup$ – cardinal Apr 8 '12 at 19:20
  • 1
    $\begingroup$ (When I looked on that page of the book, I saw what I think was at least one other typo in the question before that as well. Also, this same problem, with the same [conjectured] errors appears to be reproduced in Wasserman's All of Nonparametric Statistics as well, on page 39. Once you've completed the exercise, you might consider sending a note to the author so that he can add it to the errata.) $\endgroup$ – cardinal Apr 8 '12 at 19:23
  • 2
    $\begingroup$ Alex, it is important to understand that the bootstrap does not require simulation. The two concepts are separable: the bootstrap is a well-defined statistic, equal to a complicated function of the sample. In some cases (as in this situation), the complicated function simplifies greatly and has a closed form in terms of easily computed quantities: no simulation or actual resampling are needed. However, in many practical cases it is easier to approximate the bootstrap with a simulation rather than work out a computationally simple expression (if one even exists). $\endgroup$ – whuber Apr 9 '12 at 15:08
  • 1
    $\begingroup$ @whuber and cardinal, thank you each for the explanations, I understand much better now. I believe I've worked out the question (the last term is indeed different) and will notify the author once I type it up. Is it accepted to post the solution here as well? Finally, how can I credit you for the help? $\endgroup$ – AlexK Apr 10 '12 at 18:39
  • 2
    $\begingroup$ Hi AlexK. Yes, please do post your solution as an answer! I was hoping this would be the outcome. I do not want to speak for @whuber too much (though I doubt he'll mind in this instance), but do not be concerned with "crediting" us. I, for one, am happy to see you've arrived at a positive result and have benefitted from the site. I hope you'll continue to frequent it and participate. Cheers. $\endgroup$ – cardinal Apr 15 '12 at 2:27
2
$\begingroup$

A little late, but anyways... First, to simplify later calculations, rewrite the sample mean in terms of an expression containing the central moments under the empirical measure. Let $S_n = \frac{1}{n}\sum(X_i - \bar{X}_n) = 0$. Then $$ \bar{X}_n = S_n +\bar{X}_n = \frac{1}{n}\sum (X_i - \bar{X}_n) + \bar{X}_n $$ Now, Var$(\bar{X}_n^2) = E(\bar{X}_n^4) - (E\bar{X}_n^2)^2$. We'll tackle the first term. Note that $\bar{X}_n$ is the mean under the empirical measure, so we treat it as a constant when taking expectations. $$ \begin{align} E(\bar{X}_n^4) &= E(S_n + \bar{X}_n)^4 \\ &= E(S_n^4 + 4\bar{X}_nS_n^3 + 6\bar{X}_n^2S_n^2 + 4\bar{X}_n^3S_n + \bar{X}_n^4) \\ &= E(S_n^4) + 4\bar{X}_nE(S_n^3) + 6\bar{X}_n^2E(S_n^2) + \bar{X}_n^4 \end{align} $$ where we used that $S_n = 0$ to drop the second-to-last term. In the following expansions, terms involving a product with $nS_n$ will not be written. $$ \begin{align} E(S_n^4) &= E\left(\frac{1}{n^4}\left[\sum(X_i - \bar{X}_n)^4 + \sum\sum(X_i - \bar{X}_n)^2(X_j - \bar{X}_n)^2\right]\right) \\ &= \frac{\hat{a}_4}{n^3} + \frac{3(n-1)\hat{a}_2^2}{n^3}\\ E(S_n^3) &= \left(\frac{1}{n^3}\sum(X_i - \bar{X}_n)^3\right) = \frac{\hat{a}_3}{n^2}\\ E(S_n^2) &= \left(\frac{1}{n^2}\sum(X_i - \bar{X}_n)^2\right) = \frac{\hat{a}_2}{n} \end{align} $$ These are straightforward sums of products with some combinatorics to count the number of terms. Doing similar calculations for the second term of the variance and putting it all together: $$ Var(\bar{X}_n^2) = \frac{4\bar{X}_n^2\hat{a}_2}{n} + \frac{4\bar{X}_n\hat{a}_3}{n^2} + \frac{\hat{a}_4 + (2n - 3)\hat{a}_2^2}{n^3} $$

$\endgroup$
  • $\begingroup$ Shouldn't it rather be $$E(S_n^4)=E(\frac{1}{n^4}[\sum_{i=1}^n(X_i^*-\bar X_n)^4 + \sum_{i\neq j}(X_i^*-\bar X_n)^2(X_j^*-\bar X_n)^2])=\frac{\hat \alpha_4}{n^3} + \frac{(n-1)\hat\alpha_2^2}{n^3},$$ which leads to the corrected term $\frac{\hat\alpha_4 - \hat\alpha_2^2}{n^3}$ in the end? $\endgroup$ – Jonas Dahlbæk Jun 29 '17 at 23:49
2
$\begingroup$

I actually think that the book may in fact have the right answer.

My suggested change to the process:

$E(S_n^4) = E(\frac{1}{n^4}[\Sigma(X_i-\bar{X}_n)^4+\frac{1}{n^2}\frac{1}{n^4}\Sigma\Sigma(X_i-\bar{X}_n)^2(X_j-\bar{X_n})^2)$ actually simplifies to $\frac{\hat{\alpha_4}}{n^3} + \frac{\hat{\alpha}_2^2}{n^3}$ which subsequently gets canceled when you find that $E(\bar{X}_n^2)^2 = E((\bar{X}_n+S_n)^2)^2 = \bar{X}_n^4 + 2\bar{X}_n^2\frac{\hat{\alpha}_2}{n} + \frac{\hat{\alpha}_2^2}{n^2}$.

Thus the final result is as the book gives: $$v_{boot} = E(\bar{X}_n^4) - E(\bar{X}_n^2)^2 = \frac{4\bar{X}_n^2\hat{\alpha}_2}{n} + \frac{4\bar{X}_n\hat{\alpha}_3}{n^2} + \frac{\hat{\alpha}_4}{n^3}$$

Hope this clarifies the process. Please submit corrections or suggestions if somehow my reasoning was flawed.

$\endgroup$
0
$\begingroup$

I think AlexK is correct. The term $\displaystyle \sum_{i\neq j}(X_i^*-\bar X_n)^2(X_j^*-\bar X_n)^2$ has $3n(n-1)$ parts in total (counting duplicates): From 4 factors you can chose 2 in $4\choose2$ ways and the remaining 2 in $2\choose 2$ ways so the coefficients of the distinct terms are 6. From $n$ indices you can choose 2 indices in $n\choose 2$ ways. Hence there are $3n(n-1)$ terms of this form.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.