Let $X_{1},...,X_{n}$be distinct observations (no ties). Let $X_{1}^{*},...,X_{n}^{*}$denote a bootstrap sample (a sample from the empirical CDF) and let $\bar{X}_{n}^{*}=\frac{1}{n}\sum_{i=1}^{n}X_{i}^{*}$. Find $E(\bar{X}_{n}^{*})$ and $\mathrm{Var}(\bar{X}_{n}^{*})$.

What I have so far is that $X_{i}^{*}$ is $X_{1},...,X_{n}$ each with probability $\frac{1}{n}$ so $$ E(X_{i}^{*})=\frac{1}{n}E(X_{1})+...+\frac{1}{n}E(X_{n})=\frac{n\mu}{n}=\mu $$ and $$E(X_{i}^{*2})=\frac{1}{n}E(X_{1}^{2})+...+\frac{1}{n}E(X_{n}^{2})=\frac{n(\mu^{2}+\sigma^{2})}{n}=\mu^{2}+\sigma^{2}\>, $$ which gives $$ \mathrm{Var}(X_{i}^{*})=E(X_{i}^{*2})-(E(X_{i}^{*}))^{2}=\mu^{2}+\sigma^{2}-\mu^{2}=\sigma^{2} \>. $$

Then, $$E(\bar{X}_{n}^{*})=E(\frac{1}{n}\sum_{i=1}^{n}X_{i}^{*})=\frac{1}{n}\sum_{i=1}^{n}E(X_{i}^{*})=\frac{n\mu}{n}=\mu $$ and $$ \mathrm{Var}(\bar{X}_{n}^{*})=\mathrm{Var}(\frac{1}{n}\sum_{i=1}^{n}X_{i}^{*})=\frac{1}{n^{2}}\sum_{i=1}^{n}\mathrm{Var}(X_{i}^{*})$$ since the $X_{i}^{*}$'s are independent. This gives $\mathrm{Var}(\bar{X}_{n}^{*})=\frac{n\sigma^{2}}{n^{2}}=\frac{\sigma^{2}}{n}$

However, I don't get the same answer when I condition on $X_{1},\ldots,X_{n}$ and use the formula for conditional variance: $$ \mathrm{Var}(\bar{X}_{n}^{*})=E(\mathrm{Var}(\bar{X}_{n}^{*}|X_{1},...,X_{n}))+\mathrm{Var}(E(\bar{X}_{n}^{*}|X_{1},\ldots,X_{n})) \>. $$

$E(\bar{X}_{n}^{*}|X_{1},\ldots,X_{n})=\bar{X}_{n}$ and $\mathrm{Var}(\bar{X}_{n}^{*}|X_{1},\ldots,X_{n})=\frac{1}{n^{2}}(\sum X_{i}^{2}-n\bar{X}_{n}^{2})$ so plugging these into the formula above gives (after some algebra) $\mathrm{Var}(\bar{X}_{n}^{*})=\frac{(2n-1)\sigma^{2}}{n^{2}}$.

Am I doing something wrong here? My feeling is that I am not using the conditional variance formula correctly but I'm not sure. Any help would be appreciated.

  • $\begingroup$ Maybe your V(E(X|X1..Xn)) is not correctly calculated. The answer should be the same. $\endgroup$ – user35367 Nov 27 '13 at 2:17
  • $\begingroup$ You're probably right--but this answer doesn't seem terribly informative. Perhaps you could point to which part is not correct? $\endgroup$ – whuber Nov 27 '13 at 2:21

The correct answer is $\frac{n-1}{n^2}S^2$. The solution is #4 here


This may be a late answer, but what is wrong in your calculation is the following: you have assumed that unconditionally your bootstrap sample is iid. This is false: conditional on your sample, the bootstrap sample is indeed iid, but unconditionally you lose independence (but you still have identically distributed random variables). This is essentially Exercise 13 in Larry Wasserman All of nonparametric statistics.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.