The data at hand consists of $n$ iid random variables represented as $X_j$, where $j \in \{1,\ldots,n\}$. We know $\forall i,\, \operatorname{E}\left(X_i\right) = \mu$, and that $\operatorname{Var}\left(X_i\right) = \sigma^2$.

Suppose we generate $B$ bootstrap samples from this data, with the $i$th element of the $b$th bootstrap sample denoted by $X_i^{∗b}$.


What are $\operatorname{E}\left(X_i^{∗b}\right)$ and $\operatorname{Var}\left(X_i^{∗b}\right)$?


1 Answer 1


(i) Showing that $\operatorname{E}\left(X_i^{∗b}\right) = \mu$

$X_i^{∗b}$ is "composed" of random variables. To simplify the process of finding $\operatorname{E}\left(X_i^{∗b}\right)$, start with a conditional expectation.

\begin{equation*} \operatorname{E}\left(X_i^{∗b}\,\vert\,X_1,\ldots,X_n\right) = \sum_{j = 1}^{n}\frac{X_j}{n} \end{equation*} This expected value is a random variable itself. Therefore: \begin{equation*} \begin{split} \operatorname{E}\left(X_i^{∗b}\right) &= \operatorname{E}\left[\operatorname{E}\left(X_i^{∗b}\,\vert\,X_1,\ldots, X_n\right)\right]\\ &= \operatorname{E}\left(\sum_{j = 1}^{n}\frac{X_j}{n}\right)\\ & = \frac{1}{n}\sum_{j = 1}^{n}\operatorname{E}\left(X_j\right)\\ & = \frac{1}{n}\sum_{j = 1}^{n}\mu\\ & = \mu.\,\square \end{split} \end{equation*}

(ii) Showing that $\operatorname{Var}\left(X_i^{∗b}\right) = \sigma^2$

Recall that: \begin{equation*} \begin{split} \operatorname{Var}\left(X_i^{∗b}\right) &= \operatorname{E}\left[\left(X_i^{∗b}\right)^2\right] - \left[\operatorname{E}\left(X_i^{∗b}\right)\right]^2\\ &= \operatorname{E}\left[\left(X_i^{∗b}\right)^2\right] - \mu^2. \end{split} \end{equation*} Let us find $\operatorname{E}\left[\left(X_i^{∗b}\right)^2\right]$ in a manner similar to (i). We will need to also apply the ``law of the unconscious statistician'', which states that: \begin{equation*} \operatorname{E}[g(X)] = \sum _{x}g(x)f_{X}(x) \end{equation*} for some function $g$ acting on a random variable $X$. \begin{equation*} \operatorname{E}\left[\left(X_i^{∗b}\right)^2\,\bigg|\,X_1,\ldots,X_n\right] = \sum_{j = 1}^{n}\frac{\left(X_j\right)^2}{n} \end{equation*} \begin{equation*} \begin{split} \implies \operatorname{E}\left[\left(X_i^{∗b}\right)^2\right] &= \operatorname{E}\left\{\operatorname{E}\left[\left(X_i^{∗b}\right)^2\,\bigg|\, X_1,\ldots,X_n\right]\right\}\\ &= \operatorname{E}\left[\sum_{j = 1}^{n}\frac{\left(X_j\right)^2}{n}\right]\\ & = \frac{1}{n}\sum_{j = 1}^{n}\operatorname{E}\left[\left(X_j\right)^2\right]\\ & = \frac{1}{n}\sum_{j = 1}^{n}\left(\sigma^2 + \mu^2\right)\\ & = \sigma^2 + \mu^2 \end{split} \end{equation*} \begin{equation*} \begin{split} \implies \operatorname{Var}\left(X_i^{∗b}\right) &= \sigma^2 + \mu^2 - \mu^2\\ &= \sigma^2.\,\square \end{split} \end{equation*}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.