Expectation and variance of bootstrap samples Background
The data at hand consists of $n$ iid random variables 
represented as $X_j$, where $j \in \{1,\ldots,n\}$. We know $\forall i,\, 
\operatorname{E}\left(X_i\right) = \mu$, and that 
$\operatorname{Var}\left(X_i\right) = \sigma^2$.
Suppose we generate $B$ bootstrap samples from this data, with the $i$th 
element of the $b$th bootstrap sample denoted by $X_i^{∗b}$.
Question
What are $\operatorname{E}\left(X_i^{∗b}\right)$ and $\operatorname{Var}\left(X_i^{∗b}\right)$?
 A: (i) Showing that $\operatorname{E}\left(X_i^{∗b}\right) = \mu$
$X_i^{∗b}$ is "composed" of random variables. To 
simplify the process of finding $\operatorname{E}\left(X_i^{∗b}\right)$, start with a conditional expectation.
\begin{equation*}
\operatorname{E}\left(X_i^{∗b}\,\vert\,X_1,\ldots,X_n\right) = \sum_{j =  
1}^{n}\frac{X_j}{n}
\end{equation*}
This expected value is a random variable itself. Therefore:
\begin{equation*}
\begin{split}
\operatorname{E}\left(X_i^{∗b}\right) &= 
\operatorname{E}\left[\operatorname{E}\left(X_i^{∗b}\,\vert\,X_1,\ldots,
X_n\right)\right]\\ &= \operatorname{E}\left(\sum_{j =  
1}^{n}\frac{X_j}{n}\right)\\
& = \frac{1}{n}\sum_{j =  1}^{n}\operatorname{E}\left(X_j\right)\\
& = \frac{1}{n}\sum_{j =  1}^{n}\mu\\
& = \mu.\,\square
\end{split}
\end{equation*}
(ii) Showing that $\operatorname{Var}\left(X_i^{∗b}\right) = 
\sigma^2$
Recall that:
\begin{equation*}
\begin{split}
\operatorname{Var}\left(X_i^{∗b}\right) &= 
\operatorname{E}\left[\left(X_i^{∗b}\right)^2\right] - 
\left[\operatorname{E}\left(X_i^{∗b}\right)\right]^2\\ &= 
\operatorname{E}\left[\left(X_i^{∗b}\right)^2\right] - \mu^2.
\end{split}
\end{equation*}
Let us find $\operatorname{E}\left[\left(X_i^{∗b}\right)^2\right]$ in a manner similar to (i). We will need to also apply the ``law of the unconscious statistician'', which states that: \begin{equation*}
\operatorname{E}[g(X)] = \sum _{x}g(x)f_{X}(x)
\end{equation*}
for some function $g$ acting on a random variable $X$.
\begin{equation*}
\operatorname{E}\left[\left(X_i^{∗b}\right)^2\,\bigg|\,X_1,\ldots,X_n\right] = 
\sum_{j =  1}^{n}\frac{\left(X_j\right)^2}{n}
\end{equation*}
\begin{equation*}
\begin{split}
\implies \operatorname{E}\left[\left(X_i^{∗b}\right)^2\right] &= 
\operatorname{E}\left\{\operatorname{E}\left[\left(X_i^{∗b}\right)^2\,\bigg|\,
X_1,\ldots,X_n\right]\right\}\\
&= \operatorname{E}\left[\sum_{j =  1}^{n}\frac{\left(X_j\right)^2}{n}\right]\\
& = \frac{1}{n}\sum_{j =  
1}^{n}\operatorname{E}\left[\left(X_j\right)^2\right]\\
& = \frac{1}{n}\sum_{j =  1}^{n}\left(\sigma^2 + \mu^2\right)\\
& = \sigma^2 + \mu^2
\end{split}
\end{equation*}
\begin{equation*}
\begin{split}
\implies \operatorname{Var}\left(X_i^{∗b}\right) &= \sigma^2 + \mu^2 - \mu^2\\
&= \sigma^2.\,\square
\end{split}
\end{equation*}
