# correlation of bootstrap sample means

Given a sample $\{x_1,\dots,x_n\}$, $z_1$ and $z_2$ are two bootstrap realizations of sample means, that is, $$z_1 = \frac{1}{n}\sum\{x\in\text{bootstrap sample 1}\}$$ $$z_2 = \frac{1}{n}\sum\{x\in\text{bootstrap sample 2}\}$$ , how to compute the $corr(z_1, z_2)$?

UPDATE

To make sure I understand the problem correctly, here is what I'm trying to solve:

Here is what I tried, as @probabilityislogic suggested, $$z = \frac{1}{n}\sum(x_ik_i) ;k_1+k_2+\cdots+k_n = n$$ , since this is a bootstrap sample, so $x_i$ are all constant and $k_i$ are random variables that $$(k_1,k_2,\cdots,k_n) \sim Mult(k_1k_2\cdots k_n|n, p_1,p_2,\cdots,p_n)$$ . I tried to compute the mean and variance like this, \begin{align*} E(z) &= \sum_k\begin{pmatrix}n\\ k\end{pmatrix}p^kz\\ &= \sum_k\begin{pmatrix}n\\ k\end{pmatrix}p^k(\frac{1}{n}\sum_{i=1}^n x_ik_i)\\ &= \frac{1}{n}\left[\sum_k\begin{pmatrix}n\\ k\end{pmatrix}p^k(x_1k_1+\cdots+x_nk_n)\right]\\ &= \frac{1}{n}[np_1x_1+\cdots+np_nx_n]\\ &= p_1x_1+\cdots+p_nx_n \end{align*} , and since each $k_i$ could be any number in $[0,n]$, then $p_i = \frac{1}{n+1}$, right? If so, $$E(z) = \frac{n}{n+1}\bar x$$ .

To compute covariance, \begin{align*} Cov(z_1, z_2) = \sum_{k_{z1}, k_{z2}}\left[\text{Pr}(k_{z1},k_{z2})(z_1 - E(z))(z_2 - E(z))\right] \end{align*} , where $\text{Pr}(k_{z1}, k_{z2})$ is the joint probability of $k$ for $z_1,z_2$. Am I going the right way here for covariance of $z_1$ and $z_2$?

I can't seem to figure this out here, seems too complicated to me.

• One easier way to figure this out is to define $n_i$ as the number of times unit $i$ is sampled in a given bootstrap replication. Then you have $z_1=\frac{1}{n}\sum_{i=1}^{n}n_ix_i$. And you also have $(n_1,n_2,\dots,n_n)\sim Multinomial(n;\frac{1}{n},\frac{1}{n},\dots,\frac{1}{n})$. Try figuring out the correlation this way (you treat the $x_i$ as fixed values). Jun 9, 2014 at 2:01
• This seems rather like routine bookwork; it's fairly easy to get the covariance out (and from there, the correlation). Is this for some subject? Jun 9, 2014 at 2:24
• @Glen_b, yes, I read it in a book, but no derivation provided in the book, I was considering doing the derivation someway that I'm not sure if it's right, so I post it here in order to get some hint. Jun 9, 2014 at 2:43
• When I did it, I approached it in a similar fashion to that suggested by probabilityislogic; I considered two vector random variables representing the number of repeats of each observation, and computed the covariance matrix (which is simple for a multinomial). From there it's simple to compute the covariance of the sums of the two samples, and from there the covariances of the sample means. Then convert that to a correlation. But there are other ways to achieve the same end. Jun 9, 2014 at 3:14
• Some alternative comments. First, expectation is a linear operator, so you can pull $x_i$s through, and be left with $E k_i$. Second, you are sampling from a finite population $x_1, \ldots, x_n$, so a good sampling book with sufficient explanation of the finite population sampling and inference will have all the necessary results, like expectations and covariances of the multinomial variables. Finally, the problem is not clearly formulated. There are two probability spaces, that of the normal $x$s, and that of the bootstrap samples. Correlation wrt to which field are they asking about? Jun 12, 2014 at 2:47

I used to think the answer should be 0, but I see what I did wrong before. The problem is not looking for the conditioned correlation, so without loss of generality, we can look at the case where $\mu=0,\sigma^2=1$:

Let $x_i^{(k)}$ be sample $i$ from the $k$th bootstrap.

Then $\bar{x}_k^* = \frac{1}{n} \sum_{i}{x_i^{(k)}}$.

Key values we need to compute are:

$E\ x_i^{(k)}=E(E[x_i^{(k)}|\mathbf{x}])=E \bar{x}=0$

$var(x_i^{(k)})=E(var(x_i^{(k)}|\mathbf{x}))+var(E[x_i^{(k)}|\mathbf{x}])=\frac{n-1}{n}+\frac{1}{n}=1$

$cov(x_i^{(1)},x_j^{(2)})=E(x_i^{(1)}-0)(x_j^{(2)}-0)=E(E[x_i^{(1)}x_j^{(2)}|\mathbf{x}])=E\bar{x}^2=\frac{1}{n}$

(same is true for $(x_i^{(k)},x_j^{(k)}) \, i\neq j$)

With above, calculation for the following should be easy:

$cov(\bar{x}_1^*,\bar{x}_2^*)=\frac{1}{n^2} (\sum_{i,j}{cov(x_i^{(1)},x_j^{(2)})})=\frac{1}{n}$

$var(\bar{x}^*_i)=\frac{1}{n^2}\left(n*var(x_1^{(1)})+n(n-1)*cov(x_1^{(1)},x_2^{(1)}) \right)\\ \quad \quad \ \ = \frac{2n-1}{n^2}$

$\rightarrow corr(\bar{x}_1^*,\bar{x}_2^*)=\frac{n}{2n-1}$

• I find it odd that the conditional covariance is 0 but the unconditional covariance is not (after all, covariance is an expectation). Any thoughts? Feb 25, 2022 at 11:18