Given a sample $\{x_1,\dots,x_n\}$, $z_1$ and $z_2$ are two bootstrap realizations of sample means, that is, $$z_1 = \frac{1}{n}\sum\{x\in\text{bootstrap sample 1}\}$$ $$z_2 = \frac{1}{n}\sum\{x\in\text{bootstrap sample 2}\}$$ , how to compute the $corr(z_1, z_2)$?


To make sure I understand the problem correctly, here is what I'm trying to solve:

enter image description here

Here is what I tried, as @probabilityislogic suggested, $$z = \frac{1}{n}\sum(x_ik_i) ;k_1+k_2+\cdots+k_n = n$$ , since this is a bootstrap sample, so $x_i$ are all constant and $k_i$ are random variables that $$(k_1,k_2,\cdots,k_n) \sim Mult(k_1k_2\cdots k_n|n, p_1,p_2,\cdots,p_n)$$ . I tried to compute the mean and variance like this, \begin{align*} E(z) &= \sum_k\begin{pmatrix}n\\ k\end{pmatrix}p^kz\\ &= \sum_k\begin{pmatrix}n\\ k\end{pmatrix}p^k(\frac{1}{n}\sum_{i=1}^n x_ik_i)\\ &= \frac{1}{n}\left[\sum_k\begin{pmatrix}n\\ k\end{pmatrix}p^k(x_1k_1+\cdots+x_nk_n)\right]\\ &= \frac{1}{n}[np_1x_1+\cdots+np_nx_n]\\ &= p_1x_1+\cdots+p_nx_n \end{align*} , and since each $k_i$ could be any number in $[0,n]$, then $p_i = \frac{1}{n+1}$, right? If so, $$E(z) = \frac{n}{n+1}\bar x$$ .

To compute covariance, \begin{align*} Cov(z_1, z_2) = \sum_{k_{z1}, k_{z2}}\left[\text{Pr}(k_{z1},k_{z2})(z_1 - E(z))(z_2 - E(z))\right] \end{align*} , where $\text{Pr}(k_{z1}, k_{z2})$ is the joint probability of $k$ for $z_1,z_2$. Am I going the right way here for covariance of $z_1$ and $z_2$?

I can't seem to figure this out here, seems too complicated to me.

  • $\begingroup$ One easier way to figure this out is to define $n_i$ as the number of times unit $i$ is sampled in a given bootstrap replication. Then you have $z_1=\frac{1}{n}\sum_{i=1}^{n}n_ix_i$. And you also have $(n_1,n_2,\dots,n_n)\sim Multinomial(n;\frac{1}{n},\frac{1}{n},\dots,\frac{1}{n})$. Try figuring out the correlation this way (you treat the $x_i$ as fixed values). $\endgroup$ Jun 9, 2014 at 2:01
  • $\begingroup$ This seems rather like routine bookwork; it's fairly easy to get the covariance out (and from there, the correlation). Is this for some subject? $\endgroup$
    – Glen_b
    Jun 9, 2014 at 2:24
  • $\begingroup$ @Glen_b, yes, I read it in a book, but no derivation provided in the book, I was considering doing the derivation someway that I'm not sure if it's right, so I post it here in order to get some hint. $\endgroup$
    – avocado
    Jun 9, 2014 at 2:43
  • $\begingroup$ When I did it, I approached it in a similar fashion to that suggested by probabilityislogic; I considered two vector random variables representing the number of repeats of each observation, and computed the covariance matrix (which is simple for a multinomial). From there it's simple to compute the covariance of the sums of the two samples, and from there the covariances of the sample means. Then convert that to a correlation. But there are other ways to achieve the same end. $\endgroup$
    – Glen_b
    Jun 9, 2014 at 3:14
  • 1
    $\begingroup$ Some alternative comments. First, expectation is a linear operator, so you can pull $x_i$s through, and be left with $E k_i$. Second, you are sampling from a finite population $x_1, \ldots, x_n$, so a good sampling book with sufficient explanation of the finite population sampling and inference will have all the necessary results, like expectations and covariances of the multinomial variables. Finally, the problem is not clearly formulated. There are two probability spaces, that of the normal $x$s, and that of the bootstrap samples. Correlation wrt to which field are they asking about? $\endgroup$
    – StasK
    Jun 12, 2014 at 2:47

1 Answer 1


I used to think the answer should be 0, but I see what I did wrong before. The problem is not looking for the conditioned correlation, so without loss of generality, we can look at the case where $\mu=0,\sigma^2=1$:

Let $x_i^{(k)}$ be sample $i$ from the $k$th bootstrap.

Then $\bar{x}_k^* = \frac{1}{n} \sum_{i}{x_i^{(k)}}$.

Key values we need to compute are:

$E\ x_i^{(k)}=E(E[x_i^{(k)}|\mathbf{x}])=E \bar{x}=0$



(same is true for $(x_i^{(k)},x_j^{(k)}) \, i\neq j$)

With above, calculation for the following should be easy:

$cov(\bar{x}_1^*,\bar{x}_2^*)=\frac{1}{n^2} (\sum_{i,j}{cov(x_i^{(1)},x_j^{(2)})})=\frac{1}{n}$

$var(\bar{x}^*_i)=\frac{1}{n^2}\left(n*var(x_1^{(1)})+n(n-1)*cov(x_1^{(1)},x_2^{(1)}) \right)\\ \quad \quad \ \ = \frac{2n-1}{n^2}$

$\rightarrow corr(\bar{x}_1^*,\bar{x}_2^*)=\frac{n}{2n-1}$

  • $\begingroup$ I find it odd that the conditional covariance is 0 but the unconditional covariance is not (after all, covariance is an expectation). Any thoughts? $\endgroup$
    – Arshdeep
    Feb 25, 2022 at 11:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.