Given a sample $\{x_1,\dots,x_n\}$, $z_1$ and $z_2$ are two bootstrap realizations of sample means, that is, $$z_1 = \frac{1}{n}\sum\{x\in\text{bootstrap sample 1}\}$$ $$z_2 = \frac{1}{n}\sum\{x\in\text{bootstrap sample 2}\}$$ , how to compute the $corr(z_1, z_2)$?


To make sure I understand the problem correctly, here is what I'm trying to solve:

enter image description here

Here is what I tried, as @probabilityislogic suggested, $$z = \frac{1}{n}\sum(x_ik_i) ;k_1+k_2+\cdots+k_n = n$$ , since this is a bootstrap sample, so $x_i$ are all constant and $k_i$ are random variables that $$(k_1,k_2,\cdots,k_n) \sim Mult(k_1k_2\cdots k_n|n, p_1,p_2,\cdots,p_n)$$ . I tried to compute the mean and variance like this, \begin{align*} E(z) &= \sum_k\begin{pmatrix}n\\ k\end{pmatrix}p^kz\\ &= \sum_k\begin{pmatrix}n\\ k\end{pmatrix}p^k(\frac{1}{n}\sum_{i=1}^n x_ik_i)\\ &= \frac{1}{n}\left[\sum_k\begin{pmatrix}n\\ k\end{pmatrix}p^k(x_1k_1+\cdots+x_nk_n)\right]\\ &= \frac{1}{n}[np_1x_1+\cdots+np_nx_n]\\ &= p_1x_1+\cdots+p_nx_n \end{align*} , and since each $k_i$ could be any number in $[0,n]$, then $p_i = \frac{1}{n+1}$, right? If so, $$E(z) = \frac{n}{n+1}\bar x$$ .

To compute covariance, \begin{align*} Cov(z_1, z_2) = \sum_{k_{z1}, k_{z2}}\left[\text{Pr}(k_{z1},k_{z2})(z_1 - E(z))(z_2 - E(z))\right] \end{align*} , where $\text{Pr}(k_{z1}, k_{z2})$ is the joint probability of $k$ for $z_1,z_2$. Am I going the right way here for covariance of $z_1$ and $z_2$?

I can't seem to figure this out here, seems too complicated to me.

  • $\begingroup$ One easier way to figure this out is to define $n_i$ as the number of times unit $i$ is sampled in a given bootstrap replication. Then you have $z_1=\frac{1}{n}\sum_{i=1}^{n}n_ix_i$. And you also have $(n_1,n_2,\dots,n_n)\sim Multinomial(n;\frac{1}{n},\frac{1}{n},\dots,\frac{1}{n})$. Try figuring out the correlation this way (you treat the $x_i$ as fixed values). $\endgroup$ – probabilityislogic Jun 9 '14 at 2:01
  • $\begingroup$ This seems rather like routine bookwork; it's fairly easy to get the covariance out (and from there, the correlation). Is this for some subject? $\endgroup$ – Glen_b Jun 9 '14 at 2:24
  • $\begingroup$ @Glen_b, yes, I read it in a book, but no derivation provided in the book, I was considering doing the derivation someway that I'm not sure if it's right, so I post it here in order to get some hint. $\endgroup$ – avocado Jun 9 '14 at 2:43
  • $\begingroup$ When I did it, I approached it in a similar fashion to that suggested by probabilityislogic; I considered two vector random variables representing the number of repeats of each observation, and computed the covariance matrix (which is simple for a multinomial). From there it's simple to compute the covariance of the sums of the two samples, and from there the covariances of the sample means. Then convert that to a correlation. But there are other ways to achieve the same end. $\endgroup$ – Glen_b Jun 9 '14 at 3:14
  • 1
    $\begingroup$ Some alternative comments. First, expectation is a linear operator, so you can pull $x_i$s through, and be left with $E k_i$. Second, you are sampling from a finite population $x_1, \ldots, x_n$, so a good sampling book with sufficient explanation of the finite population sampling and inference will have all the necessary results, like expectations and covariances of the multinomial variables. Finally, the problem is not clearly formulated. There are two probability spaces, that of the normal $x$s, and that of the bootstrap samples. Correlation wrt to which field are they asking about? $\endgroup$ – StasK Jun 12 '14 at 2:47

I used to think the answer should be 0, but I see what I did wrong before. The problem is not looking for the conditioned correlation, so without loss of generality, we can look at the case where $\mu=0,\sigma^2=1$:

Let $x_i^{(k)}$ be sample $i$ from the $k$th bootstrap.

Then $\bar{x}_k^* = \frac{1}{n} \sum_{i}{x_i^{(k)}}$.

Key values we need to compute are:

$E\ x_i^{(k)}=E(E[x_i^{(k)}|\mathbf{x}])=E \bar{x}=0$



(same is true for $(x_i^{(k)},x_j^{(k)}) \, i\neq j$)

With above, calculation for the following should be easy:

$cov(\bar{x}_1^*,\bar{x}_2^*)=\frac{1}{n^2} (\sum_{i,j}{cov(x_i^{(1)},x_j^{(2)})})=\frac{1}{n}$

$var(\bar{x}^*_i)=\frac{1}{n^2}\left(n*var(x_1^{(1)})+n(n-1)*cov(x_1^{(1)},x_2^{(1)}) \right)\\ \quad \quad \ \ = \frac{2n-1}{n^2}$

$\rightarrow corr(\bar{x}_1^*,\bar{x}_2^*)=\frac{n}{2n-1}$

  • 1
    $\begingroup$ This is more of a comment than an answer to the question. With a bit more reputation, you will be able to post comments. For the moment I've added the comment for you, and I'm flagging this post for deletion. If you'd like to expand this into answer instead, you can edit it from this same account and then flag it for moderator attention and ask that we restore it. If you have a new question, please ask it by clicking the Ask Question button. $\endgroup$ – Glen_b Aug 7 '15 at 21:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.