I collected data from 3 rats ($r_1, r_2,r_3$). Each rat was put under two treatment conditions. Under each condition, from each rat, I collected responses of many brain cells (neurons, $n = 150-200$) simultaneously. From the responses of all the neurons of a given rat, I compute a single number ($Z$). I can't compute $Z$ for each neuron; I have to combine the measurement of all neurons to get it. I want to test if the treatment causes any significant difference in $Z$ in rats. What is the correct bootstrapping procedure?

I have the following in mind:

Method 1:

Resample the 3 rats with replacement (eg. [$r_1, r_1, r_3]$). From each selected rat, sample the neurons with replacement (eg. [$n_1, n_2, n_2, n_{55}, \ldots$]). Compute $Z$ for the two treatments ($Z_1$ & $Z_2$) using these selected neurons. Compute the difference $D = Z_1-Z_2$. Repeat this procedure for the other two rats. Take the mean of the three $D$ values ($D_m$). Repeat this entire procedure 2000 times to get the distribution of $D_m$. Compute significance using $p$-value = $2\times min(q,1-q)$ where $q$ is the percentile value of zero in the bootstrap distribution of $D_m$.

Method 2:

Without any resampling of rats, for every bootstrap iteration, we will start with the same original 3 rats and perform all the other steps mentioned for Method 1.

Your help is greatly appreciated Thanks MS

  • $\begingroup$ 1) Your formula D = D1-D2 should be D = Z1-Z2? 2) How you select number of repeation your procedure (2000)? $\endgroup$
    – Nick
    Sep 9, 2016 at 0:13
  • $\begingroup$ You are right. It should be Z1-Z2. I fixed it. As for the the number of replications, a colleague who works on related problems said 2000 is OK. I guess this number varies depending on the what is computed in each replication. See Section 6.4 "The number of bootstrap replications B" in Efron & Tibshirani's book "An Introduction to the Bootstrap". I have tried to increase the replications and the shape of the bootstrap distribution doesn't change noticeably; so I stayed with 2000. $\endgroup$
    – user130451
    Sep 9, 2016 at 0:46

1 Answer 1


If the results are consistent across rats, both methods should work equally well. However, if your test statistic $D$ varies a lot across rats, then resampling (your Method 1) will give you a distribution that captures the variability across rats. The graphic below is fairly self explanatory and should help you understand why you might want to resample.

enter image description here

One problem I see with both methods is that they don't take differences in the number of observations across rats into account. For example, suppose you recorded twice as many neurons from $r_1$ compared to $r_2$, you want your test statistic to give more weight to data from $r_1$. So I suggest you modify your mean test statistic by weighting the difference $D$ in each rat by the number of neurons you recorded from that rat. Something like:

$$D_m = \frac{\sum_in_iD_i}{\sum_in_i}$$

where $n_i$ and $D_i$ denote the number of neurons and the value of the test statistic from the $i^{th}$ rat.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.