# Bootstrap - resampling - two levels - small number of subjects (n = 3)

I collected data from 3 rats ($r_1, r_2,r_3$). Each rat was put under two treatment conditions. Under each condition, from each rat, I collected responses of many brain cells (neurons, $n = 150-200$) simultaneously. From the responses of all the neurons of a given rat, I compute a single number ($Z$). I can't compute $Z$ for each neuron; I have to combine the measurement of all neurons to get it. I want to test if the treatment causes any significant difference in $Z$ in rats. What is the correct bootstrapping procedure?

I have the following in mind:

Method 1:

Resample the 3 rats with replacement (eg. [$r_1, r_1, r_3]$). From each selected rat, sample the neurons with replacement (eg. [$n_1, n_2, n_2, n_{55}, \ldots$]). Compute $Z$ for the two treatments ($Z_1$ & $Z_2$) using these selected neurons. Compute the difference $D = Z_1-Z_2$. Repeat this procedure for the other two rats. Take the mean of the three $D$ values ($D_m$). Repeat this entire procedure 2000 times to get the distribution of $D_m$. Compute significance using $p$-value = $2\times min(q,1-q)$ where $q$ is the percentile value of zero in the bootstrap distribution of $D_m$.

Method 2:

Without any resampling of rats, for every bootstrap iteration, we will start with the same original 3 rats and perform all the other steps mentioned for Method 1.

Your help is greatly appreciated Thanks MS

• 1) Your formula D = D1-D2 should be D = Z1-Z2? 2) How you select number of repeation your procedure (2000)? – Nick Sep 9 '16 at 0:13
• You are right. It should be Z1-Z2. I fixed it. As for the the number of replications, a colleague who works on related problems said 2000 is OK. I guess this number varies depending on the what is computed in each replication. See Section 6.4 "The number of bootstrap replications B" in Efron & Tibshirani's book "An Introduction to the Bootstrap". I have tried to increase the replications and the shape of the bootstrap distribution doesn't change noticeably; so I stayed with 2000. – user130451 Sep 9 '16 at 0:46

If the results are consistent across rats, both methods should work equally well. However, if your test statistic $D$ varies a lot across rats, then resampling (your Method 1) will give you a distribution that captures the variability across rats. The graphic below is fairly self explanatory and should help you understand why you might want to resample.
One problem I see with both methods is that they don't take differences in the number of observations across rats into account. For example, suppose you recorded twice as many neurons from $r_1$ compared to $r_2$, you want your test statistic to give more weight to data from $r_1$. So I suggest you modify your mean test statistic by weighting the difference $D$ in each rat by the number of neurons you recorded from that rat. Something like:
$$D_m = \frac{\sum_in_iD_i}{\sum_in_i}$$
where $n_i$ and $D_i$ denote the number of neurons and the value of the test statistic from the $i^{th}$ rat.