Should I be concerned if the cells of values obtained from bootstrapping are correlated? I have data from 40 people, each measured once in 2 conditions. For reasons tangential to my question I'm not comfortable using parametric analyses to compare the two conditions, so I'm employing bootstrapping. However, when I run a bootstrap analysis (resampling people with replacement, computing condition means, then repeating 1000 times), the condition means I obtain appear to be correlated across iterations. I noticed this because when I plot the 95% confidence interval in each condition the intervals capture each others' means, but when I compute a difference score between the conditions means within each iteration, the 95% confidence interval of this difference score excludes zero. Going back to the results of the bootstrapping, I find that the correlation of the conditions across the bootstrap iterations is .7! Should this affect my confidence in the interval generated for the difference score? Alternatively, does this tell me something (possibly about individual differences) about my data otherwise?
 A: While you didn't say anything about the actual condtions (experimental treatment), think of the following example: you measure the weight of 40 people at some point during the day, then set them the task to drink (as much as possible of) 5 quarts of water (experimental manipulation), and them weigh them again (hinting at "yes" being the answer to your final question...).
Now, the values for the means of each group will, pre and post treatment, very likely overlap (given that, unless you select people with the same pre-treatment weight, the general population shows a great variability). The mean of the difference between post and pre (post - pre), on the other hand, is likely to be significantly different from (i.e. greater than) 0.
In other words, without knowing a little more about your conditions (e.g. how does the metric change between conditions? do you expect it to show signs of stability?) and also about the variability on that measure across the population (if you think of the measure as a random variable, each subject has his/her own probability distribution of getting a certain score under a certain condition; and the measurement error might be very small compared to the reliable difference of each participants measure compared to the population mean).
Does any of this help?
