# Should I be concerned if the cells of values obtained from bootstrapping are correlated?

I have data from 40 people, each measured once in 2 conditions. For reasons tangential to my question I'm not comfortable using parametric analyses to compare the two conditions, so I'm employing bootstrapping. However, when I run a bootstrap analysis (resampling people with replacement, computing condition means, then repeating 1000 times), the condition means I obtain appear to be correlated across iterations. I noticed this because when I plot the 95% confidence interval in each condition the intervals capture each others' means, but when I compute a difference score between the conditions means within each iteration, the 95% confidence interval of this difference score excludes zero. Going back to the results of the bootstrapping, I find that the correlation of the conditions across the bootstrap iterations is .7! Should this affect my confidence in the interval generated for the difference score? Alternatively, does this tell me something (possibly about individual differences) about my data otherwise?

• Assuming you've got a valid rationale for comparing the same people in the two conditions without a true control group, I'm not sure the correlation across bootstrap iterations is really something to be worried about in itself. It seems like you are, in effect, calculating CIs for the Average Treatment Effect on the Treated (no Wikipedia entry yet, sorry) and interpreting that as a meaningful outcome of interest. That said, I'm having a hard time following exactly what steps you took to estimate the differences between conditions. Is it a simple difference of means? Something else? Commented Feb 27, 2011 at 22:17
• @ashaw: Comparison of conditions experienced by the same individuals is a standard methodological approach that increases power by permitting removal of between-Ss variance that usually obscures effect variance in completely between-Ss designs. Regarding the computation of the difference score CI, I have edited the post above to clarify that on each iteration the condition means were computed and collapsed to a difference, yielding a distribution of differences from which the 95% interval. Commented Feb 28, 2011 at 18:19
• that helps for sure. Apologies if you read the first phrase of my comment to suggest that the approach was invalid in any way. Rather, I meant to point out that the validity of my response was contingent on the validity of the study design. In any event, now I can mull over a real response a little more... Commented Feb 28, 2011 at 18:52