Skip to main content
1 of 3
splint
  • 93
  • 1
  • 7

Effect size and bootstrapping in paired t test

I have multiple paired t-tests, such as one giving results

t(14)=2.7, p=.017

although people seem to do effect sizes in different ways in repeated samples, I have taken the mean difference divided by the standard deviation of the differences (I'll call this d, though maybe I should call it something else?) and get 0.70. I also have a very strong correlation between the samples, not sure if that is problematic.

I would like to put confidence limits around my effect size estimate. To do so, I randomly resample from the difference scores, compute d in the same way and repeat 1000 times. My question is whether this is a good approach, rather than, say, just giving confidence limits around the unstandardised difference or resampling from the original samples. My bootstrap gives me a mean d of 0.79 with confidence limits of [0.4, 1.4]. I've tried this on other random data too. Why am I getting a consistently higher d from bootstrapping, and why are the intervals asymmetric? Is this because of skew in the (difference) scores, and does this make this approach more or less robust?

Thanks!

splint
  • 93
  • 1
  • 7