Combining effect sizes to learn about the difference between two groups?

Question

I am interesting in learning if there is difference between the control and the treatment group in a paper I am reading. Unfortunately the paper only compares the respective conditions to a 0 effect size. The paper does not have open data or report standard deviations.

The relevent results reported in the paper are:

A one-sample t test on the recommended IAT measure indicated that the mean D score for the participants in the control condition (M = −.19 was significantly less than zero, t(29) = −2.50, p = .019. The estimated Cohen’s d for this effect was .46. In contrast, a one-sample t test indicated that the mean D score for the participants in the perspective taking condition (M = −.13) was not significantly less than zero, t(29) = −1.61, p = .119.

Is it possible to compare the two effect sizes to learn about the difference between the two conditions instead?

Answer 1

I assume that what you are interested in is the difference in differences. Simply differencing the mean D for the treatment and control is this value (i.e., -.19 - (-.13) = -.06). You can compute a t-test on this difference by recovering the standard deviations of the difference scores from the t-value: $$ s_D = \sqrt{\frac{M_D^2 N}{t^2}} . $$ For the control group, this is: $$ s_D = \sqrt{\frac{-.19^2(30)}{-2.50^2}} = 0.416 . $$ For the treatment group this is: $$ s_D = \sqrt{\frac{-.13^2(30)}{-1.61^2}} = 0.442. $$ With these values, you can compute a t-test comparing these two mean difference scores: $$ t = \frac{M_{D_{Tx}} - M_{D_{Cg}}}{ \sqrt{ \left(\frac{ s_1^2(N_1 - 1) + s_2^2(N_2 - 1)}{N_1 + N_2 - 2}\right) \left(\frac{N_1 + N_2}{N_1N_2}\right) } } , $$ $$ t = \frac{-0.19 - (-0.13)}{ \sqrt{ \left(\frac{ 0.442^2(30 - 1) + 0.416(30 - 1)}{30 + 30 - 2}\right) \left(\frac{30 + 30}{30(30)}\right) } } = -0.1397 . $$

This is a more meaningful test of any treatment effect and is the interaction in a 2 by 2 split-plot ANOVA (time by treatment).

Note that the authors computed Cohen's d as the mean difference divided by the standard deviation of the mean difference. A more standard Cohen's d for paired data is to divide by either the pre-test standard deviation or both the pooled pre and post-test standard deviations. Dividing by the standard deviation in differences is standardizing change relative to variability in change across people rather than simply variability in the measure across people. I think that the latter is far more interpretable.