1
$\begingroup$

When I calculate Effect Sizes (Cohen's d) on the same study using different methods I get completely different effect sizes.

For example: (t=4.81, P<0.001, d=0.41)

I also calculated d=0.41 using reported means and standard deviations, i.e.:

image

But, using just the t value and sample size of 16, I get 1.76:

enter image description here

0.41 and 1.76 are vastly different. Are these both Cohen's d, and if so, aren't they supposed to be equivalent?

$\endgroup$
2
  • $\begingroup$ I think $d=\frac{\bar{x_1}-\bar{x_2}}{s} $ is the formula for Cohen's d. Where is the second fromula from? Do you have a reference? $\endgroup$
    – Deep North
    Jul 8 '15 at 2:19
  • $\begingroup$ From here (formula 2). I also get effect sizes of a similar magnitude when calculating from t using online calculators such as this one $\endgroup$
    – EtTuBrute
    Jul 8 '15 at 2:32
0
$\begingroup$

I did some simulations, it seems the results of two formulas are the same. Just list my R code and resutls below.

set.seed(123)
x<-rnorm(100,mean=5, sd=2)
y<-rnorm(100,mean=10, sd=3)
u1<-mean(x)
sd1<-sd(x)
u2<-mean(y)
sd2<-sd(y)
t.test(y,x) 
s_pool<-sqrt((99*sd1^2+99*sd2^2)/200)

d1<-(u2-u1)/s_pool

d2<-13.1186*sqrt((200/10000)*(200/198))

$d_1$=1.864599

$d_2$=1.864597

I got almost the same results.

The t is 13.1186 here.

$\endgroup$
3
  • $\begingroup$ So it does work! In my case it seems the problem is that the studies are reporting pre and post measures, so the t-test is paired. $\endgroup$
    – EtTuBrute
    Jul 8 '15 at 5:51
  • $\begingroup$ Then it is not two indepdent samples anymore. $\endgroup$
    – Deep North
    Jul 8 '15 at 5:55
  • $\begingroup$ That's the answer then. I am mistakenly trying to deduct effect size using an independent t-test in a study that used a dependent t-test. $\endgroup$
    – EtTuBrute
    Jul 8 '15 at 8:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.