1
$\begingroup$

When I calculate Effect Sizes (Cohen's d) on the same study using different methods I get completely different effect sizes.

For example: (t=4.81, P<0.001, d=0.41)

I also calculated d=0.41 using reported means and standard deviations, i.e.:

image

But, using just the t value and sample size of 16, I get 1.76:

enter image description here

0.41 and 1.76 are vastly different. Are these both Cohen's d, and if so, aren't they supposed to be equivalent?

Improve this question
$\endgroup$
2
  • $\begingroup$ I think $d=\frac{\bar{x_1}-\bar{x_2}}{s} $ is the formula for Cohen's d. Where is the second fromula from? Do you have a reference? $\endgroup$
    – Deep North
    Commented Jul 8, 2015 at 2:19
  • $\begingroup$ From here (formula 2). I also get effect sizes of a similar magnitude when calculating from t using online calculators such as this one $\endgroup$
    – EtTuBrute
    Commented Jul 8, 2015 at 2:32

1 Answer 1

Reset to default
0
$\begingroup$

I did some simulations, it seems the results of two formulas are the same. Just list my R code and resutls below.

set.seed(123)
x<-rnorm(100,mean=5, sd=2)
y<-rnorm(100,mean=10, sd=3)
u1<-mean(x)
sd1<-sd(x)
u2<-mean(y)
sd2<-sd(y)
t.test(y,x) 
s_pool<-sqrt((99*sd1^2+99*sd2^2)/200)

d1<-(u2-u1)/s_pool

d2<-13.1186*sqrt((200/10000)*(200/198))

$d_1$=1.864599

$d_2$=1.864597

I got almost the same results.

The t is 13.1186 here.

Improve this answer
$\endgroup$
3
  • $\begingroup$ So it does work! In my case it seems the problem is that the studies are reporting pre and post measures, so the t-test is paired. $\endgroup$
    – EtTuBrute
    Commented Jul 8, 2015 at 5:51
  • $\begingroup$ Then it is not two indepdent samples anymore. $\endgroup$
    – Deep North
    Commented Jul 8, 2015 at 5:55
  • $\begingroup$ That's the answer then. I am mistakenly trying to deduct effect size using an independent t-test in a study that used a dependent t-test. $\endgroup$
    – EtTuBrute
    Commented Jul 8, 2015 at 8:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.