2
$\begingroup$

I am having trouble understanding the statistical significance of Cohen's d.

My reasoning goes like this: To conclude that experimental group differs from control group significantly (p < 0.05), the experimental group's mean must be 1.96z below or above the control group's mean. In this case the Cohen's d will also be about 2, right? If Cohen's d is lower than 2, also the z difference between groups will be lower than that, which means the means from two groups won't be statistically different from each other anymore.

Please correct my flawed understanding.

$\endgroup$
1
  • 7
    $\begingroup$ Cohen's $d$ is determined using the standard deviation (typically of the available sample). Hypothesis testing is performed using the standard error $\frac{sd}{\sqrt{N}}$. The former assesses how big of a standardized difference you observed. The latter is used to determine the probability that you obtained a sample with the characteristics you did from a population of interest. $\endgroup$ Jan 22, 2018 at 10:48

1 Answer 1

3
$\begingroup$

A Cohen's $d$ is a standardized measure of difference. However, it is not weighted by the sample size. To go from a Cohen's $d$ to a $t$ statistics, the former must be mulitplied by $\sqrt{\tilde{n}/2}$ in which $\tilde{n}$ is the harmonic mean of the sample sizes. Conversely, the $t$ statistic is an effect size magnified by the sample size. The larger the sample, the bigger $t$ will end up. Hence, you could for example have a $t$ statistic of 1.96 (borderline significant) obtained from a sample of two groups, each containing two million participants. The resulting Cohen's $d$ is in that case a mere 0.00196! (yet, borderline significant!).

See 10.20982/tqmp.14.4.p242 for more.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.