Difference between Cohen's d and Hedges' g for effect size metrics - Cross Validated most recent 30 from stats.stackexchange.com 2019-06-25T01:45:23Z https://stats.stackexchange.com/feeds/question/1850 http://www.creativecommons.org/licenses/by-sa/3.0/rdf https://stats.stackexchange.com/q/1850 18 Difference between Cohen's d and Hedges' g for effect size metrics Elpezmuerto https://stats.stackexchange.com/users/559 2010-08-18T17:35:17Z 2018-09-28T23:28:42Z <p>For an effect size analysis, I am noticing that there are differences between Cohen's d, Hedges's g and Hedges' g*. </p> <ul> <li>Are these three metrics normally very similar? </li> <li>What would be a case where they would produce different results? </li> <li>Also is it a matter of preference which I use or report with?</li> </ul> https://stats.stackexchange.com/questions/1850/-/1885#1885 8 Answer by chl for Difference between Cohen's d and Hedges' g for effect size metrics chl https://stats.stackexchange.com/users/930 2010-08-19T08:50:44Z 2010-08-19T08:50:44Z <p>To my understanding, Hedges's g is a somewhat more accurate version of Cohen's d (with pooled SD) in that we add a correction factor for small sample. Both measures generally agree when the homoscedasticity assumption is not violated, but we may found situations where this is not the case, see e.g. McGrath &amp; Meyer, <em>Psychological Methods</em> 2006, <strong>11(4)</strong>: 386-401 (<a href="http://www.bobmcgrath.org/Pubs/When_effect_sizes_disagree.pdf">pdf</a>). Other papers are listed at the end of my reply.</p> <p>I generally found that in almost every psychological or biomedical studies, this is the Cohen's d that is reported; this probably stands from the well-known rule of thumb for interpreting its magnitude (Cohen, 1988). I don't know about any recent paper considering Hedges's g (or Cliff delta as a non-parametric alternative). Bruce Thompson has a <a href="http://people.cehd.tamu.edu/~bthompson/apaeffec.htm">revised version</a> of the APA section on effect size.</p> <p>Googling about Monte Carlo studies around effect size measures, I found this paper which might be interesting (I only read the abstract and the simulation setup): <a href="http://www.coedu.usf.edu/main/departments/me/documents/cohen.pdf">Robust Confidence Intervals for Effect Sizes: A Comparative Study of Cohen’s d and Cliff’s Delta Under Non-normality and Heterogeneous Variances</a> (pdf).</p> <p>About your 2nd comment, the <code>MBESS</code> R package includes various utilities for ES calculation (e.g., <code>smd</code> and related functions).</p> <p><strong>Other references</strong></p> <ol> <li>Zakzanis, K.K. (2001). Statistics to tell the truth, the whole truth, and nothing but the truth: Formulae, illustrative numerical examples, and heuristic interpretation of effect size analyses for neuropsychological researchers. <em>Archives of Clinical Neuropsychology</em>, 16(7), 653-667. (<a href="http://www.sciencedirect.com/science?_ob=ArticleURL&amp;_udi=B6VDJ-43TFJ8B-4&amp;_user=10&amp;_coverDate=10%2F31%2F2001&amp;_rdoc=1&amp;_fmt=high&amp;_orig=search&amp;_sort=d&amp;_docanchor=&amp;view=c&amp;_searchStrId=1435354249&amp;_rerunOrigin=google&amp;_acct=C000050221&amp;_version=1&amp;_urlVersion=0&amp;_userid=10&amp;md5=12f831e4b738bda3a217d40d6cfd329f">pdf</a>)</li> <li>Durlak, J.A. (2009). How to Select, Calculate, and Interpret Effect Sizes. <em>Journal of Pediatric Psychology</em> (<a href="http://jpepsy.oxfordjournals.org/cgi/content/full/jsp004v1">pdf</a>)</li> </ol> https://stats.stackexchange.com/questions/1850/-/1930#1930 18 Answer by Robert J. Grissom for Difference between Cohen's d and Hedges' g for effect size metrics Robert J. Grissom https://stats.stackexchange.com/users/1013 2010-08-19T20:52:10Z 2010-08-19T20:52:10Z <p>Both Cohen's d and Hedges' g pool variances on the assumption of equal population variances, but g pools using n - 1 for each sample instead of n, which provides a better estimate, especially the smaller the sample sizes. Both d and g are somewhat positively biased, but only negligibly for moderate or larger sample sizes. The bias is reduced using g*. The d by Glass does not assume equal variances, so it uses the sd of a control group or baseline comparison group as the standardizer for the difference between the two means.</p> <p>These effect sizes and Cliff's and other nonparametric effect sizes are discussed in detail in my book:</p> <p>Grissom, R. J., &amp; Kim, J, J. (2005). Effect sizes for research: A broad practical approach. Mahwah, NJ: Erlbaum.</p> https://stats.stackexchange.com/questions/1850/-/56612#56612 3 Answer by joshoff for Difference between Cohen's d and Hedges' g for effect size metrics joshoff https://stats.stackexchange.com/users/24553 2013-04-19T16:18:08Z 2013-04-19T16:37:47Z <p>If you're just trying to understand the basic meaning of Hedges' g, as I am, you might also find this helpful:</p> <blockquote> <p>The magnitude of Hedges’ g may be interpreted using Cohen's (1988 ) convention as small (0.2), medium (0.5), and large (0.8).  </p> </blockquote> <p>Their definition is short and clear: </p> <blockquote> <p>Hedges’ g is a variation of Cohen's d that corrects for biases due to small sample sizes (Hedges &amp; Olkin, 1985).  footnote</p> </blockquote> <p>I would appreciate statistics experts editing this to add any important caveats to the small (0.2) medium (0.5) and large (0.8) claim, to help nonexperts avoid misinterpreting Hedges' g numbers used in social science and psychology research.</p> <p> <a href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2848393/" rel="nofollow">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2848393/</a> The Effect of Mindfulness-Based Therapy on Anxiety and Depression: A Meta-Analytic Review Stefan G. Hofmann, Alice T. Sawyer, Ashley A. Witt, and Diana Oh. J Consult Clin Psychol. 2010 April; 78(2): 169–183. doi: 10.1037/a0018555</p> <p> Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Erlbaum; Hillsdale, NJ: 1988 (cited in )</p> https://stats.stackexchange.com/questions/1850/-/175414#175414 1 Answer by TimeTravel for Difference between Cohen's d and Hedges' g for effect size metrics TimeTravel https://stats.stackexchange.com/users/91169 2015-10-04T12:38:31Z 2015-10-04T12:38:31Z <p>The other posters have covered the issue of similarities and differences between g and d. Just to add to this, some scholars do feel that the effect size values offered by Cohen are far too generous leading to overinterpretation of weak effects. They are also not tied to r leading to the possibility scholars may convert back and forth to obtain more favorably interpretable effect sizes. Ferguson (2009, Professional Psychology: Research and PRactice) suggested using the following values for interpretation for g:</p> <p>.41, as the recommended minimum for "practical significance." 1.15, moderate effect 2.70, strong effect</p> <p>These are obviously more rigorous/difficult to achieve and not many social science experiments are going to get to strong effects...which is probably how it should be.</p> https://stats.stackexchange.com/questions/1850/-/243065#243065 0 Answer by user136666 for Difference between Cohen's d and Hedges' g for effect size metrics user136666 https://stats.stackexchange.com/users/136666 2016-10-29T04:00:11Z 2016-10-29T04:00:11Z <p>Bruce Thompson did warn about using Cohen's (0.2) as small (0.5) as medium and (0.8) as large. Cohen never meant for these to be used as rigid interpretations. All effect sizes must be interpreted based on the context of the related literature. If you are analyzing the related effect sizes reported on your topic and they are (0.1) (0.3) (0.24) and you produce an effect of (0.4) then that may be "large". Conversely, if all the related literature has effects of (0.5) (0.6) (0.7) and you have the effect of (0.4) it may be considered small. I know this is a trivial example but imperatively important. I believe Thompson once stated in a paper, "We would merely be stupid in a different metric" when comparing interpretations of effect sizes to how social scientists were interpreting p values at the time.</p> https://stats.stackexchange.com/questions/1850/-/338043#338043 5 Answer by FelixST for Difference between Cohen's d and Hedges' g for effect size metrics FelixST https://stats.stackexchange.com/users/163818 2018-04-02T06:42:08Z 2018-09-28T23:28:42Z <p>It seems when people say Cohen's d they mostly mean: </p> <p><span class="math-container">$$d = \frac{\bar{x}_1 - \bar{x}_2}{s}$$</span></p> <p>Where <span class="math-container">$s$</span> is the pooled standard deviation, </p> <p><span class="math-container">$$s = \sqrt{\frac{\sum(x_1 - \bar{x}_1)^2 + (x_2 - \bar{x}_2)^2}{n_1 + n_2 - 2}}$$</span> </p> <p>There are other estimators for the pooled standard deviation, probably the most common apart from the above being:</p> <p><span class="math-container">$$s^* = \sqrt{\frac{\sum(x_1 - \bar{x}_1)^2 + (x_2 - \bar{x}_2)^2}{n_1 + n_2}}$$</span> </p> <p>Notation here is remarkably inconsistent, but sometimes people say that the the <span class="math-container">$s^*$</span> (i.e., the <span class="math-container">$n_1 + n_2$</span> version) version is called Cohen's <span class="math-container">$d$</span>, and reserve the name Hedge's <span class="math-container">$g$</span> for the version that uses <span class="math-container">$s$</span> (i.e., with Bessel’s correction, the n1+n2−2 version). This is a bit weird as Cohen outlined both estimators for the pooled standard deviation (e.g., <span class="math-container">$s$</span> version on p. 67, Cohen, 1977) before Hedges wrote about them (Hedges, 1981).</p> <p>Other times Hedge's g is reserved to refer to either of the bias corrected versions of a standardised mean difference that Hedges developed. Hedges (1981) showed that Cohen's d was upwardly biased (i.e., its expected value is higher than the true population parameter value), especially in small samples, and proposed a correction factor to correct for Cohen's d's bias: </p> <p>Hedges's g (the unbiased estimator):</p> <p><span class="math-container">$$g = d * (\frac{\Gamma(df/2)}{\sqrt{df/2 \,}\,\Gamma((df-1)/2)})$$</span> Where <span class="math-container">$df = n_1 + n_2 -2$</span> for an independent groups design, and <span class="math-container">$\Gamma$</span> is the gamma function. (originally Hedges 1981, this version developed from Hedges and Olkin 1985, p. 104) </p> <p>However, this correction factor is fairly computationally complex, so Hedges also provided a computationally trivial approximation that, while still slightly biased, is fine for almost all conceivable purposes: </p> <p>Hedges' <span class="math-container">$g^*$</span> (the computationally trivial approximation):</p> <p><span class="math-container">$$g^* = d*(1 - \frac{3}{4(df) - 1})$$</span> Where <span class="math-container">$df = n_1 + n_2 -2$</span> for an independent groups design. </p> <p>(Originally from Hedges, 1981, this version from Borenstein, Hedges, Higgins, &amp; Rothstein, 2011, p. 27)</p> <p>But, as for what people mean when they say Cohen's d vs. Hedges' g vs. g*, people seem to refer to any of these three estimators as Hedge's g or Cohen's d interchangeably, although I've never seen someone write "<span class="math-container">$g^*$</span>" in a non-methodology/stats research paper. If someone says "unbiased Cohen's d", you're just going to have to take your best guess at either of the last two (and I think there might even be another approximation that has been used for Hedge's <span class="math-container">$g^*$</span> too!). </p> <p>They are all virtually identical if <span class="math-container">$n &gt; 20$</span> or so, and all can be interpreted in the same way. For all practical purposes, unless you're dealing with really small sample sizes, it probably doesn't matter which you use (although if you can pick, you may as well use the one that I've called Hedges' g, as it is unbiased).</p> <p><em>References:</em></p> <p>Borenstein, M., Hedges, L. V., Higgins, J. P., &amp; Rothstein, H. R. (2011). Introduction to Meta-Analysis. West Sussex, United Kingdom: John Wiley &amp; Sons.</p> <p>Cohen, J. (1977). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ, US: Lawrence Erlbaum Associates, Inc. </p> <p>Hedges, L. V. (1981). Distribution Theory for Glass's Estimator of Effect size and Related Estimators. Journal of Educational Statistics, 6(2), 107-128. doi:10.3102/10769986006002107</p> <p>Hedges L. V., Olkin I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press</p>