Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

For an effect size analysis, I am noticing that there are differences between Cohen's d, Hedges's g and Hedges' g*.

  • Are these three metrics normally very similar?
  • What would be a case where they would produce different results?
  • Also is it a matter of preference which I use or report with?
share|improve this question
    
In case it's useful for a potential answerer formulas are listed here: en.wikipedia.org/wiki/Effect_size –  Jeromy Anglim Aug 19 '10 at 6:58
    
A simulation in R with varying n1, n2, s1, s2, and population difference would make a nice exercise. Anyone? –  Jeromy Anglim Aug 19 '10 at 8:04
    
This material is also covered here: What's the difference between Hedges' g and Cohen's d. –  gung Aug 10 '13 at 13:09

3 Answers 3

up vote 8 down vote accepted

Both Cohen's d and Hedges' g pool variances on the assumption of equal population variances, but g pools using n - 1 for each sample instead of n, which provides a better estimate, especially the smaller the sample sizes. Both d and g are somewhat positively biased, but only negligibly for moderate or larger sample sizes. The bias is reduced using g*. The d by Glass does not assume equal variances, so it uses the sd of a control group or baseline comparison group as the standardizer for the difference between the two means.

These effect sizes and Cliff's and other nonparametric effect sizes are discussed in detail in my book:

Grissom, R. J., & Kim, J, J. (2005). Effect sizes for research: A broad practical approach. Mahwah, NJ: Erlbaum.

share|improve this answer

To my understanding, Hedges's g is a somewhat more accurate version of Cohen's d (with pooled SD) in that we add a correction factor for small sample. Both measures generally agree when the homoscedasticity assumption is not violated, but we may found situations where this is not the case, see e.g. McGrath & Meyer, Psychological Methods 2006, 11(4): 386-401 (pdf). Other papers are listed at the end of my reply.

I generally found that in almost every psychological or biomedical studies, this is the Cohen's d that is reported; this probably stands from the well-known rule of thumb for interpreting its magnitude (Cohen, 1988). I don't know about any recent paper considering Hedges's g (or Cliff delta as a non-parametric alternative). Bruce Thompson has a revised version of the APA section on effect size.

Googling about Monte Carlo studies around effect size measures, I found this paper which might be interesting (I only read the abstract and the simulation setup): Robust Confidence Intervals for Effect Sizes: A Comparative Study of Cohen’s d and Cliff’s Delta Under Non-normality and Heterogeneous Variances (pdf).

About your 2nd comment, the MBESS R package includes various utilities for ES calculation (e.g., smd and related functions).

Other references

  1. Zakzanis, K.K. (2001). Statistics to tell the truth, the whole truth, and nothing but the truth: Formulae, illustrative numerical examples, and heuristic interpretation of effect size analyses for neuropsychological researchers. Archives of Clinical Neuropsychology, 16(7), 653-667. (pdf)
  2. Durlak, J.A. (2009). How to Select, Calculate, and Interpret Effect Sizes. Journal of Pediatric Psychology (pdf)
share|improve this answer
    
An anonymous user wanted to add the following definition of homoscedasticity for those who might be unfamiliar w/ the term: "a property of a set of random variables where each variable has the same finite variance". –  gung Apr 19 '13 at 16:20

If you're just trying to understand the basic meaning of Hedges' g, as I am, you might also find this helpful:

The magnitude of Hedges’ g may be interpreted using Cohen's (1988 [2]) convention as small (0.2), medium (0.5), and large (0.8). [1]

Their definition is short and clear:

Hedges’ g is a variation of Cohen's d that corrects for biases due to small sample sizes (Hedges & Olkin, 1985). [1] footnote

I would appreciate statistics experts editing this to add any important caveats to the small (0.2) medium (0.5) and large (0.8) claim, to help nonexperts avoid misinterpreting Hedges' g numbers used in social science and psychology research.

[1] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2848393/ The Effect of Mindfulness-Based Therapy on Anxiety and Depression: A Meta-Analytic Review Stefan G. Hofmann, Alice T. Sawyer, Ashley A. Witt, and Diana Oh. J Consult Clin Psychol. 2010 April; 78(2): 169–183. doi: 10.1037/a0018555

[2] Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Erlbaum; Hillsdale, NJ: 1988 (cited in [1])

share|improve this answer
1  
+1. Re: small-medium-large, as a 1st pass, if you have no relevant knowledge or context whatsoever, these 't-shirt sizes' are OK, but in reality, what is a small or large effect will vary by discipline or topic. Moreover, just because an effect is 'large' doesn't necessarily mean it's practically important or theoretically meaningful. –  gung Aug 10 '13 at 13:07

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.