2
$\begingroup$

In 1988 Cohen suggested the following interpretation for effect size: "small~0.20, Medium~0.50, Large~0.80". I'm aware that these values are rather arbitrary and that an extensive disclaimer in this respect is already in Cohen's work.

Yet, is there a similar interpretation for Cliff's delta?

I found something similar for Cramer's V Small 0.1 0.2, Medium 0.3 0.5, Large 0.5. But nothing about Cliff's delta.

$\endgroup$
1
  • 2
    $\begingroup$ Cohen's classification was internally inconsistent and contradictory: depending on which of several equivalent statistical procedures you might choose in some circumstances (e.g., t-test, ANOVA, regression), his thresholds were not equivalent! $\endgroup$
    – whuber
    Jul 8, 2021 at 17:54

3 Answers 3

5
$\begingroup$

You can name effect sizes whatever you want and your names would be no less valid than Cohen's "T-shirt effect sizes". The thing about the T-shirt effect sizes is that, despite their widespread use, there's nothing to recommend them beyond "Cohen said so". (Meanwhile, Cohen said a lot of other, more important things about data analysis, but these have been largely ignored, probably because they would require taking data analysis more seriously.) A $d$ of .2 could be small in one context and large in another, which is unavoidable considering that $d$ is standardized and hence throws away the original units.

In general, if you're confronted with an effect and want to know how big it is in an intuitive sense, you should compare it to a plot of the data it came from, rather than standardizing it and looking for T-shirt sizes for the standardized form.

$\endgroup$
3
$\begingroup$

There is also this paper:

Appropriate statistics for ordinal level data: Should we really be using t-test and cohen’s d for evaluating group differences on the NSSE and other surveys? 2006, Ramano et al.

It suggests: 0.147 (small), 0.33 (medium), and 0.474 (large) based on a conversion (assuming normal distributions) from the commonly used thresholds for Cohen's d. Which of course makes it a "Cohen said so" case again @Kodiologist ;). In detail:

Because Cliff did not suggest the use of his delta statistic beyond hypothesis testing and confidence interval estimation, he did not suggest corresponding values to represent small, medium, and large effects. However, Cohen (1988) presents interpretations of the effect size index d in terms of the non-overlap between two normal distributions. This provides a direct bridge between d and delta. That is, with two normal distributions, a difference in means that represents a d effect size of 0.20 will have of delta value of 0.147, a d effect size of 0.50 corresponds to a delta value of 0.33, and a d effect size of 0.80 corresponds to a delta of 0.474.

$\endgroup$
2
$\begingroup$

Vargha and Delaney (2000) give interpretations of their A statistic:

small, >= 0.56; medium, >= 0.64; large, >= 0.71

The corresponding values they give for Cliff's delta are:

small, >= 0.11; medium, >= 0.28; large, >= 0.43

Be cautious in evaluating negative Cliff's delta values and A values less than 0.50.

But, because both VDA and Cliff's delta are easy to express as probabilities (e.g., of an observation in one group being larger than an observation in another group), there is less need with these statistics than perhaps with some others to convey the small, medium, large interpretations.

Reference:

Vargha, A. and H.D. Delaney. A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong. 2000. Journal of Educational and Behavioral Statistics 25(2):101–132.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.