3
$\begingroup$

There is a common language effect size (CLES) defined for Cohen's d (such as probability of superiority). Is there anything similar for Cramer's V or generally for r family (measure of association) effect size?

$\endgroup$
2
  • $\begingroup$ Cramer's v is for categorical variables. What does superiority means in that context? $\endgroup$
    – Tal Galili
    Feb 5, 2021 at 11:50
  • $\begingroup$ The probability of superiority of E over C is the probability that a randomly chosen value from the E distribution is greater than a randomly chosen value from C. It is kind of jargon-free version of Cohen's d. I don't know yet what CLES for Cramer's V would look like. $\endgroup$
    – Royalblue
    Feb 5, 2021 at 14:28

1 Answer 1

3
$\begingroup$

In their review of common language effect sizes (CLES), Mastrich & Hernandez (2021) do mention counterparts for a couple of r family effect sizes (see the table at the end of this answer), but they do not mention a CLES counterpart to Cramér's V.

However, for 2X2 contingency tables that examine the frequency of success versus failure of a treatment and control groups, they mention the binomial effect size display (BESD) as a CLES analogous to the $\phi$ coefficient (phi). The absolute value of the $\phi$ coefficient happens to equal Cramér's V for 2X2 contingency tables.

So in theory, in this specific situation, you could use the BESD as a common language effect size for Cramér's V – with all the important caveats related to using the BESD in general, at the risk otherwise to make an erroneous interpretation of the table, e.g. see the paper by Randolph & Edmondson previously linked, in particular pp. 4-5. For 2X2 tables, definitely check if it's a better idea to use odds ratio, risk ratio, and risk difference, as effect size measures.

In addition, Cramér's V does not have a sign, so from a practical point of view you might also have to take extra precautions to convert it correctly to the BESD – be careful not to mix up the treatment and the control group in your calculations.

For other situations than 2X2 treatment-control contingency tables, the absence of a CLES analogous to Cramér's V (or other effect size based on the chi-square statistic, like Cohen's $\omega$ or Tschuprow's $T$) may come from the fact that two very different contingency tables can have the same chi-square-based effect size. Given that, translating an effect size like Cramér's V directly to a CLES is probably not a straightforward thing to do - assuming this is even possible.

In addition, an effect size like Cramér's V, while it may be useful in the research process, is not necessarily what you want to communicate to an audience unfamiliar with statistics. For instance, the interesting thing about your table may be specific cells deviating a lot from independence; or the fact that a specific row or column is very different from others. Cramér's V won't inform you about all of this, so a CLES based on Cramér's V won't either.

Here is a related question you may find interesting: interpreting Cramer's V results.


In case you don't have access to the Mastrich & Hernandez's paper, here is a list of the CLES they identified along their "regular" effect size counterpart, and the original references:

"regular" effect size CLES counterpart Reference
Cohen's d (T-Test) $CL_d$ https://doi.org/10.1037/0033-2909.111.2.361
Post-hoc Cohen's d (One-way ANOVA) $CL_{d-All}$ see previous
Mann-Whitney U A https://doi.org/10.1037/1082-989x.7.4.485 ; https://doi.org/10.2466/11.IT.3.1 ; https://doi.org/10.1037/1082-989X.13.1.19
$\phi$ (2x2 contingency tables) BESD https://doi.org/10.1037/0022-0663.74.2.166
Pearson's $r$ $CL_r$ https://doi.org/10.1037/0033-2909.116.3.509
Kendall's $\tau$ $B_p$ https://doi.org/10.3758/s13428-018-1089-5
Multiple regression coefficient (no interaction) $CL_{\beta j}$ https://doi.org/10.1037/apl0000296
Multiple regression coefficient (with interactions) $CL_{\beta}$ see previous
Mahalanobis Distance D $CL_D$ https://doi.org/10.1177/2168479014542603
Odds ratio Number needed to treat https://doi.org/10.1016/S0895-4356(01)00510-8

Note that one of the two authors offer an online tool to convert these "regular" effect sizes to their CLES counterparts: https://ivanhernandez.com/software/clescalculator.html

References

Mastrich, Z., & Hernandez, I. (2021). Results everyone can understand: A review of common language effect size indicators to bridge the research-practice gap. Health Psychology, 40(10), 727–736. https://doi.org/10.1037/hea0001112

Randolph, Justus J. and Edmondson, R. Shawn (2005) "Using the Binomial Effect Size Display (BESD) to Present the Magnitude of Effect Sizes to the Evaluation Audience," Practical Assessment, Research, and Evaluation: Vol. 10, Article 14. DOI: https://doi.org/10.7275/zqwr-mx46

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.