Is there a common language effect size for Cramer's V?

Question

There is a common language effect size (CLES) defined for Cohen's d (such as probability of superiority). Is there anything similar for Cramer's V or generally for r family (measure of association) effect size?

Answer 1

In their review of common language effect sizes (CLES), Mastrich & Hernandez (2021) do mention counterparts for a couple of r family effect sizes (see the table at the end of this answer), but they do not mention a CLES counterpart to Cramér's V. In addition, I did not find other references mentioning a CLES analogous to Cramér's V.

However, for 2X2 contingency tables that examine the frequency of success versus failure of a treatment and control groups, Mastrich & Hernandez mention the binomial effect size display (BESD) as a CLES analogous to the $\phi$ coefficient (phi). The absolute value of the $\phi$ coefficient happens to equal Cramér's V for 2X2 contingency tables.

So in theory, in this specific situation, you could use the BESD as a common language effect size for Cramér's V – with all the important caveats related to using the BESD in general, at the risk otherwise to make an erroneous interpretation of the table, e.g. see the paper by Randolph & Edmondson previously linked, in particular pp. 4-5. For 2X2 tables, definitely check if it's a better idea to use odds ratio, risk ratio, and risk difference, as effect size measures.

In addition, Cramér's V does not have a sign, so from a practical point of view you might also have to take extra precautions to convert it correctly to the BESD – be careful not to mix up the treatment and the control group in your calculations.

For other situations than 2X2 treatment-control contingency tables, the apparent absence in the literature of a CLES analogous to Cramér's V (or other effect size based on the chi-square statistic, like Cohen's $\omega$ or Tschuprow's $T$) may come from the fact that two very different contingency tables can have the same chi-square-based effect size. Given that, translating an effect size like Cramér's V directly to a CLES is probably not a straightforward thing to do – assuming this is even possible.

In addition, an effect size like Cramér's V, while it may be useful in the research process, is not necessarily what you want to communicate to an audience unfamiliar with statistics. For instance, the interesting thing about your table may be specific cells deviating a lot from independence; or the fact that a specific row or column is very different from others. Cramér's V won't inform you about all of this, so a CLES based on Cramér's V won't either.

Here is a related question you may find interesting: interpreting Cramer's V results.

---

In case you don't have access to the Mastrich & Hernandez's paper, here is a list of the CLES they identified along their "regular" effect size counterpart, and the original references:

"regular" effect size	CLES counterpart	Reference
Cohen's d (T-Test)	$CL_d$	McGraw, K. O., & Wong, S. P. (1992). A common language effect size statistic. Psychological Bulletin, 111(2), 361–365. https://doi.org/10.1037/0033-2909.111.2.361
Post-hoc Cohen's d (One-way ANOVA)	$CL_{d-All}$	ibid.
Mann-Whitney U	A	Delaney, H. D., & Vargha, A. (2002). Comparing several robust tests of stochastic equality with ordinally scaled variables and small to moderate sized samples. Psychological Methods, 7(4), 485–503. https://doi.org/10.1037/1082-989X.7.4.485 Kerby, D. S. (2014). The Simple Difference Formula: An Approach to Teaching Nonparametric Correlation. Comprehensive Psychology, 3. https://journals.sagepub.com/doi/full/10.2466/11.IT.3.1 Ruscio, J. (2008). A probability-based measure of effect size: Robustness to base rates and other factors. Psychological Methods, 13(1), 19–30. https://doi.org/10.1037/1082-989X.13.1.19
$\phi$ (2x2 contingency tables)	BESD	Rosenthal, R., & Rubin, D. B. (1982). A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology, 74(2), 166–169. https://doi.org/10.1037/0022-0663.74.2.166
Pearson's $r$	$CL_r$	Dunlap, W. P. (1994). Generalizing the common language effect size indicator to bivariate normal correlations. Psychological Bulletin, 116(3), 509–511. https://doi.org/10.1037/0033-2909.116.3.509
Kendall's $\tau$	$B_p$	Li, J. C.-H., & Waisman, R. M. (2019). Probability of bivariate superiority: A non-parametric common-language statistic for detecting bivariate relationships. Behavior Research Methods, 51(1), 258–279. https://doi.org/10.3758/s13428-018-1089-5
Multiple regression coefficient (no interaction)	$CL_{\beta j}$	Krasikova, D. V., Le, H., & Bachura, E. (2018). Toward customer-centric organizational science: A common language effect size indicator for multiple linear regressions and regressions with higher-order terms. Journal of Applied Psychology, 103(6), 659–675. https://doi.org/10.1037/apl0000296
Multiple regression coefficient (with interactions)	$CL_{\beta}$	ibid.
Mahalanobis Distance D	$CL_D$	Liu, X.S. (2015). Multivariate Common Language Effect Size. Therapeutic Innovation & Regulatory Science, 49, 126–131 . https://doi.org/10.1177/2168479014542603
Odds ratio	Number needed to treat	Bender, R., & Blettner, M. (2002). Calculating the “number needed to be exposed” with adjustment for confounding variables in epidemiological studies. Journal of Clinical Epidemiology, 55(5), 525–530. https://doi.org/10.1016/S0895-4356(01)00510-8

Note that one of the two authors offer an online tool to convert these "regular" effect sizes to their CLES counterparts: https://ivanhernandez.com/software/clescalculator.html

References

Mastrich, Z., & Hernandez, I. (2021). Results everyone can understand: A review of common language effect size indicators to bridge the research-practice gap. Health Psychology, 40(10), 727–736. https://doi.org/10.1037/hea0001112

Randolph, Justus J. and Edmondson, R. Shawn (2005) "Using the Binomial Effect Size Display (BESD) to Present the Magnitude of Effect Sizes to the Evaluation Audience," Practical Assessment, Research, and Evaluation: Vol. 10, Article 14. DOI: https://doi.org/10.7275/zqwr-mx46