I have a dataset within which I have a particular response variable that I'm interested in and numerous predictor variables. All variables are nominal and have as many as 15 possible values. When I cross tabulate any given predictor variable with the response variable, I get many cells with 0 counts, making performing a chi-squared test of independence inappropriate. That's fine, because I can use Fisher's exact test, but it's a problem in terms of calculating effect size since Cramer's V and every other method I've found that works for nominal data like mine seems to rely on chi-squared. Are there any alternatives to Cramer's V that don't have this problem? Or, if I'm misunderstanding something, is it still valid to user Cramer's V even if a chi-squared test is inappropriate?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
6
The effect sizes I assume you are considering --- Cramer's V, (phi), Contingency coefficient C, and Cohen's w --- can all be calculated with the chi-square value. But the chi-square is simply calculated from the difference of observed values from expected values. This is way Cohen defines his w in Cohen (1988).
I assume that because there's no inference with these statistics, that it is fine to report them even if some test using the chi-square statistic would not be appropriate. It's like saying the difference between two means is some value, without addressing whether or not you could use a t-test or not in this case.
answered Sep 6, 2017 at 13:44
-
$\begingroup$ That's good news. So does that mean that the "not too many expected frequencies of less than 5 in the contingency table" rule has nothing to do with the actual chi-squared value? $\endgroup$ Commented Sep 6, 2017 at 20:35
-
$\begingroup$ I can't give a definitive answer to this. To me, there's no harm in calculating a statistic based on expected and observed values, but it's possible things get wiggy if the expected counts or low, or especially if zeros are common. $\endgroup$ Commented Sep 6, 2017 at 23:35
-
$\begingroup$ You might keep on eye on this new thread, and see if there's helpful information: researchgate.net/post/… $\endgroup$ Commented Sep 6, 2017 at 23:36
-
$\begingroup$ I have many expected counts which are less than 1. $\endgroup$ Commented Sep 6, 2017 at 23:48
-
1$\begingroup$ @Glen_b 's comment on the following question confirms my guess that low expected counts are problematic only when computing p-values, not when calculating effect sizes. stats.stackexchange.com/questions/290887/… $\endgroup$ Commented Sep 8, 2017 at 13:04