2
$\begingroup$

I have a dataset within which I have a particular response variable that I'm interested in and numerous predictor variables. All variables are nominal and have as many as 15 possible values. When I cross tabulate any given predictor variable with the response variable, I get many cells with 0 counts, making performing a chi-squared test of independence inappropriate. That's fine, because I can use Fisher's exact test, but it's a problem in terms of calculating effect size since Cramer's V and every other method I've found that works for nominal data like mine seems to rely on chi-squared. Are there any alternatives to Cramer's V that don't have this problem? Or, if I'm misunderstanding something, is it still valid to user Cramer's V even if a chi-squared test is inappropriate?

$\endgroup$
1
$\begingroup$

The effect sizes I assume you are considering --- Cramer's V, (phi), Contingency coefficient C, and Cohen's w --- can all be calculated with the chi-square value. But the chi-square is simply calculated from the difference of observed values from expected values. This is way Cohen defines his w in Cohen (1988).

I assume that because there's no inference with these statistics, that it is fine to report them even if some test using the chi-square statistic would not be appropriate. It's like saying the difference between two means is some value, without addressing whether or not you could use a t-test or not in this case.

$\endgroup$
  • $\begingroup$ That's good news. So does that mean that the "not too many expected frequencies of less than 5 in the contingency table" rule has nothing to do with the actual chi-squared value? $\endgroup$ – joshisanonymous Sep 6 '17 at 20:35
  • $\begingroup$ I can't give a definitive answer to this. To me, there's no harm in calculating a statistic based on expected and observed values, but it's possible things get wiggy if the expected counts or low, or especially if zeros are common. $\endgroup$ – Sal Mangiafico Sep 6 '17 at 23:35
  • $\begingroup$ You might keep on eye on this new thread, and see if there's helpful information: researchgate.net/post/… $\endgroup$ – Sal Mangiafico Sep 6 '17 at 23:36
  • $\begingroup$ I have many expected counts which are less than 1. $\endgroup$ – joshisanonymous Sep 6 '17 at 23:48
  • 1
    $\begingroup$ @Glen_b 's comment on the following question confirms my guess that low expected counts are problematic only when computing p-values, not when calculating effect sizes. stats.stackexchange.com/questions/290887/… $\endgroup$ – Sal Mangiafico Sep 8 '17 at 13:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.