What are some alternatives to the chi-squared test for categorical variables with tables larger than 2 x 2 and cells with a count less than 5, if I don't want to merge classes?

  • 2
    $\begingroup$ The Chi-Square-test can also be used with larger tables than 2x2. Could you explain why the Chi-Square-test should not be appropriate for your problem? Additionally, could you state the problem you're hoping to solve? $\endgroup$ Jun 28 '15 at 16:33
  • $\begingroup$ I have a 2 x 3 contingency table, and cells with a count less than 5 $\endgroup$
    – Israel
    Jun 28 '15 at 16:37
  • 2
    $\begingroup$ Thanks, please edit your question and add this information as not everyone reads the comments. A usual rule of thumb regarding the Chi-Square-test is that its results can be inaccurate if the expected cell counts are lower than 5. Usually, a Fisher-Test is recommended in these cases. Barnard's test may also be an option. $\endgroup$ Jun 28 '15 at 16:52

There are some common misunderstandings here. The chi-squared test is perfectly fine to use with tables that are larger than $2\!\times\! 2$. In order for the actual distribution of the chi-squared test statistic to approximate the chi-squared distribution, the traditional recommendation is that all cells have expected values $\ge 5$. Two things must be noted here:

  1. It does not matter what the observed cell counts are—they could well be $0$ with no problem—only the expected counts matter.

  2. This traditional rule of thumb is now known to be too conservative. It can be fine to have $\le 20\%$ of the cells with expected counts $< 5$ as long as no expected counts are $<1$. See:

If your expected counts do not match this more accurate criterion, there are some alternative options available:

  1. Your best bet is probably to simulate the sampling distribution of the test statistic, or to use a permutation test. In R, for example, you can simply set chisq.test(..., simulate.p.value=TRUE). Other software should also make this possible.

  2. You could use an alternative test, such as Fisher's exact test. Although Fisher's exact test is often recommended in this situation, it is worth noting that it makes different assumptions and may not be appropriate. Namely, Fisher's exact test assumes the row and column counts were set in advance and only the arrangement of the row x column combinations can vary (see: Given the power of computers these days, is there ever a reason to do a chi-squared test rather than Fisher's exact test?). If you are uncomfortable with this assumption, simulating the chi-squared will be a better option.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.