What are some alternatives to the chi-squared test for categorical variables with tables larger than 2 x 2 and cells with a count less than 5, if I don't want to merge classes?
There are some common misunderstandings here. The chi-squared test is perfectly fine to use with tables that are larger than $2\!\times\! 2$. In order for the actual distribution of the chi-squared test statistic to approximate the chi-squared distribution, the traditional recommendation is that all cells have expected values $\ge 5$. Two things must be noted here:
It does not matter what the observed cell counts are—they could well be $0$ with no problem—only the expected counts matter.
This traditional rule of thumb is now known to be too conservative. It can be fine to have $\le 20\%$ of the cells with expected counts $< 5$ as long as no expected counts are $<1$. See:
- Campbell Ian, 2007, Chi-squared and Fisher-Irwin tests of two-by-two tables with small sample recommendations, Statistics in Medicine, 26, 3661 - 3675
If your expected counts do not match this more accurate criterion, there are some alternative options available:
Your best bet is probably to simulate the sampling distribution of the test statistic, or to use a permutation test. In R, for example, you can simply set
chisq.test(..., simulate.p.value=TRUE). Other software should also make this possible.
You could use an alternative test, such as Fisher's exact test. Although Fisher's exact test is often recommended in this situation, it is worth noting that it makes different assumptions and may not be appropriate. Namely, Fisher's exact test assumes the row and column counts were set in advance and only the arrangement of the row x column combinations can vary (see: Given the power of computers these days, is there ever a reason to do a chi-squared test rather than Fisher's exact test?). If you are uncomfortable with this assumption, simulating the chi-squared will be a better option.