Warning in R - Chi-squared approximation may be incorrect I have data showing fire fighter entrance exam results. I am testing the hypothesis that exam results and ethnicity are not mutually independent. To test this, I ran a Pearson chi-square test in R. The results show what I expected, but it gave a warning that "In chisq.test(a) : Chi-squared approximation may be incorrect."
> a
       white black asian hispanic
pass       5     2     2        0
noShow     0     1     0        0
fail       0     2     3        4
> chisq.test(a)

    Pearson's Chi-squared test

data:  a
X-squared = 12.6667, df = 6, p-value = 0.04865

Warning message:
In chisq.test(a) : Chi-squared approximation may be incorrect

Does anyone know why it gave a warning? Is it because I am using a wrong method?
 A: It gave the warning because many of the expected values will be very small and therefore the approximations of p may not be right.
In R you can use chisq.test(a, simulate.p.value = TRUE) to use simulate p values. 
However, with such small cell sizes, all estimates will be poor. It might be good to just test pass vs. fail (deleting "no show") either with chi-square or logistic regression. Indeed, since it is pretty clear that the pass/fail grade is a dependent variable, logistic regression might be better. 
A: Please see the "Assumptions" section of Pearson's chi-squared test article.
In a nutshell, when counts in any of the cells in your table are fewer than 5 then one of the assumptions is broken. I think that's what the error message is referring to. In the article linked you can also find about the correction that can be applied.
A: The issue is that the chi-square approximation to the distribution of the test statistic relies on the counts being roughly normally distributed. If many of the expected counts are very small, the approximation may be poor.
Note that the actual distribution of the chi-square statistic for independence in contingency tables is discrete, not continuous. 
The noshow category will be a big contributor to the problem; one thing to consider is merging noshow and fail. You'll still get the warning but it won't affect the results nearly so much and the distribution should be quite reasonable (the rule that's being applied before the warning is given is too strict).
But in any case, if you're willing to condition on the margins (as you do when running Fisher's exact test) you can deal with the problem very easily in R; set the simulate.p.value argument to TRUE; then you aren't reliant on the chi-square approximation to the distribution of the test statistic.
A: For such small counts, you could use Fisher's exact test:
> fisher.test(a)

        Fisher's Exact Test for Count Data

data:  a 
p-value = 0.02618
alternative hypothesis: two.sided 

A: Your main question talks about the sample size, but I see that more than two groups are compared. If the p-value from the test is 0.05 or less, it would be difficult to interpret the results. Therefore, I am sharing a brief script that I use in such situations:
# Load the required packages:
library(MASS) # for chisq
library(descr) # for crosstable

CrossTable(a$exam_result, a$ethnicity
       fisher = T, chisq = T, expected = T,
       prop.c = F, prop.t = F, prop.chisq = F, 
       sresid = T, format = 'SPSS')

This code will generate both Pearson's Chi-square and Fisher's Chi square. It produces counts as well as proportions of each of the table entries.
Based on the standardised residuals or z-values scores i.e., 
sresid

If it is outside the range |1.96| i.e., less than -1.96 or greater than 1.96, then it is significant p < 0.05. The sign would then indicate whether positively related or negatively.
A: Your counts per cell are too low. The general rule of thumb is, if the count is bellow 5, use fisher.test. 
> fisher.test(a)

The Fisher exact test extends well to small and large counts, while the chisq.test is generally used for larger counts. You have several values that are 0 and all are below 5, so the Fisher test is what you need! 
