Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have data showing fire fighter entrance exam results. I am testing the hypothesis that exam results are dependent on ethnicity. To test this, I ran a Pearson chi-square test in R. The results show what I expected, but it gave a warning that "In chisq.test(a) : Chi-squared approximation may be incorrect."

> a
       white black asian hispanic
pass       5     2     2        0
noShow     0     1     0        0
fail       0     2     3        4
> chisq.test(a)

    Pearson's Chi-squared test

data:  a
X-squared = 12.6667, df = 6, p-value = 0.04865

Warning message:
In chisq.test(a) : Chi-squared approximation may be incorrect

Does anyone know why it gave a warning? Is it because I am using a wrong method? Thanx in advance.

share|improve this question
add comment

4 Answers

up vote 7 down vote accepted

It gave the warning because many of the expected values will be very small and therefore the approximations of p may not be right.

In R you can use chisq.test(a, simulate.p.value = TRUE) to use simulate p values.

However, with such small cell sizes, all estimates will be poor. It might be good to just test pass vs. fail (deleting "no show") either with chi-square or logistic regression. Indeed, since it is pretty clear that the pass/fail grade is a dependent variable, logistic regression might be better.

share|improve this answer
    
can you please show how can i apply logistic regression ? –  user1883491 Jan 7 at 12:39
2  
I haven't time right now to do so in detail but look at glm and search on logistic regression. –  Peter Flom Jan 7 at 12:49
add comment

The issue is that the chi-square approximation to the distribution of the test statistic relies on the counts being roughly normally distributed. If many of the expected counts are very small, the approximation may be poor.

The noshow category will be a big contributor to the problem; one thing to consider is merging noshow and fail. You'll still get the warning but it won't affect the results nearly so much and the distribution should be quite reasonable (the rule that's being applied before the warning is given is too strict).

But in any case, you can deal with the problem very easily in R; set the simulate.p.value argument to TRUE; then you aren't reliant on the chi-square approximation to the distribution of the test statistic.

share|improve this answer
add comment

For such small counts, you could use Fisher's exact test:

> fisher.test(a)

        Fisher's Exact Test for Count Data

data:  a 
p-value = 0.02618
alternative hypothesis: two.sided 
share|improve this answer
add comment

Please see the "Assumptions" section of Pearson's chi-squared test article.

In a nutshell, when counts in any of the cells in your table are fewer than 5 then one of the assumptions is broken. I think that's what the error message is referring to. In the article linked you can also find about the correction that can be applied.

share|improve this answer
3  
There are two problems with your less-than-five count rule of thumb. The first is that the correct statement refers to the expected counts rather than the actual counts. The second is that it is too severe. The $\chi^2$ approximation often works well even when a small proportion of expected counts is less than five. In this case, where all the column marginals are five or less, it is obvious that every expected count is small and so we are advised to be cautious. Also, the correction mentioned in the Wikipedia article applies only in the one DF case; this case has 6 DF. –  whuber Jan 7 at 15:01
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.