I have the following dataset and I am trying to run a Chi-square test between the control and pilot.

data table

The only issue is that, for a Chi-Square test to return proper results, the sample size need to be greater than 5. In my case, most of my observations are under 5.

Can anyone suggest me a different kind of statistical test I can run instead?

I have looked at the Fisher Exact Test but it seems to only work for 2 by 2 data frames.


One possibility is to use Monte Carlo, that is, a chi-squared test with a simulated p value. I will show how to do that in R:

mytab <- cbind( control=c(0, 1, 0, 1, 1, 1, 3, 3, 8, 10, 3),
               pilot=c(1, 3, 2, 2, 2, 5, 3, 5, 1, 4, 2) )

chisq.test( mytab, sim=TRUE,  B=20000 ) 

    Pearson's Chi-squared test with simulated p-value (based on 20000

data:  mytab
X-squared = 16.037, df = NA, p-value = 0.07505

Technically you can do an exact test, but you probably must do the calculations yourself since I am not aware that those kind of tests are implemented. The point is that a Chi Square test is based on the fact that one can approximate binomial / multinomial counts with normal distributions when the number of samples (counts in our case) are large enough. Since you do not have enough samples, you should consider the original assumed distribution, which is the multinomial distribution for this case.

This is described in this paper Small numbers in chi-square and G–tests. A carefull read of it gives one enough hints on how to carry the computation but, as a wrod of warning, it is a tedious word to do.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.