2
$\begingroup$

I have the following dataset and I am trying to run a Chi-square test between the control and pilot.

data table

The only issue is that, for a Chi-Square test to return proper results, the sample size need to be greater than 5. In my case, most of my observations are under 5.

Can anyone suggest me a different kind of statistical test I can run instead?

I have looked at the Fisher Exact Test but it seems to only work for 2 by 2 data frames.

$\endgroup$
2
  • 1
    $\begingroup$ What happened with Sagittarius? Do you have a row of 0s (Sagittarius = 0 for both control and pilot) that you've omitted? $\endgroup$
    – dipetkov
    Aug 20, 2023 at 10:58
  • $\begingroup$ A Fisher's exact test can be used on larger than 2x2 data frames. It would be fine with, say, a 5x2 data frame. $\endgroup$
    – Jay
    Feb 13 at 0:57

2 Answers 2

4
$\begingroup$

One possibility is to use Monte Carlo, that is, a chi-squared test with a simulated p value. I will show how to do that in R:

mytab <- cbind( control=c(0, 1, 0, 1, 1, 1, 3, 3, 8, 10, 3),
               pilot=c(1, 3, 2, 2, 2, 5, 3, 5, 1, 4, 2) )

chisq.test( mytab, sim=TRUE,  B=20000 ) 

    Pearson's Chi-squared test with simulated p-value (based on 20000
    replicates)

data:  mytab
X-squared = 16.037, df = NA, p-value = 0.07505
$\endgroup$
3
  • 2
    $\begingroup$ Speculative question since the OP might not be around to answer. The table has 11 of the 12 zodiac signs; Sagittarius is missing. I think that a proper representation of the data is to include a row of 0s. So here is the question: Does the chisq.test makes sense then? $\endgroup$
    – dipetkov
    Aug 20, 2023 at 11:01
  • 1
    $\begingroup$ @dipetkov I'm not sure if your question is in fact rhetorical, but I was wondering the same thing after reading your other comment under the question. I find it could make a good question of its own. $\endgroup$
    – J-J-J
    Feb 13 at 7:51
  • 2
    $\begingroup$ @J-J-J I'm still curious about this but it also sounded like a question that would have been asked before. Found this thread. Has links to further relevant threads. $\endgroup$
    – dipetkov
    Feb 17 at 23:54
1
$\begingroup$

Technically you can do an exact test, but you probably must do the calculations yourself since I am not aware that those kind of tests are implemented. The point is that a Chi Square test is based on the fact that one can approximate binomial / multinomial counts with normal distributions when the number of samples (counts in our case) are large enough. Since you do not have enough samples, you should consider the original assumed distribution, which is the multinomial distribution for this case.

This is described in this paper Small numbers in chi-square and G–tests. A carefull read of it gives one enough hints on how to carry the computation but, as a wrod of warning, it is a tedious word to do.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.