One of my research hypotheses is that individuals from Southeast Asia who are ethnically Chinese are more likely to experience racially motivated hate crimes than their counterparts from other ethnic groups.

Respondents were recruited via non-probability sampling methods for my survey, and the data gathered for the hypothesis above are all nominal, with a sample size of 300, which means that the nonparametric Chi square test of independence is the most appropriate method of analyses.

However, there were 8 choices for ethnic groups (reflecting the heterogeneity of ethnicities in Southeast Asia) including the choice for "Chinese". I am expecting a frequency of < 5 in some of those cells due to the lack of response from individuals from particular ethnic groups. Is it appropriate then / even possible , to combine Chi-square test of independence with a Fisher Exact Test (to be used only for the ethnic groups with expected frequency of < 5)? Otherwise, how else go about the analysis?

  • 1
    $\begingroup$ Could you explain why you believe a chi-squared test would apply to a "non-probability" sample? $\endgroup$
    – whuber
    Commented Feb 24, 2021 at 17:44

1 Answer 1


The concern expressed in @whuber's comment needs to be taken seriously. However, assuming you have obtained appropriate data, your question what to do when you get error messages about low expected counts can be answered quickly.

Suppose you have randomly chosen subjects from 8 ethnic groups, who either report racially motivated hate crimes or not. Maybe your contingency table is as follows:

   [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]
hc    2    6   10    7    4    4    8    9
no   25   14   16   24   29   24   11    7

[1] 200

If you do a chi-squared test, you will get an error message because some of the expected counts are smaller than 5.


        Pearson's Chi-squared test

data:  MAT
X-squared = 23.265, df = 7, p-value = 0.001533

Warning message:
In chisq.test(MAT) : Chi-squared approximation 
 may be incorrect

    [,1] [,2] [,3]  [,4]  [,5] [,6]  [,7] [,8]
hc  6.75    5  6.5  7.75  8.25    7  4.75    4
no 20.25   15 19.5 23.25 24.75   21 14.25   12

In my fake data, I see only two expected counts below 5. Some statisticians would not be concerned about an inaccurate P-value as long as most of the expected counts exceed 5, but the remaining a few have values between 3 and 5. However, if you have a matrix of expected counts that is much more sparse than this one, then you need another method of analysis.

One alternative method is Fisher's exact test. With a matrix as large as $2\times 8,$ you may encounter excessive running times for Fisher's test or a message about insufficient storage space. [Fisher's exact test was originally proposed for $2\times 2$ tables, but some software will compute P-values for larger matrices. This may depend on your software and how it is used on your computer.]

An alternative in R is to simulate a P-value by including the parameter sim=T as an argument in the chisq.test procedure.

chisq.test(MAT, sim=T)

        Pearson's Chi-squared test with simulated
        p-value (based on 2000 replicates)

data:  MAT
X-squared = 23.265, df = NA, p-value = 0.0009995

For my data, this option works fine, giving a P-value a little smaller than the doubtful one above.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.