1
$\begingroup$

Fisher exact test is said to be used with a total sample (n) < 1000, whereas chi-squared test should be used when each category (/cell in a contingency table) >=5. What if you have an mxn contingency table where the total sample size > 1000 but some of the cells have 0 or 1 sample size?

$\endgroup$
3
  • 1
    $\begingroup$ The rule of thumb $\ge 5$ applies to expected frequencies. It's widely considered very conservative, and I could cite authorities for $\ge 1$ as a fairly safe rule of thumb. But it's always good advice to watch out when you have very small expected frequencies. For 0 or 1 "sample size" here, read "observed frequency"? $\endgroup$
    – Nick Cox
    May 20, 2016 at 16:26
  • $\begingroup$ Correct Nick; sample sizes here I was always referring to observed frequencies. $\endgroup$
    – dter
    May 20, 2016 at 16:34
  • $\begingroup$ I'd take a step back and consider whether a hypothesis of no association between variables makes sense any way. When some categories in a large sample are very rare, it's often a sign that you are a long way from independence of rows and columns. That's certainly not a rule as small frequencies can be expected under independence, but I've often seen it in practice. $\endgroup$
    – Nick Cox
    May 20, 2016 at 16:37

1 Answer 1

1
$\begingroup$

How far above 1,000 is your sample size? If it's not far above 1,000, you can use Fisher's exact test - it's simply recommended that you don't because of computational limitations.

If Fisher's exact test is too computationally intensive and you need the chi-square test, would try to "bin" the variables differently. That is, collapse categories until you have at least 5 in each cell. You could, alternatively, use Yates' correction to account for the undercounts in certain cells.

$\endgroup$
4
  • $\begingroup$ Sample size most of the time is around 1000 but can go to 1,500 (or more). What makes things 'worse' is the fact that even after computing fisher's exact test I would need to do all pair-wise comparisons (and FDR correction) in order to determine which pair was significant and to calculate the order ratio or another measure of effect size. With chi-squared this is very convenient for mxn matrix with the Cramer-V and residuals to determine which group is 'significant'; no need for pair-wise comparisons. $\endgroup$
    – dter
    May 20, 2016 at 16:38
  • $\begingroup$ I've not seen Yates' correction used in decades. Difficult to know how far that is fashion and how far logic. $\endgroup$
    – Nick Cox
    May 20, 2016 at 16:38
  • $\begingroup$ Let's assume the expected frequencies are all >=5 (although from what I've been reading this is not really 'true' link ), if the total sample size is say 50, would a chi square perform 'poorly' compared to a fisher? (since Fisher is said to be better for smaller sample sizes). $\endgroup$
    – dter
    May 20, 2016 at 16:48
  • $\begingroup$ For a one-off problem when assumptions are in doubt I see no great difficulty in trying both Fisher and chi-square tests. If they both agree, there's some reassurance; if they disagree you at least know to tread carefully. $\endgroup$
    – Nick Cox
    May 21, 2016 at 10:03

This site is temporarily in read-only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .