# What to do when assumptions are not met in a chi squared test?

I am doing an analysis on a “a priori segmentation”. In order to examine significant differences between the two segments crosstabs and the chi-square test were used.

Type of variables: categorical

Sample size: 253

Program used: SPSS version 21.

Problem: in several ways the assumption for a chi-square test is not met. Under some tables it is written (e.g.): 4 cells (22.2%) have expected count less than 5. The minimum expected count is 1.85 or 7 cells (50.0%) have expected count less than 5. The minimum expected count is .26. or 10 cells (62.5%) have expected count less than 5. The minimum expected count is .26.

What should be done in this case? Should the chi-square test be avoided? Which test could be used instead? The literature says that for 2x2 contingency tables a Fisher’s exact test can be used. What should be done with bigger contingency tables (variables that entail several categories)?

-
You could use Fisher exact test (?). –  user17316 Nov 29 '12 at 6:52
Please, consider giving more details to make this comment a really useful answer, in particular how Fisher test would apply in this context. –  chl Nov 29 '12 at 10:40

One solution would be to use a bootstrap test as an approximation to a permutation test. Permutation tests are exact and most powerful; in this case there are too many permutations to calculate every one of them, so you'd approximate the test with the bootstrap.

Basically, you:

1) Calculate your test statistic, label it $T_0$, on the actual data, say for illustrative purposes the same chi-square statistic you've already calculated,

2) Construct 1,000 or 10,000 or so ("many") random contingency tables under the assumption the null hypothesis is true, and for each one calculate the chi-square statistic, label them $T_1 \dots T_B$.

3) Compare your test statistic's value $T_0$ with the the test statistic values $T_1 \dots T_B$ from the randomly-generated contingency tables, and see what fraction are more extreme than $T_0$; this gives you a bootstrap p-value.

We are approximating the distribution of the test statistic under the null hypothesis by randomly generating a lot of values for the test statistic under the null hypothesis; this lets us estimate the p-value associated with the value of the statistic we actually observed.

I can't help you with the SPSS part of this, unfortunately.

Here's a reference which I've found helpful in the past: Permutation, Parametric, and Bootstrap Tests of Hypotheses (Good).

-
Very nice answer, +1 - especially the reference to Good's book, which I also like. Of course, constructing random contingency tables is not entirely trivial - this link may be helpful: stackoverflow.com/questions/948327/… –  Stephan Kolassa Nov 28 '12 at 20:31
@StephanKolassa - nice link (+1), and thanks! –  jbowman Nov 28 '12 at 21:22
+1. Two details: As the null hypothesis is not completely specified, the proximal null hypothesis - the one that best fits the data - should be used. And generating random tables involves a decision about which margins to condition on. –  Scortchi Nov 29 '12 at 17:02

In the SPSS crosstabs procedure you can use an exact test and/or you can use the bootstrap. Exact offers a Monte Carlo and exact choice. Bootstrap offers the same bootstrapping technique available for a bunch of procedures. Both are available from the Crosstabs dialog if you have the relevant add-on options.

-