# Minimum population size for chi-squared test?

I'm analyzing data from an experiment in which two independent groups were exposed to an experimental setup without and with treatment.

I am testing whether the treatment changed the second group's behaviour by performing a chi-squared test that compares group 2 (the observed) vs group 1 (the expected). The result indicates there is a significant change in behaviour X² p-value < 0.00014.

Now, I am trying to test subgroups to understand better the change, i.e., looking at gender, age, and other self reported metrics.

My question is, given that group 2 N=40 if I look at age for instance I find people in their 20s and their 60s show significant change but other age groups don't. However people in their 20s N=12 and people in their 60s N=5. Is there a heuristic / rule that says there is a minimum number of people needed to consider a result significant? For instance anything below N=5 cannot be considered significant or anything below N=20% of the population?

EDIT: Just to clarify, I am doing a chi-squared test of independence (between group 1&2) not a chi-square goodness of fit test.

EDIT 2: With this edit I consider the question closed. None of the answers / comments gave me a definitive solution, which I believe says more about the question than the answers. I was hoping for a definitive answer along the lines you need at least 5 people or 20% of your sample. It seems the answer is less direct as it is sensitive to many factors.

• I don't understand what you mean by "expected" -- I thought that group 1 was also collected data, and not theoretical distribution? Commented Nov 16, 2012 at 8:52
• Group one is not exposed to treatment but yes to environment so their behaviour is what is expected when the treatment is not present in a given environment. Once you introduce the treatment then you compare the behaviour in the same environment pre and post treatment. The resulting change is the observed effect of treatment in that environment. This is standard procedure in HCI/social sciences research. Commented Nov 16, 2012 at 11:26
• You are going to confuse a lot of people if you use "expected" in that way, as it means something else when talking about chi-square tests. Commented Nov 16, 2012 at 11:54
• Why are you using chi-square at all? You have some dependent variable, apparently dichotomous (although you haven't said) and several independent variables (treatment, age, gender etc). That calls for regression of some sort, probably logistic regression if I am right about the DV. Commented Nov 16, 2012 at 11:58
• I'm doing chi-square because I need to look at each variable in isolation not as a group, in which case I would probably do an ANOVA. Commented Nov 16, 2012 at 14:31

For small sample sizes, use Fisher's exact test, because the $\chi^2$ test sampling statistics has only approximately the $\chi^2$ distribution, and this approximation is problematic for small sample sizes.