I'm analyzing data from an experiment in which two independent groups were exposed to an experimental setup without and with treatment.
I am testing whether the treatment changed the second group's behaviour by performing a chi-squared test that compares group 2 (the observed) vs group 1 (the expected). The result indicates there is a significant change in behaviour X² p-value < 0.00014.
Now, I am trying to test subgroups to understand better the change, i.e., looking at gender, age, and other self reported metrics.
My question is, given that group 2 N=40 if I look at age for instance I find people in their 20s and their 60s show significant change but other age groups don't. However people in their 20s N=12 and people in their 60s N=5. Is there a heuristic / rule that says there is a minimum number of people needed to consider a result significant? For instance anything below N=5 cannot be considered significant or anything below N=20% of the population?
EDIT: Just to clarify, I am doing a chi-squared test of independence (between group 1&2) not a chi-square goodness of fit test.
EDIT 2: With this edit I consider the question closed. None of the answers / comments gave me a definitive solution, which I believe says more about the question than the answers. I was hoping for a definitive answer along the lines you need at least 5 people or 20% of your sample. It seems the answer is less direct as it is sensitive to many factors.