# Are my tests of significance valid?

I have results from three archaeological populations in which I found that the prevalence of a certain disease was:

65% (13/20) in population A,

31.25% (5/16) in population B and

46.60% (48/103) in population C

I did a chi-squared test and found that while population A was significantly more affected than population B (χ2 = 4.05, df=1, P= 0.04), population A was not significantly more affected than population C (χ2 = 2.27, df=1, P= 0.13)

However, I then learned that you can test each population (2x1 contingency table) to see if its results are significant. The sample sizes are quite small and these tests have shown that the results from each population are not significant (e.g. SPSS tells me that the fact that 5 out of 16 individuals were affected by disease in population B is not significant: χ2 = 1.80, df=1, P= 0.18) . Does then render the comparison of populations meaningless?

No, those two sorts of chi-square tests do different things.

The first sort (disease by population) tests whether disease is associated with population. That sounds like what you wanted to find out.

The second test would test whether the proportion of disease in a particular population is 50%. That is clearly NOT what you want to find out.

Given that you have 3 groups, you might try a 3x2 chi-square. You might also try (if you have learned how) a logistic regression with the DV being disease and the IV being group.

• Thank you very much for your answer! You are right that I wanted to know if disease is associated with population and, on that basis, I had actually done a 3x2 chi-square. It suggested that they were not associated (χ2 =4.18,df=2, P=0.12). The reason I did the 2x2 chi-square tests was to separate out the populations in the hope of seeing if population A could be said to be the 'worst' affected. However, my follow-up question, if I could venture to ask it, would be what can be concluded from the fact that the 2x3 test suggests no association and one of the 2x2 tests suggests the opposite – Denise Jan 16 '14 at 13:10
• The 3x2 chi square is less powerful than each 2x2 chi square. Thus, it was not significant even though one of the 2x2 ones was. Some people (but by no means all) would suggest that you shouldn't do the 2x2 tests after a nonsig 3x2. – Peter Flom - Reinstate Monica Jan 16 '14 at 13:15
• Perfect! So clear! Thank you so much for your replies and your patience with a beginner! – Denise Jan 16 '14 at 13:19