4
$\begingroup$

I'm a high school maths teacher and I was helping a pupil who had collected data for her project. I summerise the information in the table below.

$\begin{array}{c|cc|c} & Y & not Y & \\\hline X &360 & 83& 443\\ not X &46 &19 &65\\\hline & 406& 102& 508 \end{array}$

I advised her to do a chi-squared test to test the dependency between the variables X and Y. The table of expected values looks like:

$\begin{array}{c|cc|c} & Y & not Y & \\\hline X &354 & 89& 443\\ not X &52 &13 &65\\\hline & 406& 102& 508 \end{array}$

Comparing these two tables, my first thoughts were that there would be no association. Yet the calculated p-value is 0.048 which seems really rather small and goes against my intuition. It seems the bottom right cell is the one that is causing the effect (I suppose 6 objects from 19 is a fair proportion). I wondered if I have missed an assumption, and searched online and found (wikipedia) that the expected cell count should be no smaller than say 5 or 10, that is absolute values, but there is no mention made of relative count, which seems to me to be the deciding factor here.

Further, I played with the numbers a bit and found that the p-value is very susceptible to a small change, for instance if we remove one object from the bottom right cell the p-value jumps to 0.079.

So my questions:

  • Is there something wrong with this analysis?
  • Should we conclude that there is an association (assuming 5% significance)?
Improve this question
$\endgroup$
4
  • $\begingroup$ There is nothing wrong with your approach, the cells with small expecteds will dominate the total $\chi^2$. $\endgroup$
    – mdewey
    Dec 31, 2016 at 13:46
  • $\begingroup$ Related to this I can say that in general with contingency tables we should note that the chi square test is an asymptotic result and sparse cells can be an issue as to how well the chi-square distribution approximates the true distribution under the null hypothesis. What constitutes sparse cells? If I recall correctly William Cochran had a rule of thumb for this. $\endgroup$
    – Michael R. Chernick
    Dec 31, 2016 at 15:51
  • $\begingroup$ Actually take a look at the CV thread titled "For chi-square on any 2 by X contingency table should no more than 20% of the cells be less than 5?" I provided the Cochran rule of thumb which applies to general R by C contingency tables. Take a look at it. The thread also includes valuable references given by other contributors. $\endgroup$
    – Michael R. Chernick
    Dec 31, 2016 at 16:02
  • $\begingroup$ With regard to the direct questions from the OP I agree with mdewey that the methodology is correct. Given what has been said about sparse cells. You may consider not taking the p-values too seriously regarding sensitivity to small changes in the table. Both p-values are close to 0.05. So is the change really meaningful? I think not. $\endgroup$
    – Michael R. Chernick
    Dec 31, 2016 at 16:13

1 Answer 1

Reset to default
2
$\begingroup$

In the general, this Chi Square test is conservative even with expected values less than 5. This has been known for a long time. See

Bradley, D. R., Bradley, T. D., McGrath, S. G., & Cutcomb, S. D. (1979) Type I error rate of the chi square test of independence in r x c tables that have small expected frequencies. Psychological Bulletin, 86, 1200-1297.

Some recommend the Fisher Exact Test, but it is exact only if both marginal frequencies are fixed in advance. For example, the design may dictate that half are in Group A, half in Group B, and 50% pass and 50% fail. In general, the Fisher Exact Test is very conservative and not the preferred method. The Yates correction for continuity gives results very close to the Fisher Exact Test and is therefore also very conservative.

If your browser can run unsigned applets you may find this simulation informative: http://onlinestatbook.com/stat_sim/contingency/index.html

Improve this answer
$\endgroup$
2
  • $\begingroup$ In addition to the asymptotics I was thinking about mentioning Fisher's exact test. $\endgroup$
    – Michael R. Chernick
    Dec 31, 2016 at 16:04
  • $\begingroup$ @David Thank you for the response. Is it fair to say then that there is an association? $\endgroup$
    – Geoff
    Jan 3, 2017 at 13:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.