3
$\begingroup$

I am hoping to perform a chi-square test of independence on data in a 2x2 contingency table with the following values:

Group A: 627 successes, 28 failures
Group B:  59 successes,  2 failures

I understand that this is a small collection of observations, and I wish to determine the minimum number of observations required for a chi-squared test at 0.05 significance, 0.8 power. However, it seems that the normal approximation of the binomial distribution that is generally used for power analysis among comparisons of proportions is not appropriate here for at least two reasons:

  1. p(success) is nowhere near 0.5 for the populations AND
  2. the expected number for Group B Failures, (2+59)*(2+28)/716 = 2.56, is below the guideline of 5 for all cells in chi-square power analysis.

Any constructive guidance on how to perform this test of independence (or reference to other questions I may have overlooked) is greatly appreciated.

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ By the time you get to a sample size as large as needed for your stated requirement, the normal approximation will be just fine. $\endgroup$
    – Russ Lenth
    Commented Feb 17, 2015 at 18:19
  • $\begingroup$ Thank you for your quick response, @rvl. Could you please clarify your meaning a bit? Specifically, are you saying that this sample size is large enough for use with power analysis? If so, how did you arrive at that conclusion? Thank you. $\endgroup$
    – ArcticHare
    Commented Feb 17, 2015 at 18:42
  • 3
    $\begingroup$ This is a FAQ: many, many people seem to confuse the rule of thumb that most expected counts should exceed 5 with a non-rule that most actual counts should exceed 5. Only the former is correct. You can find many references to this by searching our site. $\endgroup$
    – whuber ♦
    Commented Feb 18, 2015 at 16:24
  • 1
    $\begingroup$ Sorry it took me so long to respond, but NO, I definitely do not mean that you already have enough data. I mean that if the sample size is made large enough to meet you power requirements based on the normal approximation, the result will likely be a big enough $n$ for the normal approximation to work well. You can of course verify this by examining the expected counts and seeing if they meet the rule of thumb. $\endgroup$
    – Russ Lenth
    Commented Feb 19, 2015 at 20:35

1 Answer 1

Reset to default
2
$\begingroup$

@whuber is right that only the expected counts matter. Reading some of the threads returned by his search may help you; I discuss the issue here: For chi-square on any 2 by X contingency table, should no more than 20% of the cells be less than 5? You can also see the point made under "Expected cell count" on the Wikipedia page.

Note that the expected cell count is the probability times the N. Your effects are so small that huge N will be required. Here are screen shots of G*Power using your ratio of n's, or equal n's:

enter image description here

enter image description here

Your lowest probability is $3.3\%$, and your lowest count is $6187$, meaning that the expected count would be $204\gg 5$. Thus, @rvl is right: you won't have to worry about that assumption. (You will have to worry about needing so much data to have reasonable power, I suspect.)

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for this in-depth response, @gung. Some colleagues with more experience in this area have provided similar input. $\endgroup$
    – ArcticHare
    Commented Feb 19, 2015 at 13:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.