2
$\begingroup$

I'm trying to look across a dozen or so groups of cases with respect to a condition they may have or not. I have a predetermined expected frequency for each condition status (c1 & c2, let's assume yes/no)

Now, I'm performing a simple test of significance between the observed and expected values per group as if each were a 2x2 contingency table. In cases where all values are above 5, I perform a chi-square test and all other cases I perform Fisher's exact test of significance.

Is it incorrect to perform different tests per group? I'm not necessarily interested in comparing between groups but rather trying to determine if the difference between c1 and c2 is significant from what would be expected in each condition per group.

Would it make more sense just to use Fisher's exact test on everything?

Here's a mock representation of my data. group1 and 3 use Fisher's test as there are some values below 5, while groups 2 and 4 use Chi-square.

        c1.obs  c2.obs  c1.exp  c2.exp  p.value
group1  3       21      2       22      1
group2  9       48      7       60      5.5E-7
group3  13      19      3       29      8.1E-03
group4  34      182     22      194     4.9E-03 
...
Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ It's unlikely that you should treat observed and expected as a 2x2 table though it depends on what the expected actually are. How are these expected numbers obtained? Why are they integer? $\endgroup$
    – Glen_b
    Commented Apr 11, 2015 at 2:28
  • $\begingroup$ Let's assume c1 is YES and c2 is NO. The values I showed are actually rounded. They are not really integers in my data. Also, the expected values are a fixed percentage across all groups, for example 10% of elements per group. $\endgroup$
    – rvidal
    Commented Apr 12, 2015 at 3:21
  • $\begingroup$ When you say 'values are rounded' do you mean the expected are rounded, or also the observed? How were the fixed percentages obtained? $\endgroup$
    – Glen_b
    Commented Apr 12, 2015 at 3:24
  • $\begingroup$ Expected are rounded. Observed are actual frequency counts. The percentage is an established value used per group. It's expected to encounter 10% c1 (aka YES) cases per group. $\endgroup$
    – rvidal
    Commented Apr 12, 2015 at 3:33
  • $\begingroup$ How is that value established? Why is 10% expected? Where do the expected numbers come from? It matters. Maybe they come from another (possibly larger) sample, for example. Maybe they come from some theory. Whatever it is, some of the different possible cases are different. It's still not clear to me how you get a 2x2 table from each group unless you treat expected as observed. $\endgroup$
    – Glen_b
    Commented Apr 12, 2015 at 4:10

2 Answers 2

Reset to default
3
$\begingroup$

As far as I can tell from the comments on the original post, analyzing a 2 x 2 contingency table doesn't make sense for these data. Instead you would want to compare the count of c1 out of the total observed count, and compare that to 0.10.

You might use an exact binomial test or a chi-square test goodness-of-fit test. So in R, for group 3:

yes = 13
no  = 19
binom.test(yes, (yes+no), 0.1)

   ### number of successes = 13, number of trials = 32, p-value = 5.507e-06
   ### alternative hypothesis: true probability of success is not equal to 0.1
   ### 95 percent confidence interval:
   ###  0.2369841 0.5935508
   ### sample estimates:
   ### probability of success 
   ###                0.40625 

yes = 13
no  = 19
chisq.test(c(yes, no), p=c(.10, .90))

   ### Chi-squared test for given probabilities
   ### 
   ### X-squared = 33.347, df = 1, p-value = 7.709e-09
Improve this answer
$\endgroup$
0
$\begingroup$

As far as I know you can use chi-square throughout, unless the expected counts are less than five in which case you can use Fisher's exact test. I'm not sure but I think Fisher's Exact Test is slightly more conservative in its resulting p-value, i.e., it's slightly less likely to show statistical significance (but I'm not sure of that!) Some texts say that if any expected frequencies are less than 2, or if more than 20% of the expected frequencies are <5, then don't use chi-square but it's okay to use it otherwise.

You can also combine cells or collapse categories together in your tables if this is possible, to raise the expected frequencies to greater than 5.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.