2
$\begingroup$

I would like to test if one procedure yields a significantly different success rate than the other. In procedure A, I have 2 successes and 19 failures. In procedure B, I have 2 successes and 38 failures. What kind of test can I use here because it appears that I don't have a large enough sample size for t-test for proportions .

$\endgroup$
5
  • $\begingroup$ You can use test for proportion $\endgroup$
    – SmallChess
    Mar 3, 2017 at 21:52
  • 1
    $\begingroup$ Don't you need at least 10 successes for that? $\endgroup$
    – V_ix
    Mar 3, 2017 at 22:45
  • 1
    $\begingroup$ I don't think any test will conclude those are different, even at very high significance level. $\endgroup$
    – Glen_b
    Mar 4, 2017 at 9:55
  • $\begingroup$ @Glen_b that's an interesting thought, and would probably make for a good answer here. $\endgroup$ Mar 4, 2017 at 16:16
  • $\begingroup$ @ssdecontrol I think I'd need to say more for an answer $\endgroup$
    – Glen_b
    Mar 4, 2017 at 17:11

1 Answer 1

2
$\begingroup$

D'Agostino, Chase, and Belanger published a paper in 1988 in which they find that the standard T test has adequate performance in small and unbalanced samples. Unfortunately that article is paywalled, but it is available on a certain "science hub" website.

The Fisher exact test can also be used to compare binomial proportions of any size, but carries an assumption that probably doesn't make sense here: there is a fixed number of successes available that can be distributed between groups. Therefore it is a "conditional" test, because it is conditional on this particular assumption about the data. In its place, it's fine, but D'Agostino et al. demonstrate that as an "unconditional" test it is unacceptably conservative, in that it will fail to detect truly significant differences too often.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.