I am conducting some statistical tests on a rare event (11 cases out of 171 patient sample). I have two main hypotheses, but in the mean time I also want to conduct some exploratory tests on 6 other variables, in order to explore possible associations.

(1) I am wondering if there are any rules about the number of tests that you can do based on the number of events.

(2) Normally when we do one or several statistical tests, we consider the statistical power (which is related with sample size), and with several tests we also consider the multiple comparisons problem (which is related with the number of tests). In the case of rare events, the statistical power of each of my test would be low, but will rare event also influence (or exaggerate) the multiple comparisons issue, i.e. after conducted several tests, the chance of finding false positives are much higher for rare events?

  • 2
    $\begingroup$ To my way of thinking 11 events out of 171 cases is not very rare. It is greater than 0.005. If the analysis is exploratory and has nothing to do with your main hypotheses I see no reason to put a limit on the number of tests to look at. Also multiple comparisons is not an issue because you wouldn't try to interpret p-values in an exploratory analysis. In your case if you can enroll 10,000 patients (not uncommon in clinical trials especially for pharmaceuticals) you can have reasonably powerful tests for this level of rarity. $\endgroup$ – Michael R. Chernick Sep 17 '12 at 16:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.