2
$\begingroup$
  1. From Wikipedia

    Traditional methods for multiple comparisons adjustments focus on correcting for modest numbers of comparisons, often in an analysis of variance. A different set of techniques have been developed for "large-scale multiple testing", in which thousands or even greater numbers of tests are performed. ... Particularly in the field of genetic association studies, there has been a serious problem with non-replication — a result being strongly statistically significant in one study but failing to be replicated in a follow-up study. Such non-replication can have many causes, but it is widely considered that failure to fully account for the consequences of making multiple comparisons is one of the causes.

    I was wondering what the problem with non-replication is, and why it can be because of the failure to fully account for the consequences of making multiple comparisons?

  2. For large-scale testing problems where the goal is to provide definitive results, the familywise error rate remains the most accepted parameter for ascribing significance levels to statistical tests.

    Alternatively, if a study is viewed as exploratory, or if significant results can be easily re-tested in an independent study, control of the false discovery rate (FDR) is often preferred.

    What does "the goal is to provide definitive results" mean, and why is FWER most accepted in such cases?

    What does "if a study is viewed as exploratory, or if significant results can be easily re-tested in an independent study" mean, and why is FDR preferred in such cases?

  3. The FDR, defined as the expected proportion of false positives among all significant tests, allows researchers to identify a set of "candidate positives", of which a high proportion are likely to be true. The false positives within the candidate set can then be identified in a follow-up study.

    Since FDR "allows researchers to identify a set of 'candidate positives'", which I understand as the estimate of false positive individual nulls, why do we need further that "the false positives within the candidate set can then be identified in a follow-up study"?

    Am I right that FWER also can "allow researchers to identify a set of 'candidate positives'", as the set of individual tests that rejects their nulls.

Thanks and regards!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.