Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [multiple-comparisons]

Signals situations where one is concerned about achieving intended power and size when more than one hypothesis test is performed.

42
votes
5answers
6k views

Why is multiple comparison a problem?

I find it hard to understand what really is the issue with multiple comparisons. With a simple analogy, it is said that a person who will make many decisions will make many mistakes. So very ...
10
votes
4answers
16k views

Multiple Chi-Squared Tests

I have cross classified data in a 2 x 2 x 6 table. Let's call the dimensions response, A and ...
20
votes
4answers
18k views

Post-hocs for within subjects tests?

What is the preferred method for for conducting post-hocs for within subjects tests? I've seen published work where Tukey's HSD is employed but a review of Keppel and Maxwell & Delaney suggests ...
20
votes
5answers
603 views

When can you use data-based criteria to specify a regression model?

I've heard that when many regression model specifications (say, in OLS) are considered as possibilities for a dataset, this causes multiple comparison problems and the p-values and confidence ...
27
votes
7answers
12k views

Variable selection procedure for binary classification

What are the variable/feature selection that you prefer for binary classification when there are many more variables/feature than observations in the learning set? The aim here is to discuss what is ...
4
votes
1answer
8k views

Power of Holm's multiple comparison testing compared to others [closed]

Following one-way ANOVA, there are many possible follow-up multiple comparison tests. Holm's test (or better, the Holm-Sidak) test has lots of power, but because it works in a stepwise manner, it ...