# Tag Info

### Many p-values are equal to 1 after Bonferroni correction; is it normal?

Nothing went wrong. The adjusted p-values are correct. Adjusted $p=1$ simply means no evidence at all for rejecting the null hypothesis. However ...
• 13.2k
Accepted

### Correction for multiple testing in Multiple regression analysis

(NB: I started to write this before the question was edited; I don't comment here on the "empirical aspect", i.e., what is done in the literature, but rather on what "should" be done.) This is a good ...
• 26.2k
Accepted

### When do we "stop" using multiple correction techniques?

Multiple comparisons corrections are intended to control the familywise error rate--or something like it--so they should be applied across a "family" of related hypothesis tests. In your first ...
• 21.3k

### Many p-values are equal to 1 after Bonferroni correction; is it normal?

Thanks for reading my 1997 JASA paper! If I had a do-over, I would rephrase my comment that a (single-step) Bonferroni adjusted p-value is not a probability "per se." (And I would no longer use the ...
• 5,981

The question is why, not what does the manual say? The answer is that alpha and the P value are different things, and they work in opposite ways. When you get a P value, you are asking "at what ...
• 20.8k
Accepted

### Bonferroni corrected p-values that exceed 1: possible or non-sense?

Whether you divide the thresholds by a number or multiply the $p$ values by the same number is mathematically completely equivalent: $$p\leq \frac{\alpha}{N} \quad\iff\quad Np\leq\alpha.$$ So there ...
• 128k
Accepted

### Applying the Benjamini-Hochberg correction with a low number of comparisons

These procedures address multiplicity in very different ways. Bonferroni but also others like Holm's step-down, Hochberg's step-up, and Hommel's step-up will (in order from least to most powerful) ...
• 5,133
Accepted

### Reducing threshold p-value in case of statistical tests following data exploration

Peeking at Data I don't think you have any business tinkering around with $p$ values if you are doing exploratory data analysis, as this can be a form of hypothesizing after results are known (HARKing,...
• 15.9k

### Am I right that the Bonferroni correction does NOT apply to randomness testing?

Adjustments for multiplicity or comparisons are not often helpful, in my opinion, and in the context of your question they are a very bad way to protect against false positive errors. As the datasets ...
• 16.1k

### multiple linear mixed models - multiple comparisons adjustment

As Peter Flom said, there are many reasonable points of view to what one should do in this situation, and no one correct answer. However, since you are after concrete advice, I can say what I'd do in ...
• 2,438

### Many p-values are equal to 1 after Bonferroni correction; is it normal?

Just to add to @gordon-smyth and @student-t 's answer. Another way to look at it is to adjust the $\alpha$-level yourself instead of adjusting the $p$-values via the ...
• 6,531

### What is the right Bonferroni adjustment?

To the best of my knowledge, the Bonferroni correction is based on the number of tests you actually perform, not on the total number of pairwise tests that you could perform. So if you set out to ...
• 235
Accepted

### p value correction in multiple outcomes study

Standing on PWCs You may find that there are alternative views on pairwise tests, particularly with cases like yours where there are a substantial number of mean comparisons being made. Some dictate ...
• 15.9k

### p value correction in multiple outcomes study

It sounds like you are performing a preliminary or characterisation study. In that situation you should not 'correct' for multiple comparisons at all. You should see what the data say and then design ...
• 16.1k

### multiple linear mixed models - multiple comparisons adjustment

As Jacob Cohen put it, decades ago, "this is a subject on which reasonable people can differ." This has also been discussed many times here, in various contexts. See the tag "multiple-...
• 125k

### What's wrong with Bonferroni adjustments?

Thomas Perneger is not a statistician and his paper is full of mistakes. So I wouldn't take it too seriously. It's actually been heavily criticized by others. For example, Aickin said Perneger's paper ...
• 311

### Sidak or Bonferroni?

Sidak and Bonferroni are so similar that you will probably get the same result regardless of which procedure you use. Bonferroni is only marginally more conservative than Sidak. For instance, for 2 ...
• 369
Accepted

### Bonferroni Correction & machine learning

There is a degree to which what you are talking about with p-value correction is related, but there are some details that make the two cases very different. The big one is that in parameter selection ...
• 1,212