All Questions
Tagged with bonferroni false-discovery-rate
24 questions
0
votes
0
answers
24
views
Multiple hypothesis testing adjustment when only the number of significant tests is important
We are measuring the effect of a treatment T on a latent outcome variable (specifically, "Discrimination"). We have a number of outcome variables that we think do a reasonable job as proxies ...
2
votes
1
answer
85
views
Quantitatively distinguishing between multiple-comparison tests
As someone relatively new to applied statistics, I have been trying to better understand some of the best practices for applying statistical methods. I have been recently trying to understand when to ...
- 153
0
votes
0
answers
33
views
FDR q-values interpretation
In performing 43 test on two groups of individuals (independent t-test) I obtained 27 significant results, then adjusting the pvalues with BH procedure for controlling FDR I obtained the so-called q-...
- 381
1
vote
1
answer
457
views
FDR correction stronger than Bonferroni?
Is it possible to get a situation where, in a multiple-testing scenario, for some individual tests, the Benjamini&Yukatieli (2001)-correction ends up being more stringent than Bonferroni? I.e. ...
- 301
3
votes
1
answer
136
views
Does a procedure controlling FDR at level $\alpha$ always reject at least as much as a procedure controlling FWE at $\alpha$?
I understand the false discovery rate (FDR) is weakly less than the familywise error rate (FWE), and FDR is thus a less stringent way to control for type 1 errors. However, will a procedure that ...
- 543
1
vote
0
answers
52
views
Is there a way to adjust FDR threshold using correlation across tests (aka, number of estimated independent tests, like in Bonferroni correction)
For Bonferroni correction, correlations can be used to inform correction of multiple testing using the spectral decomposition of matrices. For example, in GWAS, the threshold is established to be 5e-8 ...
- 13
0
votes
0
answers
34
views
Multiple comparison when testing one planned hypothesis
I am running one logit model containing an interaction:
logit dv year contract_type yearXcontract+Z
where contract_type and year are dummies with 4 levels.
After ...
1
vote
0
answers
38
views
Corrections for multiple hypothesis testing
Apologies if this repeats a previous query, I have read a lot of prior threads (and made myself a bit cross-eyed with the ones asking whether we should correct for every hypothesis test ever performed!...
- 21
1
vote
1
answer
2k
views
Multiple comparison correction for confidence interval
I am trying to understand multiple-comparisons adjustment for confidence intervals. I came across the following links on the topic, but a few things are not clear to me.
Stanford lecture: Section 2 ...
- 113
7
votes
4
answers
4k
views
Rejection threshold of the Benjamini-Hochberg procedure
Is there a possibility to calculate or estimate the overall rejection threshold of the Benjamini–Hochberg procedure (BH)?
For the correction of the FWER using the Bonferroni method, the significance ...
- 131
1
vote
0
answers
26
views
Adjust for multiple comparisons in regression with one outcome variable and several independent variables?
How do I adjust for multiple comparisons using FDR with linear mixed effects
first regression:
...
- 147
12
votes
2
answers
4k
views
When do we "stop" using multiple correction techniques?
I understand when performing a simple t-test, we typically control the type -1 error rate at $\alpha = .05$. This signifies that if the null hypothesis holds, the data will "incorrectly" reject the ...
- 233
6
votes
2
answers
15k
views
Wilcoxon test with multiple testing: which correction for p values?
Please, I'm not very confident in statistics, and I'm trying to respond to a reviewer for a paper on the following issues:
In my experiment I observed 15 babies during a test where they were free to ...
- 223
2
votes
1
answer
48
views
Multiple hypothesis test
I have some questions about multiple hypothesis test.
Situation 1: We have 1000 p-values, all of them are less than 0.0 5. We may say there are 50 false positive (1000*0.05) in these p-values.
...
- 29
1
vote
1
answer
286
views
Should P-values of continuous variables be corrected?
Usually, when doing multiple tests to compare means of categorical variables, it is advised to do some correction of the P-values to control the probability or proportion of false positives (...
- 221
0
votes
1
answer
355
views
After discovering alpha error inflation, how to estimate severity of the effect?
Example: I find alpha inflation in an already published report. 300 indicators had been independently tested for statistical significance, and the "significant" results then reported as 95% confident ...
- 1
0
votes
2
answers
1k
views
FDR and Bonferroni corrections and Logistic Regression / Classification in High Dimensional Space
My work involves Classification --e.g. Logistic Regressions-- in a relatively High Dimensional setting (i.e. 300 to 1,500 variables).
I wonder if the Bonferroni and FDR corrections have any relevance ...
- 171
2
votes
0
answers
889
views
multiple test correction procedure for survey data
I clustered ~ 20,000 students into three groups. Now, I also have answers from some of those students to ~ 100 survey questions. I would like to compare the differences between the clusters regarding ...
3
votes
1
answer
14k
views
Bonferonni versus FDR
My question is rather simple: When should you use the Bonferroni correction over the FDR correction?
I've read about that the Bonferroni correction applies a more conservative approach in comparison ...
- 63
0
votes
0
answers
61
views
how to recover power lost due to correcting for multiple comparisons?
The right hand side plot descrives 7 treatments over the same 6 animals.
The groups are paired.
Measurements are log2 transformed (important for the design of the study)
The assumtions for repeated ...
- 131
4
votes
2
answers
2k
views
Number of false positives after Bonferroni or FDR
Let's say I am doing 100,000 tests. If the tests are independent, at an alpha of 0.05, we would expect (on average) 5000 of these tests to be false positives.
The Bonferroni method would penalize ...
- 193
33
votes
3
answers
4k
views
Why aren't multiple hypothesis corrections applied to all experiments since the dawn of time?
We know that we must apply Benjamini Hochberg-like corrections for multiple hypothesis testing to experiments based on a single data set, in order to control the false discovery rate, otherwise all ...
- 1,151
2
votes
0
answers
480
views
How is plot of p-values distribution related to calculation of the false discovery rate?
I'm running t-tests for all combinations of pairwise comparisons of average length of stay for patients among 30 hospitals.
My first approach was to use a Bonferroni corrected significance level of ....
- 6,286
9
votes
2
answers
918
views
Extent of multiple testing correction
A bit of a strange question. In my fourth year biostatistics class today, we were discussing when and when not to use multiple testing correction, and the professor made an offhand comment. He asked ...
- 2,640