# Questions tagged [false-discovery-rate]

An expected fraction of rejected null hypotheses that are falsely rejected, i.e. the fraction of significant findings that are actually not true. One method to control FDR in multiple testing is Benjamini-Hochberg procedure.

263 questions
Filter by
Sorted by
Tagged with
1 vote
14 views

### Adjust p-value of the all output or just the variable of interest?

I am running linear mixed effect models using the lme4 package in R with the following output. ...
• 11
38 views

### calculating expected value of false discovery rate - Benjamini-hochberg

I recently got a question that cannot seem to solve, as a beginner in the statistical field. assume we have 20 tests. 5 with the p-value of 1/330 5 with the p-value of 1/150 10 with the p-value of 1/...
26 views

### Do the p-values produced by lmerTest::lmer need to be adjusted for multiple comparisons?

I am running a fairly simple linear mixed model with multiple observations per patient, comparing a lab value to a reference group (healthy control). Here's some fake data: ...
• 1
13 views

### Applying threshold defined by false discovery rate to new data

I'm thinking how to correctly apply the results of statistical analysis from one experiment to new data. Suppose I have a data of two groups, n samples/instances ...
• 185
16 views

### How to adjusting p-values from results of trend analysis on water quality data

I have results from Mann Kendall test for trends at 100 sample locations for 5 different analytes at each location (i.e., 500 p-values). If I had the trend results from 500 sample locations for 1 ...
103 views

### Q-value for FDR correction in R stats package

I used FDR correction (i.e., Benjamin-Hochberg tests) from the R 'Stats' package, but I'm confused of the q value used in the package. The notations in the 'p.adjust' function does not include the ...
• 69
6 views

### How can one control for FDR if we do not know the prevalence?

I am trying to understand the concept of FDR, and particularly "controlling" the FDR in multiple comparison testing. I undesratdn, and have used many times, the concept of α (type 1 error ...
18 views

### When running BH Procedure and there are different types of statistical tests, are they all included?

Let's say I'm running 100 statistical tests. Some of them are t tests, some are ANOVAs (and for each ANOVA subsequent post-hoc t tests should a significant result be obtained), some of them ...
5 views

### After Dunn test with 4 groups, I got 6 p-values for 6 combinations. Can I just choose 4 of 6 p-values to get lower FDRs?

After Shapiro-test(p-value<0.05) and Kruskal-Wallis null rejection, Someone has done the Dunn test with 4 groups, He got 6 p-values for 6 combinations. Can he just choose 4 of 6 p-values to get ...
1 vote
36 views

### FDR correction stronger than Bonferroni?

Is it possible to get a situation where, in a multiple-testing scenario, for some individual tests, the Benjamini&Yukatieli (2001)-correction ends up being more stringent than Bonferroni? I.e. ...
38 views

### Should I perform multiple testing correction (FDR) for all the P-values regardless if they derive from Ttest or Wilcoxon?

There are 100 participants and I want to test if 50 metabolites change before and after intervention (antiobesity drug). First, I check the normality for each variable, and then I perform a T-test or ...
• 1
1 vote
29 views

### False rejection control techniques review

As far as I know, in multiple hypothesis testing scenarios there are methods to control for the False Discovery Rate (FDR) and the Family-Wise Error Rate (FWER) but I'm not aware of any method for ...
• 133
16 views

### Why is the distribution of the FDR corrected P-values shifted so drastically towards high values?

I trying to understand why my distribution of the FDR-corrected P-values (Q-values) is shifted so drastically to the high values. In this figure, on the left is the distribution of the (raw) values ...
17 views

### When p equals the alpha value, should we reject H0 when correcting for multiple comparison using false discovery rate?

I am using statsmodels in Python. I found a case where a p-value corrected with false discover rate (using Benjamini & Hochberg method) was corrected to be ...
37 views

### Question about power of FDR and FWE

I understand that false discovery rate (FDR) is $\leq$ familywise error rate (FWE). I have read this means that controlling FDR is hence "more powerful" than controlling FWE. What does ...
• 353
48 views

### Does a procedure controlling FDR at level $\alpha$ always reject at least as much as a procedure controlling FWE at $\alpha$?

I understand the false discovery rate (FDR) is weakly less than the familywise error rate (FWE), and FDR is thus a less stringent way to control for type 1 errors. However, will a procedure that ...
• 353
43 views

### Which p-values do I use for FDR in 3 separate multiple linear regression models?

I'm trying to carry out Multiple Hypothesis Testing Correction using adjusted p-values (FDR Method) from my multiple linear regression models. I am testing 11 independent variables of interests ...
16 views

### False Discovery Rate From Posterior Samples of Bayesian Models

This is likely a naive question but here goes. There are a number of papers on Bayesian FDR (Efron/Efron and Tibshirani) etc. All of these models seem to assume that the score(p value or z values, or ...
306 views

### How many coin flips are needed to reliably know a coin of weight w is unfair?

I want to find out how many flips I need to flip a coin to reliably know that it is an unfair coin. The issue is that as the coin becomes closer to 50/50, the more false-negatives you will have if you ...
20 views

### FDR in multiple comparisons with the same dependant variable

Let's say I have a set of 200 models that follows the structure below, my goal being to identify the genes that are significantly related to the response variable Response ~ Gene[i] + Age + Time + ...
21 views

### How to compute p-values for shrinkage estimates of partial correlation?

I am using the pcor.shrink function from the corpcor R package to compute the shrinkage estimates of partial correlation ...
• 647
19 views

### How to control for multiple testing with multiple hypotheses and multiple exploratory analyses for each hypothesis testing

I recently conducted a 3 (between-subject: two intervention groups and one control group) x 5 (within-subject: 5 dependent variables) field experiment in which two behavior interventions (indenpendent ...
• 31
28 views

### Is the FDR linear-step up procedure Benjamini and Hochberg valid for empircal p-values from non-parametric tests (e.g., bootstrap)?

I have empirical p-values from bootstrapped hypothesis tests with 10,000 resamples each. Because there are a finite number of resamples, some of the p-values are "0", i.e., < 0.0001 with ...
• 51
24 views

### Analysis sense check: preregister planned contrasts which we wish to conduct even if associated ANOVA is nonsignificant?

I could use a sense check on my analysis plan for an experimental study which has not yet been run; I will preregister it once I am confident that it is reasonable. There are seven between-...
• 683
1 vote
22 views

### Is there a way to adjust FDR threshold using correlation across tests (aka, number of estimated independent tests, like in Bonferroni correction)

For Bonferroni correction, correlations can be used to inform correction of multiple testing using the spectral decomposition of matrices. For example, in GWAS, the threshold is established to be 5e-8 ...
35 views

### How do I correct for multiple testing when I have multiple models?

I have 12 concentrations that I would like to compare between 2 groups, some subjects have 2 measures. I know the concentration is also depending on age and sex so I added those to the linear model ...
• 169
25 views

### Multiple comparison when testing one planned hypothesis

I am running one logit model containing an interaction: logit dv year contract_type yearXcontract+Z where contract_type and year are dummies with 4 levels. After ...
76 views

• 46.2k
1k views

### Rejection threshold of the Benjamini-Hochberg procedure

Is there a possibility to calculate or estimate the overall rejection threshold of the Benjamini–Hochberg procedure (BH)? For the correction of the FWER using the Bonferroni method, the significance ...
• 121