# Questions tagged [false-discovery-rate]

An expected fraction of rejected null hypotheses that are falsely rejected, i.e. the fraction of significant findings that are actually not true. One method to control FDR in multiple testing is Benjamini-Hochberg procedure.

117 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
582 views

### How is it possible to control false discovery rate (FDR) without knowing the power and the prevalence of the nulls?

If we have p-value 0.05, to calculate probability of our discovery to be false positive, we need to use complex formula with prevalence (prior probability) and statistical power. If we have lots of ...
• 1,731
144 views

### A puzzling observation by Bradley Efron in his article in Science regarding Bayes’ Theorem in the 21st Century

Mr. Effron has published an interesting article in Science magazine with the enticing title "Bayes' Theorem in the 21st Century". The article is quite short and can be found here: http://web.ipac....
• 245
5k views

I know the p-value and I may know what FDR (false discovery rate) do and its goal. But I confuse between q-value (often known as FDR) and adjusted p-value (p.ajust ...
788 views

### Practical meaning of the PRDS property in Benjamini-Hochberg procedure under dependence

I am reading paper by Benjamini and Yekutieli (2001) on controlling FDR under dependence. My question is to figure out, in practical applications, whether the PRDS property is fulfilled in a given ...
• 7,567
1k views

### How to estimate False Discovery Rate from p-value distribution?

I have learned many models and I calculated p-values for the cross-validation errors. I want to select significant models based on the false discovery rate (FDR). How can I estimate the FDR from p-...
• 2,723
662 views

### FDR and the the Benjamini-Hochberg Method

I am trying to understand the Benjamini-Hochberg Method for controlling the false discovery rate. Mathematically, if we are given with m hypothesis testing procedures, we sort the P-Values and reject ...
439 views

### The method of knock-offs by Barber & Candes for variable selection and FDR control

The knock-off method is a recent approach to variable selection and FDR control presented in two papers to be found here https://statweb.stanford.edu/~candes/papers/FDR_regression.pdf and here https://...
• 171
196 views

### R : qvalue package - smoother or bootstrap method for pi0 estimation

I've a bunch of 500 p-values and I want to estimate the pi0 (proportion of true H0). When using the qvalue R package, two method can be used (smoother or bootstrap). Both giving very different results ...
584 views

### Multiple testing correction alternatives

I have a data set with about 6,000 Fishers exact tests. If I want to do a multiple testing correction to control the FWER what are the advantages of one of these methods vs the other calculating a q-...
457 views

### running multiple t-tests across large dataset

I'm trying to work out if a t-test is the most appropriate in this situation: I have a data frame which looks like the one below but my data frame has aprox 37,000 rows. I'd like to run a t-test on ...
• 173
123 views

### Does a procedure controlling FDR at level $\alpha$ always reject at least as much as a procedure controlling FWE at $\alpha$?

I understand the false discovery rate (FDR) is weakly less than the familywise error rate (FWE), and FDR is thus a less stringent way to control for type 1 errors. However, will a procedure that ...
• 543
138 views

### Correct Sample Space for Calculating P-Value

Suppose there are two variables, x and y, and we wish to calculate some test statistic (say the Pearson correlation), and estimate its significance. The pvalue is calculated by estimating the ...
• 4,590
897 views

### p-value distribution and the applicability of different multiple testing correction methods

I came across this issue as I was analysing some genomic data sets looking for differential DNA methylation. I have generated p-values from the comparisons I am making, and am now looking into the ...
• 225
33 views

### False rejection control techniques review

As far as I know, in multiple hypothesis testing scenarios there are methods to control for the False Discovery Rate (FDR) and the Family-Wise Error Rate (FWER) but I'm not aware of any method for ...
• 460
455 views

• 11
1 vote
15 views

### Should one correct for multiple comparison for secondary outcomes in RCTs or clinical trials?

This might a simple question answered elsewhere, but it comes up often in NHST discussions and I've heard conflicting statements. Imagine a metanalysis of 50 trials on 100,000 patients total to ...
• 323
1 vote
98 views

### Algorithm Validation and Alternative Approaches for FDR Control in Combined Datasets

We are currently working on a project involving multiple datasets reanalyzed with the same pipeline, where all data points (patient feature) have been filtered at 1% False Discovery Rate (FDR). As ...
1 vote
15 views

### Calculating FDR in batches while taking into account the total number of tests

I am running a large number (potentially hundreds of millions - billions) null hypothesis significance tests (specifically a Poisson test, but the question is general). I would like to apply to them ...
• 121
1 vote
14 views

### FDR Correction needed on a pixel-wise comparison?

I have a question, which twirls my mind but I can't find a robust basis to answer it! I have a time-frequency data (let's say 5 columns x 5 rows) for each participant in two different groups. I have ...
1 vote
28 views

### How to Determine the Number of Independent Tests When Considering Multiple Testing

I have a question regarding how to determine the number of independent tests when performing multiple testing corrections. Suppose my research hypothesis is whether age is related to cognition decline ...
• 347
1 vote
88 views

### Can the critical P-value returned by the Benjamini-Krieger-Yekutieli (BKY) procedure be greater than the false discovery rate?

If I use the Benjamini-Krieger-Yekutieli (BKY) procedure for an FDR correction, is it possible for the critical P-value returned to be greater than the desired false discovery rate? I just tried ...
• 61
1 vote
24 views

### Exploratory factor analysis (EFA) for transcriptomic data

I am doing an EFA for transcriptomic data (n=202, p=190). I did log-transformed the data because of skewness. My question is, do I have to do false discovery rate (FDR) analysis at all prior to my EFA....
• 11
1 vote
69 views

### How do I use Benjamini-Hochberg procedure for post-test segmentation in A/B-tests correctly?

I am currently working on evaluating the results of an A/B-test. For this, I am also looking into some relevant post-test segments. Overall, I have no statistically significant difference between ...
• 11
1 vote