Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [false-discovery-rate]

An expected fraction of rejected null hypotheses that are falsely rejected, i.e. the fraction of significant findings that are actually not true. One method to control FDR in multiple testing is Benjamini-Hochberg procedure.

0
votes
0answers
5 views

Should filtering be done before or after correction for multiple testing?

I am trying to find allele-specific bias in some biological data. An equivalent problem that doesn't require understanding biology would be trying to determine which coins, if any, in a giant bucket ...
1
vote
0answers
30 views

Estimating the False Reject Rate (FRR) of a classifier in production

I have trained a binary classifier which runs in production on remote distributed devices (which are out of my control). The model was trained on positive and negative samples, and I have chosen the ...
4
votes
0answers
53 views

How is it possible to control false discovery rate (FDR) without knowing the power and the prevalence of the nulls?

If we have p-value 0.05, to calculate probability of our discovery to be false positive, we need to use complex formula with prevalence (prior probability) and statistical power. If we have lots of ...
0
votes
0answers
23 views

False Discovery Rate (FDR) q-value

If tests are not completely independent (as in the case of 30 proteins tested for associations, partly correlated to each other, and measured by a multiplex immunoassay) it is justified to consider ...
0
votes
0answers
65 views

What is the difference between q-value and adjusted p-value (p.adjust)?

I know the p-value and I may know what FDR (false discovery rate) do and its goal. But I confuse between q-value (often known as FDR) and adjusted p-value (p.ajust ...
0
votes
0answers
6 views

How to combine Benjamini-Hochberg with statistical power?

Suppose I am investigating a question which involves many statistical T-tests. The normal Benjamini-Hochberg procedure tells me how to control the false discovery rate. However, suppose that some or ...
0
votes
0answers
22 views

A minimum False Discovery Rate of 0.8 when doing multiple hypothesis testing on Logistic Regression Coefficient p-values?

I currently have 8,000 predictor variables and a binary indicator response variable. I then did a logistic regression on each of the predictor variables, i.e., response regressed on predictor variable ...
0
votes
0answers
6 views

What is the difference between the FDR in benjamini-hochberg/bonferonni vs a local FDR?

I am wondering if the local FDR in Efron's literature is different than the FDR associated with Benjamini-Hochberg and if it is perhaps talking about something else.
0
votes
0answers
14 views

Grouping hypotheses, and applying FDR to each group separately?

Suppose I have two sets of hypotheses, $A$ and $B$, and I want to control FDR. When is it valid to control the FDR of $A$ and $B$ separately, vs controlling the FDR of $A \cup B$ jointly? One ...
1
vote
1answer
47 views

False discovery rate in binary logistic regression analysis?

I am analyzing a population-representative cross sectional survey to identify associations between several factors and cardiovascular disease. I am performing binary logistic regression analysis with ...
0
votes
0answers
18 views

FDR control without P-values

I classify $n$ objects into 2 categories, using some criteria and threshold $\alpha$. To estimate the number of false discoveries, I create a sample of $n$ permuted objects (keeping covariance ...
1
vote
0answers
50 views

FDR for Bayes Factors

I have a dataset, 2 models $M_0$ and $M_1$. Most of my events belong to model $M_0$, so I detect events, belonging to $M_1$, only if the corresponding Bayes factor is bigger than 20. So far so good, ...
0
votes
1answer
34 views

How to compensate for different statistical power in multiple comparisons? [closed]

Imagine we are performing 10 comparisons on a single variable of choice. The 10 respective experiments have different statistical powers. Now imagine that the least powerful and the most powerful ...
1
vote
0answers
24 views

Pairwise non-parametric test?

I have 5 groups of patients (different sizes), for which I measured some property that is continuous but not normally distributed. I would like to compare every pair of groups in respect to this ...
0
votes
0answers
54 views

Linear Regression FDR control

How can I set false discovery rate (FDR) to 0 or be sure that FDR is smaller than some threshold value(e.g 0.1) in a logistic regression model? I tried cross validation for different parameters in ...
2
votes
1answer
25 views

Which of these scenarios are multiple or repeated comparisons?

I believed I knew what multiple comparison or repeated testing meant but upon reading more on the subject and listening to others I have become more confused. My confusion started with A/B testing ...
2
votes
2answers
490 views

Wilcoxon test with multiple testing: which correction for p values?

Please, I'm not very confident in statistics, and I'm trying to respond to a reviewer for a paper on the following issues: In my experiment I observed 15 babies during a test where they were free to ...
1
vote
1answer
65 views

Formulae for FDR values [duplicate]

Can anybody give a formula for the "FDR values"? (I write "FDR values" because strictly speaking, you don't have an FDR value for each test, but an FDR value for the whole study, but nevertheles, you ...
1
vote
0answers
77 views

“FDR<0.05”, or “FDR 0.05”?

I thought FDR is a threshold that we set, so we should speak about it without a < (or ) sign. For example, in a manuscript I ...
0
votes
1answer
171 views

FDR correction when a lot of results are expected to be True Positives

I have 11 tests and I expect at least half of them to be truly different (I expect that from prior knowledge, there are papers that directly indicate possible difference). But the power of each test ...
1
vote
1answer
25 views

Multiple hypothesis test

I have some questions about multiple hypothesis test. Situation 1: We have 1000 p-values, all of them are less than 0.0 5. We may say there are 50 false positive (1000*0.05) in these p-values. ...
0
votes
2answers
234 views

Questions about how to adjust p.value by FDR

I am adjusting a group of p-value by FDR in R. For example p.value= c(0.01,0.02,0.03,0.04,0.5,0.8,0.9) I see some papers used the following function. ...
0
votes
0answers
20 views

modified False Discovery Rate

In "Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection" by Candes et al, 2016, the authors discuss two notions of false discovery rate. Throughout, $q$ is the ...
3
votes
1answer
81 views

FDR controlling demonstration in R

I'm trying to demonstrate BH method for controlling false discovery rate in R. The problem is that I get quite unexpected results (so I assume that something is going wrong with this) ...
0
votes
0answers
41 views

Help with power analyses - adjusting alpha level correctly

I will be conducting a series of ANCOVAs and MANCOVAs for a 2 x 3 x 3 design with 2 covariates (1 covariate will only be relevant for a subset of analyses). There are 9 main DVs in the study. Three of ...
0
votes
0answers
17 views

Inference while controlling False Discovery rates given known dependence between test statistics

I have a number $n$ of standard normal test statistics $\boldsymbol{\beta}$, each of which belonging to a hypothesis I want to test. So under H$_0$ $$\boldsymbol{\beta} \sim N(\mathbf{0}, \boldsymbol{...
2
votes
1answer
390 views

About Kruskal-Wallis, Mann-Whitney U and multiple comparisons correction

I have 5 independent samples which I want to compare, to define if there are significant differences in the distribution of values across samples. The samples' sizes are: 4562, 1116, 314, 151, 77 I ...
1
vote
0answers
55 views

Quantile Regression and False Discovery

Context: quantile regression with a binary predictor, but this question can be generalized to other quantile regression model structures and possibly splines/adaptive models. In quantile regression ...
2
votes
1answer
149 views

What does “multiple backtesting” mean in the “False Strategy Theorem” of de Prado and Lewis?

I just read a paper by Lopez de Prado and Lewis that describes how multiple testing increases the likelihood of a false positive discovery. There is a crucial point that I do not understand in the ...
1
vote
0answers
39 views

Why the local Bayes fdr is greater than the Bayes FDR?

My question is related to empirical Bayes and large-scale inference. It is explained that the local Bayes false discovery rate (fdr) is greater than the Bayes false discovery rate (FDR). It is argued ...
3
votes
1answer
423 views

Multiple comparisons in mixed effects model

tl;dr In a random-slopes model, how should one adjust for multiple comparisons when performing inference on the group-specific slopes (the BLUPs)? Note 1: Bretz et al, the R package 'multcomp', and ...
2
votes
0answers
80 views

FDR and the the Benjamini-Hochberg Method

I am trying to understand the Benjamini-Hochberg Method for controlling the false discovery rate. Mathematically, if we are given with m hypothesis testing procedures, we sort the P-Values and reject ...
3
votes
2answers
285 views

Are family-wise error rate procedures for multiple hypotheses tests broken because there is no rigorous definition of “family”?

For example, a researcher could cleverly adjust how may hypotheses tests fall within the "family" definition in order to achieve more (or fewer) rejections when calculating the Bonferroni correction. ...
0
votes
1answer
37 views

Do I need to correct for multiple testing if I perform one mediation model with 3 parallel mediators?

I am performing a mediation model with 3 parallel mediators, 1 independent variable and 1 dependent variable. My question is: is this considered a unique analysis, or should I control for multiple ...
19
votes
2answers
6k views

FPR (false positive rate) vs FDR (false discovery rate)

The following quote comes from the famous research paper Statistical significance for genome wide studies by Storey & Tibshirani (2003): For example, a false positive rate of 5% means that on ...
1
vote
1answer
68 views

Should P-values of continuous variables be corrected?

Usually, when doing multiple tests to compare means of categorical variables, it is advised to do some correction of the P-values to control the probability or proportion of false positives (...
4
votes
0answers
76 views

A puzzling observation by Bradley Efron in his article in Science regarding Bayes’ Theorem in the 21st Century

Mr. Effron has published an interesting article in Science magazine with the enticing title "Baye's Theorem in the 21st Century". The article is quite short and can be found here: http://web.ipac....
0
votes
3answers
79 views

Is this an example of a multiple comparisons problem?

I am conducting a comprehensive research determining the effect of 3 independent variables on the metabolic rate of an organism. However, the set-up of my data analysis leads me to believe that I ...
0
votes
1answer
451 views

fpr, fdr and fwe for feature selection

Scikit-learn's univariate feature selection module offers three similar sounding methods for feature selection SelectFpr - false positive rate ...
3
votes
1answer
471 views

What is different about the q-value and local FDR?

What is different about the q-value and local FDR when both are defined as posterior probabilities of the null being true? For example in Storey (2010), Under these modeling assumptions, it ...
2
votes
1answer
337 views

Which confidence interval adjustment should I do when using FDR p valures adjustment?

I need to do multiple comparison, and I want adjust the p-values by false discovery rate (fdr). However, it is impossible also adjust the confidence intervals by <...
0
votes
1answer
21 views

What is the expectation operator of the False Discovery Rate taken under?

The False Discovery Rate is usually defined as: $$ FDR = E(Q) $$ where $Q$ is the proportion of false discoveries. I am wondering what this expectation is under. Is it a generic sample expectation?
0
votes
1answer
241 views

Bayesian interpretation of FDR

Suppose we want to apply a rejection rule on N tests. We may write the following table: ...
3
votes
2answers
434 views

Should I use q-values or p-values when presenting results?

I am presenting an analysis in which I performed a number of tests for N variables. The variables which are significant at q < 0.05 are presented in a table (q is the Benjamini-Hochberg corrected p-...
3
votes
1answer
1k views

Are p values produced by `p.adjust(method=“fdr”)` actually probabilities?

I've been trying to clarify for myself how to interpret the p values produced by p.adjust with method="fdr/BH". I'm aware of this question: https://stackoverflow.com/questions/10323817/r-unexpected-...
1
vote
0answers
39 views

is there a recommended number of p-values when using q-value estimation for FDR control?

I have created a Spearman correlation matrix for 6 variables (A1,..A6) to evaluate their relationships. I have thus obtained 15 p-values: ...
1
vote
1answer
57 views

Multiple Test Correction: robust against many insignificant p-values

I work in bioinformatics so I've seen my fair share of null multiple tests (figure 1) As well as a clear signal in p-value distributions (figure2) But I've also occasionally seen a type of p-value ...
1
vote
0answers
84 views

False discovery rate of Benjamini Hochberg at level $\alpha$ under global null

Suppose that we have null hypotheses $H_1, \cdots, H_n$. What is the FDR rate from Benjamini-Hochberg at level $\alpha$ the under the global null i.e. the assumption that the null hypotheses are all ...
0
votes
0answers
36 views

Examples of False-Discovery Rates Being Used Outside Of Biology

I feel like the go-to motivation behind FDR is microarrays. I would like to know of applications that use FDRs that are not biostatistics related.
1
vote
0answers
26 views

Filtering out noise/insignificant data when testing millions of contingency tables for association

I have a large dataset of around 20 million 2x2 contingency tables, as below: Y=1 Y=0 E=1 a b E=0 c d I want to measure the effect that exposure (...