Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [false-discovery-rate]

An expected fraction of rejected null hypotheses that are falsely rejected, i.e. the fraction of significant findings that are actually not true. One method to control FDR in multiple testing is Benjamini-Hochberg procedure.

0
votes
0answers
10 views

A minimum False Discovery Rate of 0.8 when doing multiple hypothesis testing on Logistic Regression Coefficient p-values?

I currently have 8,000 predictor variables and a binary indicator response variable. I then did a logistic regression on each of the predictor variables, i.e., response regressed on predictor variable ...
0
votes
0answers
4 views

What is the difference between the FDR in benjamini-hochberg/bonferonni vs a local FDR?

I am wondering if the local FDR in Efron's literature is different than the FDR associated with Benjamini-Hochberg and if it is perhaps talking about something else.
0
votes
0answers
11 views

Grouping hypotheses, and applying FDR to each group separately?

Suppose I have two sets of hypotheses, $A$ and $B$, and I want to control FDR. When is it valid to control the FDR of $A$ and $B$ separately, vs controlling the FDR of $A \cup B$ jointly? One ...
1
vote
1answer
33 views

False discovery rate in binary logistic regression analysis?

I am analyzing a population-representative cross sectional survey to identify associations between several factors and cardiovascular disease. I am performing binary logistic regression analysis with ...
0
votes
0answers
16 views

FDR control without P-values

I classify $n$ objects into 2 categories, using some criteria and threshold $\alpha$. To estimate the number of false discoveries, I create a sample of $n$ permuted objects (keeping covariance ...
1
vote
0answers
24 views

FDR for Bayes Factors

I have a dataset, 2 models $M_0$ and $M_1$. Most of my events belong to model $M_0$, so I detect events, belonging to $M_1$, only if the corresponding Bayes factor is bigger than 20. So far so good, ...
0
votes
1answer
29 views

How to compensate for different statistical power in multiple comparisons? [closed]

Imagine we are performing 10 comparisons on a single variable of choice. The 10 respective experiments have different statistical powers. Now imagine that the least powerful and the most powerful ...
1
vote
0answers
22 views

Pairwise non-parametric test?

I have 5 groups of patients (different sizes), for which I measured some property that is continuous but not normally distributed. I would like to compare every pair of groups in respect to this ...
0
votes
0answers
35 views

Linear Regression FDR control

How can I set false discovery rate (FDR) to 0 or be sure that FDR is smaller than some threshold value(e.g 0.1) in a logistic regression model? I tried cross validation for different parameters in ...
2
votes
1answer
24 views

Which of these scenarios are multiple or repeated comparisons?

I believed I knew what multiple comparison or repeated testing meant but upon reading more on the subject and listening to others I have become more confused. My confusion started with A/B testing ...
2
votes
2answers
221 views

Wilcoxon test with multiple testing: which correction for p values?

Please, I'm not very confident in statistics, and I'm trying to respond to a reviewer for a paper on the following issues: In my experiment I observed 15 babies during a test where they were free to ...
1
vote
1answer
61 views

Formulae for FDR values [duplicate]

Can anybody give a formula for the "FDR values"? (I write "FDR values" because strictly speaking, you don't have an FDR value for each test, but an FDR value for the whole study, but nevertheles, you ...
1
vote
0answers
66 views

“FDR<0.05”, or “FDR 0.05”?

I thought FDR is a threshold that we set, so we should speak about it without a < (or ) sign. For example, in a manuscript I ...
0
votes
1answer
53 views

FDR correction when a lot of results are expected to be True Positives

I have 11 tests and I expect at least half of them to be truly different (I expect that from prior knowledge, there are papers that directly indicate possible difference). But the power of each test ...
1
vote
1answer
24 views

Multiple hypothesis test

I have some questions about multiple hypothesis test. Situation 1: We have 1000 p-values, all of them are less than 0.0 5. We may say there are 50 false positive (1000*0.05) in these p-values. ...
0
votes
2answers
139 views

Questions about how to adjust p.value by FDR

I am adjusting a group of p-value by FDR in R. For example p.value= c(0.01,0.02,0.03,0.04,0.5,0.8,0.9) I see some papers used the following function. ...
0
votes
0answers
16 views

modified False Discovery Rate

In "Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection" by Candes et al, 2016, the authors discuss two notions of false discovery rate. Throughout, $q$ is the ...
3
votes
1answer
72 views

FDR controlling demonstration in R

I'm trying to demonstrate BH method for controlling false discovery rate in R. The problem is that I get quite unexpected results (so I assume that something is going wrong with this) ...
0
votes
0answers
31 views

Help with power analyses - adjusting alpha level correctly

I will be conducting a series of ANCOVAs and MANCOVAs for a 2 x 3 x 3 design with 2 covariates (1 covariate will only be relevant for a subset of analyses). There are 9 main DVs in the study. Three of ...
0
votes
0answers
17 views

Inference while controlling False Discovery rates given known dependence between test statistics

I have a number $n$ of standard normal test statistics $\boldsymbol{\beta}$, each of which belonging to a hypothesis I want to test. So under H$_0$ $$\boldsymbol{\beta} \sim N(\mathbf{0}, \boldsymbol{...
0
votes
0answers
19 views

Applying false discovery rate on filtered p-values

I am checking a million hypotheses using two similar statistical tests (A and B). I want to identify hypotheses significant w.r.t to both A and B. The p-values from both A and B are expected to be ...
1
vote
1answer
298 views

About Kruskal-Wallis, Mann-Whitney U and multiple comparisons correction

I have 5 independent samples which I want to compare, to define if there are significant differences in the distribution of values across samples. The samples' sizes are: 4562, 1116, 314, 151, 77 I ...
1
vote
0answers
50 views

Quantile Regression and False Discovery

Context: quantile regression with a binary predictor, but this question can be generalized to other quantile regression model structures and possibly splines/adaptive models. In quantile regression ...
2
votes
1answer
135 views

What does “multiple backtesting” mean in the “False Strategy Theorem” of de Prado and Lewis?

I just read a paper by Lopez de Prado and Lewis that describes how multiple testing increases the likelihood of a false positive discovery. There is a crucial point that I do not understand in the ...
1
vote
0answers
35 views

Why the local Bayes fdr is greater than the Bayes FDR?

My question is related to empirical Bayes and large-scale inference. It is explained that the local Bayes false discovery rate (fdr) is greater than the Bayes false discovery rate (FDR). It is argued ...
3
votes
1answer
336 views

Multiple comparisons in mixed effects model

tl;dr In a random-slopes model, how should one adjust for multiple comparisons when performing inference on the group-specific slopes (the BLUPs)? Note 1: Bretz et al, the R package 'multcomp', and ...
1
vote
0answers
58 views

FDR and the the Benjamini-Hochberg Method

I am trying to understand the Benjamini-Hochberg Method for controlling the false discovery rate. Mathematically, if we are given with m hypothesis testing procedures, we sort the P-Values and reject ...
2
votes
2answers
244 views

Are family-wise error rate procedures for multiple hypotheses tests broken because there is no rigorous definition of “family”?

For example, a researcher could cleverly adjust how may hypotheses tests fall within the "family" definition in order to achieve more (or fewer) rejections when calculating the Bonferroni correction. ...
0
votes
1answer
23 views

Do I need to correct for multiple testing if I perform one mediation model with 3 parallel mediators?

I am performing a mediation model with 3 parallel mediators, 1 independent variable and 1 dependent variable. My question is: is this considered a unique analysis, or should I control for multiple ...
0
votes
0answers
53 views

Using local FDR and q-values in practice?

I've learned that the local FDR and q-values are the estimated FDR given a fixed rejection region, which is the opposite of other frequentist methods where you fix the error rate first to estimate the ...
0
votes
0answers
20 views

Using q-values in practice?

I have been trying to wrap my head around the concept of q-values for a while, but I'm fairly new to statistics so have been struggling with understanding papers. From Storey's paper, I'm given to an ...
18
votes
2answers
4k views

FPR (false positive rate) vs FDR (false discovery rate)

The following quote comes from the famous research paper Statistical significance for genome wide studies by Storey & Tibshirani (2003): For example, a false positive rate of 5% means that on ...
1
vote
1answer
64 views

Should P-values of continuous variables be corrected?

Usually, when doing multiple tests to compare means of categorical variables, it is advised to do some correction of the P-values to control the probability or proportion of false positives (...
4
votes
0answers
70 views

A puzzling observation by Bradley Efron in his article in Science regarding Bayes’ Theorem in the 21st Century

Mr. Effron has published an interesting article in Science magazine with the enticing title "Baye's Theorem in the 21st Century". The article is quite short and can be found here: http://web.ipac....
0
votes
3answers
71 views

Is this an example of a multiple comparisons problem?

I am conducting a comprehensive research determining the effect of 3 independent variables on the metabolic rate of an organism. However, the set-up of my data analysis leads me to believe that I ...
0
votes
1answer
332 views

fpr, fdr and fwe for feature selection

Scikit-learn's univariate feature selection module offers three similar sounding methods for feature selection SelectFpr - false positive rate ...
3
votes
1answer
405 views

What is different about the q-value and local FDR?

What is different about the q-value and local FDR when both are defined as posterior probabilities of the null being true? For example in Storey (2010), Under these modeling assumptions, it ...
2
votes
1answer
273 views

Which confidence interval adjustment should I do when using FDR p valures adjustment?

I need to do multiple comparison, and I want adjust the p-values by false discovery rate (fdr). However, it is impossible also adjust the confidence intervals by <...
0
votes
1answer
21 views

What is the expectation operator of the False Discovery Rate taken under?

The False Discovery Rate is usually defined as: $$ FDR = E(Q) $$ where $Q$ is the proportion of false discoveries. I am wondering what this expectation is under. Is it a generic sample expectation?
0
votes
1answer
221 views

Bayesian interpretation of FDR

Suppose we want to apply a rejection rule on N tests. We may write the following table: ...
3
votes
2answers
369 views

Should I use q-values or p-values when presenting results?

I am presenting an analysis in which I performed a number of tests for N variables. The variables which are significant at q < 0.05 are presented in a table (q is the Benjamini-Hochberg corrected p-...
3
votes
1answer
1k views

Are p values produced by `p.adjust(method=“fdr”)` actually probabilities?

I've been trying to clarify for myself how to interpret the p values produced by p.adjust with method="fdr/BH". I'm aware of this question: https://stackoverflow.com/questions/10323817/r-unexpected-...
1
vote
0answers
36 views

is there a recommended number of p-values when using q-value estimation for FDR control?

I have created a Spearman correlation matrix for 6 variables (A1,..A6) to evaluate their relationships. I have thus obtained 15 p-values: ...
1
vote
1answer
47 views

Multiple Test Correction: robust against many insignificant p-values

I work in bioinformatics so I've seen my fair share of null multiple tests (figure 1) As well as a clear signal in p-value distributions (figure2) But I've also occasionally seen a type of p-value ...
1
vote
0answers
81 views

False discovery rate of Benjamini Hochberg at level $\alpha$ under global null

Suppose that we have null hypotheses $H_1, \cdots, H_n$. What is the FDR rate from Benjamini-Hochberg at level $\alpha$ the under the global null i.e. the assumption that the null hypotheses are all ...
0
votes
0answers
33 views

Examples of False-Discovery Rates Being Used Outside Of Biology

I feel like the go-to motivation behind FDR is microarrays. I would like to know of applications that use FDRs that are not biostatistics related.
1
vote
0answers
25 views

Filtering out noise/insignificant data when testing millions of contingency tables for association

I have a large dataset of around 20 million 2x2 contingency tables, as below: Y=1 Y=0 E=1 a b E=0 c d I want to measure the effect that exposure (...
1
vote
1answer
149 views

Random Forest: Estimate False Discovery Rate in new data?

I have a random forest classification model built on balanced positive and negative classes. I am trying to estimate the number of false positives and false negatives in the new data, in order to ...
5
votes
1answer
158 views

Combining False Discovery Rates (FDR)?

Suppose we have a population of patients. For each patient, we measure 1000 features and observe if they suffer from any of 10 diseases. We wish to determine which features, if any, are significant ...
0
votes
2answers
358 views

What is the relation between the False Positive Rate to the True Positive and True Negative Rates?

I know that the True Negative Rate plus the False Positive Rate is equal to 1. However, I don't quite see how the True Positive Rate and False Positive Rate are related. Is there a relation between ...