Questions tagged [false-discovery-rate]

An expected fraction of rejected null hypotheses that are falsely rejected, i.e. the fraction of significant findings that are actually not true. One method to control FDR in multiple testing is Benjamini-Hochberg procedure.

Filter by
Sorted by
Tagged with
2
votes
1answer
22 views

Large p-values have lower FDRs than small p-values

This a conceptual question. I ran 10 tests and sorted the p-values in an increasing manner. I calculate the FDR as p-value * (10 / rank) Then, in this case, the larger the p-value, the lower the ...
0
votes
1answer
17 views

How to choose a relevant threshold for HWE

I'm a beginner in biostatistics and GWAS study. For a quality control (pre-imputation), I need to perform a chi-square test on each SNP (there are 700 000 of them) to test the deviation for Hardy-...
0
votes
0answers
23 views

Correcting for multiple paired comparisons within a large number of subjects

As the first step in an analysis of a large dataset, I am looking to identify a subset of subjects that are shown to have a significant response to at least one event condition. This involves looking ...
0
votes
0answers
8 views

Understanding the false discovery rate and positive selection rate

I want to know the relationship with EBIC and the FDR and PSR of some GLM, and what these two formulas mean? EBIC is defined as $$ EBIC(s) = -2\ell(\hat{\beta}(s)) + v(s)log(n)+2v(s)\gamma\...
1
vote
1answer
31 views

How to interpret False Discovery Rate?

I have performed a time series analysis and I have a dataframe of p-values. I want to performe a False Discovery Rate analysis to account for type I error. I am an R user and I have the following. <...
2
votes
0answers
25 views

Can FWER corrected p-values be called q-values?

This is a very naive question but I feel that many people in research do not have a very clear guidelines about what can or cannot be called a "q-value". I thought I had it right but I find so many ...
0
votes
0answers
12 views

Multiple test adjustments of p values: Data Mining vs Single tests

This discussion was triggered between me and a student after a significant p-value in a highly interesting correlation between two variables dissappeared after the p values were adjusted for multiple ...
0
votes
1answer
13 views

Reference on local false discovery rates

I am looking for a reference on local false discovery rates. I'm familiar with FDR and the Benjamin-Hochberg procedure, but am getting lost when it comes to the Bayesian local false discovery rate ...
0
votes
0answers
13 views

BH adjustment for multiple Multiple Regression models

I wish to conduct 8 multiple regression models, each with the same predictors (4 control variables, 1 continuous predictor, 3 dummy-coded categorical predictors, and the 3 interaction terms between ...
0
votes
0answers
43 views

How to compute p-values from modified z-scores (Iglewicz & Hoaglin) for FDR correction?

I have made good experience with using the modified Z-score for outlier detection in approximately normally distributed data, which was also discussed e.g. here and here. However, if applying many ...
0
votes
0answers
42 views

mafdr returns very small q-values

I am using matlab function for FDR procedure. The function is mafdr(). ...
0
votes
0answers
8 views

Multiple hypothesis correction yields higher FDR on lower number of tests. Why?

I have a DF (df1) with thousands of rows, where each row has a test and associated P, and their corresponding corrected P values (FDR Benjamin-Hochberg). Under ...
0
votes
1answer
77 views

When doing FDR correction in graphpad prism after Kruskal-Wallis does the programme automatically perform a Dunn test?

I used the Benjamini-Hochberg FDR correction in graphad after performing a Kruskal-Wallis test. This gives me a list of individual P-values. The graphad website says that if you select the FDR ...
3
votes
1answer
110 views

Plain language definition of positive regression dependency on each one from a subset (PRDS)

Existing questions have discussed the concept of PRDS in the context of Benjamini and Hochberg's false discovery rate correction (FDR-BH). However, I think the definitions of PRDS in these (and in ...
1
vote
1answer
42 views

Wilcoxon p-value correction for extremely small samples

I have performed thousands of Wilcoxon test. Unfortunately, for some of the tests size of one of the groups was equal to 1 or 2. ...
0
votes
0answers
29 views

FDR adjusted p-value, nothing exceed 0.05

I have a dataset that contains 8.2 million p-values. Here's the distribution of the negative log p-values; After I did the p.adjust with ...
0
votes
0answers
15 views

after FDR correction, no significant anymore?

I did pair t-tests and conditional logistic regressions on each of the 1000 variables. So I have 1000 of p-values from paired t-tests and 1000 of p-values from conditional logistic regressions. The ...
0
votes
0answers
96 views

Critical r value after false discovery rate (FDR) p correction

I would like to calculate a new threshold for my significance level based on the False Discovery Rate correction. I have a matrix of 100 x 100 with correlation coefficients. Each coefficient was ...
2
votes
0answers
120 views

How permutation based FDR control methods take into account the correlation structure?

There are a number of permutations-based False Discovery Rate (FDR) control methods, such as SAM [1], dChip [2], Ge et al [3], Meinshausen [4], and Efron [5]. It's been said that in these methods, ...
8
votes
2answers
957 views

When do we “stop” using multiple correction techniques?

I understand when performing a simple t-test, we typically control the type -1 error rate at $\alpha = .05$. This signifies that if the null hypothesis holds, the data will "incorrectly" reject the ...
0
votes
0answers
17 views

Is having too many comparisons a valid reason to increase FDR threshold level?

To briefly summarize, I have a bootstrapped data set that needs to be corrected for multiple comparisons. The figure below provides a snapshot of one set of the data. The x-axis is in bins of 60ms. ...
2
votes
2answers
76 views

Should I correct p values when the exact same dataset is not being used for multiple comparison?

What I have to do? Two courses (A and B) students CGPA and percentage of class attendance is given. I have to compare CGPA of two groups (group is based on attendance - high vs low) regarding ...
0
votes
1answer
37 views

FDR estimation from distribution and reference

I would like to have FDR/estimated FDR from two distributions with different mean and similar sigma. One of this is a reference distribution of 'real' samples (TP) while the other one is predicted (...
1
vote
1answer
101 views

False discovery rate multiple tests

I wrote an algorithm which finds a monopartite projection of a bipartite graph. The output looks quite strange to me, but since I cannot find mistakes in the code, I'm wondering if maybe they're not ...
0
votes
0answers
10 views

Should I remove empiral p-values = 1 before multi-testing?

I performed a permutation test in order to compute empirical p-values. To simplify I compare a vector with real data R to N=100,000 simuated vectors ...
2
votes
1answer
74 views

Multi test correction and hypothesis testing

I have 9 datasets with one predictor and one target attribute. For each of the dataset, I am testing for a single hypothesis - whether the attributes are associated. I have got the following based on ...
2
votes
0answers
39 views

Is this a valid non-parametric approach for computing false discovery rate (FDR)?

Summarizing the method in the reference below as I understand it: For each of $N$ "gene sets" (where a gene set represents a single null hypothesis significance test), compute a test statistic $T_i$. ...
1
vote
1answer
33 views

Help understanding why no significance after p-value adjustment

I performed multiple regression on 40 response variables against 6 predictors for 30 samples. Below are the raw and FDR adjusted p-values for the key variable in the study. I'm confused as to why my ...
3
votes
1answer
228 views

How to correct p-values of two multiple regression models

This may be a really basic question, but I haven't found any helpful answers. Or I simply don't understand. When correcting p-values of linear regression models, which p-values are corrected? My ...
-1
votes
1answer
209 views

Applying FDR in Correlation analysis

In the case of correlation analysis, we think that correlation coefficient and graph are much more important than p-value. This is because if the number of samples is sufficient (> 30~40), the p value ...
0
votes
1answer
94 views

Appropriate action to take once I've run 30 regressions with same IVs but tweaked DV

I run 30 OLS and GLM regressions using the same 9 IV/variables/features, but each with a slightly tweaked DV/target/label. More or less the same same variables are significant each time, but DV-tweaks ...
0
votes
0answers
45 views

Correction for multiple testing of permutation p-values

I simultaneously performed multiple statistical tests on my samples under investigation. For each test i obtained p-value that i had to correct for ...
1
vote
0answers
52 views

Estimating the False Reject Rate (FRR) of a classifier in production

I have trained a binary classifier which runs in production on remote distributed devices (which are out of my control). The model was trained on positive and negative samples, and I have chosen the ...
4
votes
0answers
62 views

How is it possible to control false discovery rate (FDR) without knowing the power and the prevalence of the nulls?

If we have p-value 0.05, to calculate probability of our discovery to be false positive, we need to use complex formula with prevalence (prior probability) and statistical power. If we have lots of ...
0
votes
0answers
508 views

What is the difference between q-value and adjusted p-value (p.adjust)?

I know the p-value and I may know what FDR (false discovery rate) do and its goal. But I confuse between q-value (often known as FDR) and adjusted p-value (p.ajust ...
0
votes
0answers
39 views

A minimum False Discovery Rate of 0.8 when doing multiple hypothesis testing on Logistic Regression Coefficient p-values?

I currently have 8,000 predictor variables and a binary indicator response variable. I then did a logistic regression on each of the predictor variables, i.e., response regressed on predictor variable ...
2
votes
2answers
140 views

What is the difference between the FDR in benjamini-hochberg/bonferonni vs a local FDR?

I am wondering if the local FDR in Efron's literature is different than the FDR associated with Benjamini-Hochberg and if it is perhaps talking about something else.
1
vote
1answer
110 views

False discovery rate in binary logistic regression analysis?

I am analyzing a population-representative cross sectional survey to identify associations between several factors and cardiovascular disease. I am performing binary logistic regression analysis with ...
1
vote
0answers
83 views

FDR for Bayes Factors

I have a dataset, 2 models $M_0$ and $M_1$. Most of my events belong to model $M_0$, so I detect events, belonging to $M_1$, only if the corresponding Bayes factor is bigger than 20. So far so good, ...
0
votes
1answer
43 views

How to compensate for different statistical power in multiple comparisons? [closed]

Imagine we are performing 10 comparisons on a single variable of choice. The 10 respective experiments have different statistical powers. Now imagine that the least powerful and the most powerful ...
1
vote
0answers
50 views

Pairwise non-parametric test?

I have 5 groups of patients (different sizes), for which I measured some property that is continuous but not normally distributed. I would like to compare every pair of groups in respect to this ...
2
votes
1answer
36 views

Which of these scenarios are multiple or repeated comparisons?

I believed I knew what multiple comparison or repeated testing meant but upon reading more on the subject and listening to others I have become more confused. My confusion started with A/B testing ...
2
votes
2answers
2k views

Wilcoxon test with multiple testing: which correction for p values?

Please, I'm not very confident in statistics, and I'm trying to respond to a reviewer for a paper on the following issues: In my experiment I observed 15 babies during a test where they were free to ...
1
vote
1answer
109 views

Formulae for FDR values [duplicate]

Can anybody give a formula for the "FDR values"? (I write "FDR values" because strictly speaking, you don't have an FDR value for each test, but an FDR value for the whole study, but nevertheles, you ...
2
votes
0answers
142 views

“FDR<0.05”, or “FDR 0.05”?

I thought FDR is a threshold that we set, so we should speak about it without a < (or ) sign. For example, in a manuscript I ...
0
votes
1answer
175 views

FDR correction when a lot of results are expected to be True Positives

I have 11 tests and I expect at least half of them to be truly different (I expect that from prior knowledge, there are papers that directly indicate possible difference). But the power of each test ...
2
votes
1answer
28 views

Multiple hypothesis test

I have some questions about multiple hypothesis test. Situation 1: We have 1000 p-values, all of them are less than 0.0 5. We may say there are 50 false positive (1000*0.05) in these p-values. ...
0
votes
2answers
536 views

Questions about how to adjust p.value by FDR

I am adjusting a group of p-value by FDR in R. For example p.value= c(0.01,0.02,0.03,0.04,0.5,0.8,0.9) I see some papers used the following function. ...
3
votes
1answer
145 views

FDR controlling demonstration in R

I'm trying to demonstrate BH method for controlling false discovery rate in R. The problem is that I get quite unexpected results (so I assume that something is going wrong with this) ...
0
votes
0answers
52 views

Help with power analyses - adjusting alpha level correctly

I will be conducting a series of ANCOVAs and MANCOVAs for a 2 x 3 x 3 design with 2 covariates (1 covariate will only be relevant for a subset of analyses). There are 9 main DVs in the study. Three of ...

1
2 3 4 5