Questions tagged [false-discovery-rate]

An expected fraction of rejected null hypotheses that are falsely rejected, i.e. the fraction of significant findings that are actually not true. One method to control FDR in multiple testing is Benjamini-Hochberg procedure.

Filter by
Sorted by
Tagged with
1
vote
0answers
6 views

Is there a way to adjust FDR threshold using correlation across tests (aka, number of estimated independent tests, like in Bonferroni correction)

For Bonferroni correction, correlations can be used to inform correction of multiple testing using the spectral decomposition of matrices. For example, in GWAS, the threshold is established to be 5e-8 ...
0
votes
1answer
22 views

How do I correct for multiple testing when I have multiple models?

I have 12 concentrations that I would like to compare between 2 groups, some subjects have 2 measures. I know the concentration is also depending on age and sex so I added those to the linear model ...
0
votes
0answers
22 views

Multiple comparison when testing one planned hypothesis

I am running one logit model containing an interaction: logit dv year contract_type yearXcontract+Z where contract_type and year are dummies with 4 levels. After ...
2
votes
0answers
28 views

Multiple comparisons version of Kolmogorov-Smirnov test

Suppose $\mathbf{y}_{i} $ and $ \mathbf{x}_{i}\,$ are length-$m$ vectors and $D_{i}$ is some arbitrary distribution $(i=1,...,N)$. I would like to conduct the following hypothesis test: $ H_{0}: \...
0
votes
0answers
57 views

When does a proxy/substitute variable distinguish between groups as well as the original variable?

I’m trying to determine when a proxy variable can be substituted for a real variable for the purposes of distinguishing two groups. To make this more concrete, suppose I want to determine, generally, ...
2
votes
0answers
23 views

Correct to use FDR q-values for a small number of variables?

I would like to correct for multiple testing and I was considering using FDR q-values. In the original paper by Storey, they used this approach for genomewide data, where there are several thousands ...
2
votes
1answer
30 views

Statistical significance and the two-trials rule in clinical trials

If the false discovery rate is known to be 0.40, how do we obtain the probability that two consecutive statistically significant results arising from an application of the two-trials rule are true ...
0
votes
0answers
5 views

What is cFDR & does it control the FDR like BH or IHW?

Conditional False Discovery Rate (cFDR) [1] allows to use a covariate informative of power or prior likelihood but independent of the statistic/p-values under the null to increase the number of ...
1
vote
1answer
41 views

How to summarize multiple independent false discovery rates (FDRs or q-values)?

In the context of high-throughput biological assays (e.g. genomics), where we are applying a hypothesis test (e.g. differential expression between groups of samples) to each analyte in a dataset with ...
5
votes
1answer
118 views

Understanding the Benjamini-Hochberg method proof

I'm trying to understand the proof in Benjamini & Hochberg's 1995 paper, specifically the Lemma in the appendix, as the rest of the proof is short and follows it. I got stuck somewhere after ...
1
vote
1answer
16 views

Differential expression for various cell sub-types : is there a multiple comparisons problem?

Take the following scenario: differential analysis of genes expression between two conditions did not yield DE genes. However, the material for the analysis came from a mixture of cell types. It is ...
0
votes
1answer
18 views

Calculate fraction of multiple tests for which the null hypothesis was rejected

Consider a test (e.g., a permutation test) that tests, at the individual level, if a binary event occured or not. A paper that I was recently reading, for example, tested whether or not a pair of two ...
1
vote
1answer
55 views

If one performs FDR corrections for multiple (independent) comparisons - what is the overall FDR? [duplicate]

Suppose that I perform two different comparisons on the same genomic dataset (with say 1000 genes, and three conditions). For each comparison, I adjust the pvalues using Benjamini Hochberg, ...
1
vote
0answers
66 views

When to care about FDR vs when to care about FWER

When testing multiple hypotheses on the same set of collected data, we want of course to correct for the increased likelihood of false positives from multiple tests. So far I've always used the Holm-...
0
votes
0answers
19 views

How to use FDR properly in a study with 17 different variables and tests?

I have a study with 17 variables. 1 was applied two times. 2 were applied 4 times. I am looking for differences in two groups. I have 100 participants, 50 per condition.  Some variables are relative ...
0
votes
1answer
25 views

Removal of untestable hypothesis before multiplicity correction

Dear community members, I performed thousands of tests, each of which do some regression comparison between numbers of events in cases and controls. Each test comprised thousands of cases, a 3-fold ...
0
votes
0answers
26 views

R: p-value correcting multiple windows of observations

I have scanned a genome for enrichment of gene functions, and I've used FDR to correct for multiple testing within each window (this is standard procedure of GO enrichment). However, the genome is ...
0
votes
0answers
9 views

What to report regarding FPR/Type I errors in addition to p-values to avoid misinterpretation?

I would like to report some indication/estimation of the (minimum) false discovery (i.e., type 1 error rate) of my results because that is how p-values are often misinterpreted but I am unsure how to ...
0
votes
0answers
33 views

P-values from Different Distributions

I was just considering the following hypothetical scenario, where suppose we have $X_1,...,X_n$ observations, and for each observation, we then assume that each are associated respectively with the ...
2
votes
1answer
34 views

P.adjust - different values when using a list vs. individual p-values

I am having some trouble determining the cause of an issue in my R coding. I have a series of 11 p-values, which correcting using the p.adjust function in R with BH corrections. When I create a list ...
1
vote
0answers
29 views

Corrections for multiple hypothesis testing

Apologies if this repeats a previous query, I have read a lot of prior threads (and made myself a bit cross-eyed with the ones asking whether we should correct for every hypothesis test ever performed!...
1
vote
1answer
47 views

Empirical FDR from real and null distributions — do sample sizes need to match?

Suppose I have a set of values (correlation, in this case) that come from a real distribution, and another set of values that are from a simulated null distribution. To calculate the value that ...
0
votes
1answer
40 views

Should I do the multiple comparisons after the 2 between-group paired t-test?

I have some problems with my statistics. I need to help. I have about 8 different variables on 14 subjects. Each variable contains 5 regional and 1 total value. Thus, the total number of variables is ...
0
votes
0answers
28 views

Benjamini-Hochberg FDR Correction using partially observed p-values

I am working on something called eQTL analysis. My result is extremely huge, there are about 28 million rows (with its corresponding p values) per dataset, and I have 10 dataset. I tried to do some ...
0
votes
1answer
104 views

Multiple comparison correction for confidence interval

I am trying to understand multiple-comparisons adjustment for confidence intervals. I came across the following links on the topic, but a few things are not clear to me. Stanford lecture: Section 2 ...
0
votes
0answers
15 views

Should I apply multiple testing correction or not?

I want to ask if I should apply corrections for multiple testing (e.g. FDR) in the following situations; as I know, when we have continuous variables of more than 2 groups and before we apply ANOVA, ...
1
vote
0answers
14 views

Problem in understanding False Discovery Rate (FDR)

I am facing a problem understanding the concept of FDR or False Discovery Rate in Multiple Hypothesis Testing. May be the question is silly, please try to bear with me. Actually, when we are talking ...
2
votes
0answers
59 views

Multiple Comparisons Correction for Multiple ANOVA models - Include only terms of interest?

I have a a biological dataset, in which I am fitting the following partial mixed ANOVA model (apologizes for the MATLAB syntax): ...
6
votes
1answer
101 views

Proof/derivation for false discovery rate in Benjamini-Hochberg procedure

The Benjamini-Hochberg procedure is a method that corrects for multiple comparisons and has a false discovery rate (FDR) equal to $\alpha$. Or is it the family wise error rate, FWER? I am a bit ...
7
votes
3answers
515 views

Rejection threshold of the Benjamini-Hochberg procedure

Is there a possibility to calculate or estimate the overall rejection threshold of the Benjamini–Hochberg procedure (BH)? For the correction of the FWER using the Bonferroni method, the significance ...
0
votes
0answers
14 views

Choice of covariate under Independent Hypothesis Weighting (IHW)

After reading this survey of methods for controlling FDR and considering my problem at hand (https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1716-1), IHW seems like a good choice. ...
0
votes
1answer
98 views

Interpretting and Using fdrtool?

With the help of articles like this one, I am beginning to understand false discovery rate first laid out by Benjamini and Hochberg (1995). I am also seeing the value of the "unified" ...
1
vote
0answers
15 views

Adjust for multiple comparisons in regression with one outcome variable and several independent variables?

How do I adjust for multiple comparisons using FDR with linear mixed effects first regression: ...
0
votes
0answers
9 views

FDR multiple comparsions adjustment for PCR and posthoc

How many multiple comparisons adjustments do I have to account for in my FDR corrections? I performed a principle component regression and then a posthoc regression (on 7 region pairs in the brain), ...
0
votes
0answers
21 views

Multiplicity adjustment when trying to show similarity between groups

Is there a convention regarding adjusting for multiplicity in situations where it helps your hypothesis? For example, I have two groups for a clinical trial, one for placebo and one for treatment. I ...
0
votes
0answers
56 views

When conducting multiple comparisons with adonis, the p-values of the comparisons should be used as they are or used it after correction?

I recently ran 60 tests with adonis and found some significant differences (p-vaue < 0.05) between control and treatment (I only have one control and one treatment and do not need to do the ...
1
vote
0answers
12 views

Sensitivity and specificity with multiple sequential tests

This has been on my mind for a while, and I would appreciate a statistician's view on the matter. Suppose that a given test has sensitivity $a$ and specificity $b$. Now, periodically apply this test ...
0
votes
1answer
56 views

Is the p-value dependent on n in case of paired student t-test?

I have a dataset where we believe there are a bunch of infected and uninfected subjects (total being 15). There are two ways to categorise these subjects into infected/ uninfected: 1. categorise 10 ...
1
vote
0answers
11 views

Multiple comparison in non-parametric data

According to this: https://www.statisticshowto.com/multiple-testing-problem/ in non-parametric distributions I cannot perform FDR corrections. I have been searching for hours without finding anything ...
0
votes
1answer
68 views

interpretation of Significance level (Type I error probability) from pwr.t.test()

I was looking at the sig.level=Significance level (Type I error probability) from https://www.rdocumentation.org/packages/pwr/versions/1.3-0/topics/pwr.t.test Can someone please explain me is ...
4
votes
1answer
313 views

P-value correction for multiple testing using huge datasets

First of all I apologize without the question is very basic, I am taking my first steps in data science, statistics and bioinformatics. Data information We are evaluating the correlation (using the ...
0
votes
0answers
31 views

Correlation between two non-independent samples (dichotomized data / multiple testing)

I've been looking through text books but unable to find a precise answer to this question, but it seems important so maybe I'm looking in wrong places. Imagine some population with a normal ...
2
votes
0answers
118 views

Example of when FWER control should be preferred to FDR control

In the Benjamini & Hochberg paper (1995) which introduces the False Discovery Rate (FDR), the authors compare the FDR to the Family-Wise Error Rate (FWER): Often the control of the FWER is not ...
0
votes
0answers
33 views

Confidence intervals for a binomial glm with weights

I have binomial response data with a single explanatory variable. In R I can write the model as something like ...
0
votes
0answers
38 views

FDR control in model prediction

I classified positives from negatives using, e.g., linear discriminant analysis (LDA), and perform false discovery rate (FDR) ...
2
votes
1answer
25 views

Large p-values have lower FDRs than small p-values

This a conceptual question. I ran 10 tests and sorted the p-values in an increasing manner. I calculate the FDR as p-value * (10 / rank) Then, in this case, the larger the p-value, the lower the ...
0
votes
1answer
99 views

How to choose a relevant threshold for HWE

I'm a beginner in biostatistics and GWAS study. For a quality control (pre-imputation), I need to perform a chi-square test on each SNP (there are 700 000 of them) to test the deviation for Hardy-...
1
vote
1answer
91 views

How to interpret False Discovery Rate?

I have performed a time series analysis and I have a dataframe of p-values. I want to performe a False Discovery Rate analysis to account for type I error. I am an R user and I have the following. <...
2
votes
0answers
27 views

Can FWER corrected p-values be called q-values?

This is a very naive question but I feel that many people in research do not have a very clear guidelines about what can or cannot be called a "q-value". I thought I had it right but I find so many ...
0
votes
1answer
40 views

Reference on local false discovery rates

I am looking for a reference on local false discovery rates. I'm familiar with FDR and the Benjamin-Hochberg procedure, but am getting lost when it comes to the Bayesian local false discovery rate ...

1
2 3 4 5