Questions tagged [bonferroni]

Bonferroni refers to a method of adjustment for multiple comparisons in which the p value is divided by the number of tests. It is more conservative than many other such methods.

359 questions
Filter by
Sorted by
Tagged with
15 views

How to run a post-hoc correction test on multiple chi-square test?

I have run chi-square tests on occurrence of species across two time periods. As I have 1300 species I analysed them in bulk using a multiple loop function in R: chitest1<- t(sapply(seq(nrow(...
17 views

Do I need to apply Bonferroni Correction?

In my study I wish to perform three paired t-tests. The study compared times recorded to complete a test from before and after an intervention. The test is split into two parts (A and B) and the time ...
• 1
25 views

Boxplots t tests and Bonferroni correction

I'm comparing three groups: people that have had two infections, people that have had one infection and people that have had zero infections. I'm showing them on a boxplot (regarding age for example) ...
1 vote
18 views

Hochberg/Bonferonni correction for multiple one-sample t test

I am trying to conduct a content validity analysis where I want to test if a certain item (e.g. a question) can be related to a certain construct (e.g. depression). I've let people fill in a ...
• 329
109 views

How can I manually calculate the Bonferroni correction?

I created a statistical model with one categorical factor with six levels and performed a posthoc test to obtain the significance level for every contrast (15 comparisons). I wonder if I should apply ...
• 21
1 vote
24 views

Significance levels for multiple testing adjusted p-values

May I ask what is the conventional significance levels for multiple testing adjusted p-values (no matter Bonferroni or Holm or Sidak)? For unadjusted p-values, many researchers use * 5%, ** 1% and *** ...
• 13
15 views

Bonferroni correction when there are multiple evaluation axes

As a simple example case 1 I want to find out if the characteristic A of a certain raw material is different between factories X, Y and Z bonferroni correction m = 3 and each αi = α / m There is no ...
• 1
13 views

Multiple testing correction when the dependency is known to some degree

So, let's assume the multiple statistics are computed based on a the values of a common signal (e.g. time series), and the correlation between the statistics values can be estimated and is limited by ...
13 views

All Sig. Results <.001 Bonferroni

i have a data set of 73 participants and am running a ANOVA Repeated Measures test in SPSS. When viewing the Pairwise Comparisons table for the Bonferroni correction, all my Sig. results are <.001. ...
20 views

Verifying Holm correction line on FWER graph

I'm teaching a class about family-wise error rate, and have created a graph to illustrate how, when we increase the # of comparisons, the rate of type I errors increases rapidly (Family-Wise Error ...
• 132
1 vote
56 views

Should I adjust for multiple comparisons if test completely differerent hypotheses on different data?

A typical reality: a statistical journal asks me (I cannot refuse or won't get published) to summarize several baseline characteristics and compare them via test across 2 groups, G1 and G2, namely: ...
9 views

Family-wise error Correction for multiple outcome and multiple groups

I have one simple question regarding the p-value correction for family-wise error (FWE) when performing multiple comparisons across a) multiple groups and b) multiple outcomes. Let's suppose an ...
• 578
43 views

How can we possibly know all possible test when applying the Bonferroni correction?

I never quite understood the rationale behind the Bonferroni correction. I understand that we want to reduce the FDR, but it seems odd to penalise a test based on the existence of another test, given ...
33 views

number of pairwise comparisons in Bonferroni correction

I did chi square test on data of 60 patients having COVID 19. The variable in the rows is the severity of case (mild, moderate, severe) while the variable in the columns is lymphocytes count (normal, ...
29 views

Reporting multiple one-way ANOVAs concisely

I am writing a report on my findings. I have used an ANOVA test on 15 different dependent variables. The dependent variables are execution times of some processes. I considered the variables ...
1 vote
51 views

FDR correction stronger than Bonferroni?

Is it possible to get a situation where, in a multiple-testing scenario, for some individual tests, the Benjamini&Yukatieli (2001)-correction ends up being more stringent than Bonferroni? I.e. ...
1 vote
35 views

13 views

Multi comparison (e.g., t-test) for different independent pairs: is it necessary to adjust the P-value for the multi-comparison?

I kindly ask for an opinion: if I have 6 independent groups A, B, C, D, E, F but I have to compare only the pairs (A, B), (C, D), and (E,F) (i.e. there is no no cross-testing), is it necessary to ...
93 views

Why are the stated facts about using the Mann-Whitney U test for post-hoc testing after a Kruskal-Wallis test a problem?

I've read here on several threads that using Mann-Whitney U for post-hoc tests are an issue because: the ranks that the pair-wise rank sum tests use are not the ranks used by the Kruskal-Wallis test (...
70 views

Do I have a multiple comparisons scenario here?

Edited after the first 2 responses, additional content in italics I have 4 treatments and a control to encourage people to sign-up for a class. I'll be looking at sign-up and actual attendance rates, ...
29 views

Bonferroni correction for several multiple regressions

if I want to run a linear regression for each of 10 outcomes and each model has 5 predictors, to perform bonferroni correction, does it imply the new significance level to be 1 - alpha/50 ?
• 41
26 views

Anova or Bonferroni correction needed?

I've read a bit when it is appropriate to use Anova/Bonferroni correction over multiple t-tests but I am not sure if it applies to my data. I am extracting audio features from themed Spotify playlists ...
• 45
51 views

Does a procedure controlling FDR at level $\alpha$ always reject at least as much as a procedure controlling FWE at $\alpha$?

I understand the false discovery rate (FDR) is weakly less than the familywise error rate (FWE), and FDR is thus a less stringent way to control for type 1 errors. However, will a procedure that ...
• 353
38 views

Using the Bonferroni approach, what should the confidence level be for the individual intervals

Suppose researchers carry out a one-way ANOVA involving 5 groups. After finding statistically significant evidence of a difference in the group means, they decide to use the Bonferroni correction to ...
44 views

ANOVA post-hoc correction question

I am currently analysing data from a recent experiment, in which I will perform a 2 way repeated measures ANOVA (exercise intensity [3 intensities] x time [5 time points]). I have a significant ...
207 views

Interpretation of the Holm procedure

I have used the following code in Stata to calculate the p-values using the Holm method. However, I can't understand what to report and how to interpret these results. ...
25 views

Multiple comparisons correction methods

I come from a primarily Bayesian background; in my new role at a social networking company, we exclusively use Frequentist statistics and by in large it's in an A/B testing capacity. Often, we'll look ...
• 1,706
1 vote
33 views

Need to run Bonferroni correction if only doing 1 paired-samples t-test?

Say I have a significant interaction for a 2x3 within-subjects ANOVA. If I have a theoretical justification for only running a single paired samples t-test to look at simple main effects (even when ...
• 23
101 views

Bonferroni correction comparing multiple treatments

I want to compare the impact of different drugs on different blood-parameters. We measure $m$ parameters. In our study we want to compare $n$ drugs with each other. The blood-parameters for patients ...
• 23
29 views

Bonferroni correction on ANOVA - Tukey HSD - T-test sequence

I ran ANOVA to check for association between categorical and numerical variables and followed it with Tukey HSD to find the pairs that were significantly different and then followed with t-test to ...
15 views

When applying post-hoc corrections, do you group comparisons?

Say I am using a simple Bonferroni correction. I have two dimensional data, across 4 conditions = 4x2 = 8 hypotheses. Let's say I do 8 t-tests, one for each (condition, dimension) pair. But let's say ...
65 views

Power analysis for correlations, controlling for multiple comparisons

I want to estimate number of participants required for an experiment. I will be conducting 28 correlation tests. I am seeking 80% power for an effect size of 0.15 and alpha of 0.05 (all two-tailed). ...
19 views

How to best correct for repeated testing in cox proportional hazards models? [duplicate]

I am looking into how differing brain tumor genetics affects patient survival. I have a gene dataset with around 4600 predictors. Now I calculated a cox proportional hazards model using the ...
• 531
1 vote
244 views

Adjusted significance level in SPSS for Kruskal Wallis Post Hoc Test - Bonferroni adjustment

I am making a pairwise comparison using Dunn's post hoc test with Bonferroni correction. However, I am a bit confused on the interpretation of the adjusted significance on SPSS. I am making 10 ...
• 15
257 views

How to report? Significant difference for Kruskal-Wallis test, but not for ad hoc pairwise comparison with Bonferroni

I performed Krusal-Wallis test in SPSS and got a borderline significant P (0.042). then, the post hoc pairwise comparison with Bonferroni-Dunn correction showed no pairwise significant difference. I ...
1 vote
236 views

Is this a good case for bonferroni correction (or a different correction)?

Here's my experiment : I'm comparing the responses of two cultural groups (high/low context, total n = 550) on several dependent variables (instruments). The idea is to see if these two groups differ ...
276 views

How to correctly do a nested test (t-test, rank-sum, etc)

Let's say there are two groups of participants. Each group is randomly split into control subgroup A and task subgroup B. For each of the 4 cases some random variable $x$ is measured, whose mean is ...
• 1,589
1 vote
54 views

Does the Bonferroni Correction apply across metrics/KPIs?

Say I am running an A/B test with three total variants and two treatments (Control, T1, and T2). I am also calculating multiple KPIs--say two KPIs (K1 and K2)--for this test. If I apply the ...
• 11
42 views

Pairwise comparison - too many groups

I have 23 groups that I am comparing. I was just going to show the distribution of the groups with boxplots but a reviewer of my manuscript wants me to do group testing. The trouble is that pairwise ...
• 101
37 views

Beginner questions about mediation analysis in psych intervention (2 intervention + 2 mediator system)

I am studying, and am fairly new to research methods and design, and hence hope to get some guidance on my analysis plan. I have 3 questions here, and attaching a structural model as reference. ...
1 vote
146 views

When (in the context of 'non-independent tests') does Bonferroni correction *not* yield the appropriate FWER?

I have a question about the oft-heard suggestion that a Bonferroni correction is appropriate in the context of posing multiple independent hypotheses and inappropriate elsewhere, in contrast to ...
1 vote
108 views

Am I (reviewer) correct or is the researcher correct re: Bonferroni?

I've been asked to review an article for a low-tier journal, such that I am the only reviewer with stats knowledge (such as it is; I used to teach stats and advanced stats, but it's been a few years ...
46 views

log2 p-value of 1 after Holm-Bonferroni correction

I obtained rawp by performing Student's t-test function on dat.filtered and then I performed correction using Holm and Holm-Bonferroni methods. I have a huge increase in log2 adjusted p-value of 1.0. ...
• 11
112 views

How to properly perform Bonferroni Correction with multiple datasets and questions

Assume I have 6 algorithms, A, B, C, D, E, F and I want to perform a series of tests to see where A outperforms the others and B outperforms the others. In this case, I have 9 hypotheses I am testing: ...
• 31
1 vote
A standard problem in statistics would be something like the following. $$H_0: \theta = \theta_0$$ $$H_a: \theta \ne \theta_0$$ This is a two-sided test. What if we did the following two tests?  ...