Questions tagged [bonferroni]

Bonferroni refers to a method of adjustment for multiple comparisons in which the p value is divided by the number of tests. It is more conservative than many other such methods.

Filter by
Sorted by
Tagged with
0
votes
0answers
15 views

Why are the stated facts about using the Mann-Whitney U test for post-hoc testing after a Kruskal-Wallis test a problem?

I've read here on several threads that using Mann-Whitney U for post-hoc tests are an issue because: the ranks that the pair-wise rank sum tests use are not the ranks used by the Kruskal-Wallis test (...
0
votes
2answers
53 views

Do I have a multiple comparisons scenario here?

Edited after the first 2 responses, additional content in italics I have 4 treatments and a control to encourage people to sign-up for a class. I'll be looking at sign-up and actual attendance rates, ...
0
votes
0answers
19 views

Bonferroni correction for several multiple regressions

if I want to run a linear regression for each of 10 outcomes and each model has 5 predictors, to perform bonferroni correction, does it imply the new significance level to be 1 - alpha/50 ?
0
votes
0answers
24 views

Anova or Bonferroni correction needed?

I've read a bit when it is appropriate to use Anova/Bonferroni correction over multiple t-tests but I am not sure if it applies to my data. I am extracting audio features from themed Spotify playlists ...
2
votes
1answer
21 views

Does a procedure controlling FDR at level $\alpha$ always reject at least as much as a procedure controlling FWE at $\alpha$?

I understand the false discovery rate (FDR) is weakly less than the familywise error rate (FWE), and FDR is thus a less stringent way to control for type 1 errors. However, will a procedure that ...
0
votes
0answers
5 views

Using the Bonferroni approach, what should the confidence level be for the individual intervals

Suppose researchers carry out a one-way ANOVA involving 5 groups. After finding statistically significant evidence of a difference in the group means, they decide to use the Bonferroni correction to ...
0
votes
2answers
36 views

ANOVA post-hoc correction question

I am currently analysing data from a recent experiment, in which I will perform a 2 way repeated measures ANOVA (exercise intensity [3 intensities] x time [5 time points]). I have a significant ...
2
votes
1answer
145 views

Interpretation of the Holm procedure

I have used the following code in Stata to calculate the p-values using the Holm method. However, I can't understand what to report and how to interpret these results. ...
0
votes
0answers
18 views

Multiple comparisons correction methods

I come from a primarily Bayesian background; in my new role at a social networking company, we exclusively use Frequentist statistics and by in large it's in an A/B testing capacity. Often, we'll look ...
1
vote
0answers
19 views

Need to run Bonferroni correction if only doing 1 paired-samples t-test?

Say I have a significant interaction for a 2x3 within-subjects ANOVA. If I have a theoretical justification for only running a single paired samples t-test to look at simple main effects (even when ...
2
votes
1answer
69 views

Bonferroni correction comparing multiple treatments

I want to compare the impact of different drugs on different blood-parameters. We measure $m$ parameters. In our study we want to compare $n$ drugs with each other. The blood-parameters for patients ...
0
votes
0answers
14 views

Bonferroni correction on ANOVA - Tukey HSD - T-test sequence

I ran ANOVA to check for association between categorical and numerical variables and followed it with Tukey HSD to find the pairs that were significantly different and then followed with t-test to ...
0
votes
0answers
6 views

When applying post-hoc corrections, do you group comparisons?

Say I am using a simple Bonferroni correction. I have two dimensional data, across 4 conditions = 4x2 = 8 hypotheses. Let's say I do 8 t-tests, one for each (condition, dimension) pair. But let's say ...
0
votes
0answers
20 views

Power analysis for correlations, controlling for multiple comparisons

I want to estimate number of participants required for an experiment. I will be conducting 28 correlation tests. I am seeking 80% power for an effect size of 0.15 and alpha of 0.05 (all two-tailed). ...
0
votes
0answers
18 views

How to best correct for repeated testing in cox proportional hazards models? [duplicate]

I am looking into how differing brain tumor genetics affects patient survival. I have a gene dataset with around 4600 predictors. Now I calculated a cox proportional hazards model using the ...
1
vote
1answer
34 views

Adjusted significance level in SPSS for Kruskal Wallis Post Hoc Test - Bonferroni adjustment

I am making a pairwise comparison using Dunn's post hoc test with Bonferroni correction. However, I am a bit confused on the interpretation of the adjusted significance on SPSS. I am making 10 ...
0
votes
1answer
56 views

How to report? Significant difference for Kruskal-Wallis test, but not for ad hoc pairwise comparison with Bonferroni

I performed Krusal-Wallis test in SPSS and got a borderline significant P (0.042). then, the post hoc pairwise comparison with Bonferroni-Dunn correction showed no pairwise significant difference. I ...
1
vote
1answer
61 views

Is this a good case for bonferroni correction (or a different correction)?

Here's my experiment : I'm comparing the responses of two cultural groups (high/low context, total n = 550) on several dependent variables (instruments). The idea is to see if these two groups differ ...
3
votes
3answers
133 views

How to correctly do a nested test (t-test, rank-sum, etc)

Let's say there are two groups of participants. Each group is randomly split into control subgroup A and task subgroup B. For each of the 4 cases some random variable $x$ is measured, whose mean is ...
1
vote
0answers
33 views

Does the Bonferroni Correction apply across metrics/KPIs?

Say I am running an A/B test with three total variants and two treatments (Control, T1, and T2). I am also calculating multiple KPIs--say two KPIs (K1 and K2)--for this test. If I apply the ...
0
votes
0answers
23 views

Pairwise comparison - too many groups

I have 23 groups that I am comparing. I was just going to show the distribution of the groups with boxplots but a reviewer of my manuscript wants me to do group testing. The trouble is that pairwise ...
0
votes
0answers
15 views

Beginner questions about mediation analysis in psych intervention (2 intervention + 2 mediator system)

I am studying, and am fairly new to research methods and design, and hence hope to get some guidance on my analysis plan. I have 3 questions here, and attaching a structural model as reference. ...
1
vote
2answers
77 views

When (in the context of 'non-independent tests') does Bonferroni correction *not* yield the appropriate FWER?

I have a question about the oft-heard suggestion that a Bonferroni correction is appropriate in the context of posing multiple independent hypotheses and inappropriate elsewhere, in contrast to ...
1
vote
0answers
102 views

Am I (reviewer) correct or is the researcher correct re: Bonferroni?

I've been asked to review an article for a low-tier journal, such that I am the only reviewer with stats knowledge (such as it is; I used to teach stats and advanced stats, but it's been a few years ...
0
votes
0answers
26 views

log2 p-value of 1 after Holm-Bonferroni correction

I obtained rawp by performing Student's t-test function on dat.filtered and then I performed correction using Holm and Holm-Bonferroni methods. I have a huge increase in log2 adjusted p-value of 1.0. ...
0
votes
1answer
56 views

How to properly perform Bonferroni Correction with multiple datasets and questions

Assume I have 6 algorithms, A, B, C, D, E, F and I want to perform a series of tests to see where A outperforms the others and B outperforms the others. In this case, I have 9 hypotheses I am testing: ...
1
vote
0answers
13 views

Is there a way to adjust FDR threshold using correlation across tests (aka, number of estimated independent tests, like in Bonferroni correction)

For Bonferroni correction, correlations can be used to inform correction of multiple testing using the spectral decomposition of matrices. For example, in GWAS, the threshold is established to be 5e-8 ...
2
votes
2answers
55 views

If we do two one-sided tests instead of a two-sided test, can we get extra power by using a better multiple testing correction than Bonferroni?

A standard problem in statistics would be something like the following. $$ H_0: \theta = \theta_0 $$ $$ H_a: \theta \ne \theta_0 $$ This is a two-sided test. What if we did the following two tests? $$ ...
1
vote
1answer
92 views

ABC test/ hypothesis testing with multiple comparisons

I need to do a hypothesis test with three versions (one control version and two test versions). From what I understand I can just run a t-test (or z-test) for each test version vs control, and then ...
0
votes
1answer
36 views

Multivariate Testing

I have an ab test with two factors (color of image and size of image). Is the correct methodology to do these steps (assuming I want to do a full factorial): Calculate the f-stat for the control (...
0
votes
0answers
24 views

Multiple comparison when testing one planned hypothesis

I am running one logit model containing an interaction: logit dv year contract_type yearXcontract+Z where contract_type and year are dummies with 4 levels. After ...
2
votes
2answers
111 views

calculate adjusted p and t value after Bonferroni Correction

I am using linear mixed effect models to analyse my data. I am looking at how 4 independent variables might affect 5 dependent variables. I read an article by von der Malsburg and Angele (2017) which ...
0
votes
1answer
37 views

Are these confidence intervals already Bonferoni adjusted?

I'm new to STATA and just have a simple question I have the simple regression output seen below. I'm wondering if the 95% Ci of B1 and B0 is already bonferoni adjusted?
0
votes
1answer
36 views

Does $\alpha$ for Tukey change like it does for Bonferroni?

If you are trying to find confidence intervals, how does $\alpha$ for Tukey and Bonferroni differ? I understand that for Bonferroni the $\alpha$ is divided by the number of groups; however, does the $\...
2
votes
1answer
32 views

Inference based on multiple hypothesis tests

This might be a rather basic question, but I really want to get it right. When doing multiple hypothesis testing, researchers typically account for the "multiple" part by controlling for the ...
0
votes
0answers
4 views

When I interpret the output of n-way ANOVA or regression, should I adjust the p-values per each main and interaction effect or coefficient?

Statistical packages report regression and ANOVA/ANCOVA models consistently in the same manner - a list of model coefficients, standard error, test statistics (Wald, F) and the p-value. Let's assume I ...
1
vote
0answers
16 views

Bonferroni adjusted p values (from a dunns test)

Are these P values to the same alpha value; ie, if adj p > 0.05 then there is no significant difference? also what does it mean when p = 1 (because this has happened for some of mine yet none of ...
1
vote
1answer
44 views

Applying multiple test correction for stock prices

I want to work out if there are any associations between the stock prices within two indexes, S&P 500 and FTSE 100. I plan to perform a simple regression for each pair of stocks. I've read that I ...
0
votes
0answers
19 views

Confidence interval calculations Tukeys vs Bonferroni vs Fishers LSD?

I am trying to work out how the Confidence interval calculations for the Tukey, Bonferroni and Fishers LSD differ and/or are similar from each other? From my understanding they all consider the ...
0
votes
1answer
49 views

Justification of the Bonferroni correction

I've come across the Bonferroni correction in a genetics course. I understand the idea that we need to account for type I errors, but I can't fully get my head around it. In the genetics scenario, it ...
1
vote
0answers
31 views

P-value corrections on a growing dataset

My study design is following: I have N different financial funds and their performance over T years. At the start of each year I want to test hypothesis that they perform better than the benchmark. ...
0
votes
1answer
40 views

Separate Bonferroni corrections for multiple categories

I have a study in which we are looking at healthcare costs and utilization by patients with a disease vs. healthy controls (HC), as well as by different stratifications of disease (severity, ...
0
votes
0answers
81 views

Limiting the Bonferroni correction to 2 of 7 explanatory variables?

I have a single response variable (X) and a hypothesis that there exists a relationship between it and two explanatory variables (A, B). I also have five potential confounders (C, D, E, F, G), so a ...
0
votes
0answers
16 views

Number of planned tests using Bonferroni for a two-way ANOVA?

I have a two-factor study with factors A and B. There are 4 factors of A and 4 factors of B, with a replication of 3 for each. $$a = 4, b = 4, n = 3$$ After looking at the ANOVA table, I can see that ...
0
votes
0answers
17 views

Controlling for Type I error in logistic regression when DVs are correlated

I am hypothesizing that there are group differences (G1 and G2) on several health symptoms (S1 - S10) measured dichotomously (yes/no). I am adjusting for several variables (V1 - V5) but to simplify ...
0
votes
0answers
12 views
4
votes
1answer
235 views

Bonferroni correction versus the F-test

Many sources emphasize the importance of the F-test p-value in multiple regression and justify this in terms of p-hacking. It's kind of intuitive that if you can't reject the null hypothesis that all ...
0
votes
0answers
17 views

Lilliefors test for paired series + Bonferroni correction

I got two series where I want to find out if they differ. The two series are paired meaning they are values for the same subjects. N = 21 After: 0.87348565, 0.88276705, 0.89083918, 0.89520159, 0....
0
votes
1answer
100 views

Is Bonferroni correction needed for paired t-test on grouped data?

I have a survey that was conducted in 3 different classes ( math, phys, bio) at the beginning and at the end of the semester ( pre and post). In the survey, there were 3 groups of questions (A, B, C) ...
0
votes
0answers
41 views

sample size calculation with multiple outcomes

I am hoping to perform a sample size calculation for a study that will have two primary outcomes and will compare two groups (experimental and control). The models will either be a simple test of post ...

1
2 3 4 5
7