Bonferroni refers to a method of adjustment for multiple comparisons in which the p value is divided by the number of tests. It is more conservative than many other such methods.

learn more… | top users | synonyms

4
votes
1answer
34 views

What is the intuition for dependence assumption in Benjamini and Hochberg (1995)?

For dealing with multiple testing, Benjamini and Hochberg (1995) assume that the tests are independent. Benjamini and Yekutieli (2001) show that (1) the 1995 result also holds under (I think) weaker ...
3
votes
1answer
33 views

Using a binomial approach to calculate significance of multiple significance tests

I have 10 data sets with normal distributions and carrying out significance tests on each one. With this many data sets, there is a good chance that some will be significant by chance. I am going to ...
2
votes
2answers
80 views

Post-hoc tests for MANOVA

I am using a MANOVA test to compare nine different dependent variables (from neuropsychological and neuropsychiatric assessment) between three groups. The output shows a significant influence from ...
0
votes
1answer
78 views

When to correct p-values in multiple comparisons?

I'm afraid that related questions didn't answer mine. We evaluate the performances of >2 classifiers (machine learning). Our Null hypothesis is that performances do not differ. We perform parametric ...
2
votes
1answer
62 views

When should I use FDR or Bonferroni in multiple comparisons?

I have the following situations : (1) Very large number of variables - say 100 variables and 200 samples (observations). Although level of each variable is just 2 or 3 or 4 max. I am doing linear ...
0
votes
2answers
23 views

What are correlations among between test statistics in multiple comparison procedures

In the book Multiple Comparisons Using R Bretz et al write that their R package multcomp improves power above other multiple comparison procedures by "accounting for the structural correlations ...
0
votes
0answers
16 views

Multiple testing correction when confirming a correlation with an extended dataset

Consider a given dataset on which I performed a set of independent tests on different (non-independent) variables, and obtained several p-values. I estimated the number of independent tests as being ...
0
votes
1answer
31 views

Multilevel Modeling and multiple testing

I am interested in working out the correct way to correct for multiple testing in multilevel models for longitudinal data, where I am investing a potential interaction between two predictors. My ...
5
votes
1answer
168 views

Does testing for assumptions affect type I error?

I just performed simple simulation. Made two "populations" with different means and the same variance. Since I prepared them I know that they: are normal, differs in location and both have the same ...
1
vote
0answers
78 views

Multiple Hypothesis testing correction on non-independent data

Consider a vector $X_i$ with $i \in [1,N]$. Let us say that you divide the vector into two $X_{i}'$ with $i \in [1,j]$ and $X_i''$ with $i \in [j+1, N]$ for several $j$; then for each pair of ...
0
votes
0answers
52 views

Bonferroni-correction after one-way ANOVA

I conducted a one-way ANOVA: I had two groups and tested, if they differ in three different DVs. Since I tested three times, I thought it was wise to go for Bonferroni-correction. But then I got the ...
0
votes
0answers
25 views

Identifying formulas from MANOVA in a research paper

I need to identify the formulas for computing the group differences with Bonferroni corrections and Cohen's d effect size in the following paper (chart on the top of page 6): Research Paper What ...
0
votes
0answers
27 views

Mann-Whitney or ANOVA?

I am conducting a randomized vignette survey and would like to know if Mann-Whitney is the best test to use . Participants will be randomized to an identical scenario where a patient deteriorates ...
3
votes
2answers
48 views

Joint Confidence Interval

Given the following model: ${\rm Wage}_i=\beta_0+\beta_1 {\rm Married}_i+\beta_2{\rm Female}_i+\beta_3 {\rm Married}_i \times {\rm Female}_i + \varepsilon_i$, I am interested in finding ...
21
votes
2answers
1k views

Is the exact value of a 'p-value' meaningless?

I had a discussion with a statistician back in 2009 where he stated that the exact value of a p-value is irrelevant: the only thing that is important is whether it is significant or not. I.e. one ...
1
vote
1answer
41 views

Partially orthogonal planned comparisons

I am comparing 8 means and want to set up a planned comparisons, rather than having my Bonferroni adjustment become overly-conservative in a post-hoc. For my groups I need to make a total of 16 ...
1
vote
1answer
42 views

Adjusting significance level with for different kinds of tests (chisq, mann whitney etc.) on the same variable

I have conducted a series of tests on a single, dichotomic variable (presence/absence of a personality disorder in a neurological illness) in order to compare several things (i.e. presence of anxiety ...
1
vote
0answers
82 views

Controlling for Type 1 Errors: Post-hoc testing on more than 1 ANOVA

I have conducted several repeated-measures ANOVA's with post-hoc testing via Tukey's HSD whenever the omnibus F-test was significant. I'm having some confusion about controlling for type 1 errors when ...
2
votes
1answer
173 views

Does one need to adjust for multiple comparisons when using MANOVA?

There are 10 dependent variables and 2 groups. When I wanted to compare 10 dependent variables between two groups, I performed MANOVA. I wonder if there is a multiple comparison problem in that case. ...
3
votes
0answers
100 views

Bonferroni correction for two different tests on the same dataset

I am having trouble figuring out the correction factor to choose for a Bonferroni correction. Let me explain. I have two datasets, control ($x$) and data from left ($y_1$, affected limb) and right ...
1
vote
1answer
30 views

Bonferroni correction for gene functional category enrichment

I have performed a clustering on 20K genes, and I want to understand how coherent resulting clusters are by calculating the significance of the overlap of the clusters with a number of known GO gene ...
0
votes
1answer
35 views

what kind of non-parametric test to use?

I collected data on one sample, the DV could be separated into two groups (success yes vs. no) and then I have several IVs with interval scale. I just don't know if to use Wilcoxon or Man-Whitney ...
7
votes
2answers
305 views

Why use Bonferroni over Holm-Bonferroni?

I can see why you might not use a more powerful method, such as the Hochberg method, over the Bonferroni correction, as they may have extra assumptions, such as the independence of hypotheses in this ...
0
votes
0answers
53 views

Multiple Comparisons Correction on Peason's Chi-Squared Tests

I have a dataset consisting of 1000 samples (rows). Each sample consists of 300 categorical data points (columns). The process of generating these data points has a known probability of generating an ...
0
votes
0answers
22 views

How to run a Simes test?

I'm looking to set up a Simes test in R. I'm trying to show that a Simes test is conservative with when a positive correlation exists. I know how correct the p-values using the p.adjust function, ...
3
votes
0answers
75 views

Is Bonferroni correction too liberal for some dependent hypotheses?

I frequently read that Bonferroni correction also works for dependent hypotheses. However, I don't think that is true and I have a counter example. Can somebody please tell me (a) where my mistake is ...
0
votes
0answers
21 views

SPSS-independent-sample-t-test [duplicate]

I want to test mean differences between 4(0,1,2,3) groups for 6 different cell-types. In other words, I would have 6 comparisons to do for each cell-type (0-1,0-2,0-3,1-2,1-3,2-3). I have run a ...
3
votes
1answer
58 views

How to compare several small groups with follow-up data?

I have 4 treatment groups: 1 control (placebo) 1 with X treatment 1 with Y treatment 1 combination-treatment-group XY and I have monitored the individuals ...
2
votes
0answers
76 views

Bonferroni Multiple comparisons — testing for any true

I am doing multiple comparisons with several thousand independent alternatives. None are statistically significant when Bonferroni corrected; however, many more are significant when uncorrected than ...
3
votes
0answers
90 views

Comparing the mean of several exponentially distributed variables

(Possibly related to Is there a nonparametric equivalent of Tukey HSD?) Given a set of several exponentially distributed variables (representing data which is modeled this way), I'd like to check ...
10
votes
1answer
555 views

How and when to use the Bonferroni adjustment

I have two questions regarding when to use a Bonferroni adjustment: Is it appropriate to use a Bonferroni adjustment in all cases of multiple testing? If one performs a test on a data set, then one ...
7
votes
0answers
173 views

p-value adjustment for Local Moran's I statistic (LISA)

I'm working with some exploratory spatial analysis in R using spdep package. I came across an option to adjust p-values of local indicators of spatial association (LISA) calculated using ...
6
votes
1answer
717 views

How to apply multiple testing correction for gene list overlap using R

I have 2 studies looking at the patient response to the same drug. Study 1 found 10,000 genes expressed above the background and 500 of them are differentially expressed and referred to as the drug ...
1
vote
1answer
85 views

How can I perform a familywise error rate correction for Johansen cointegration tests?

I want to test multiple possible cointegration relationships with the Johansen cointegration test. I'm currently using the urca package in R with the ca.jo test. I was going to use the Bonferroni ...
0
votes
2answers
9k views

Bonferroni adjustment in SPSS - what does it do?

first post, so go easy... I'm probably being a bit simplistic here, but from what I understand the Bonferroni post hoc test is simply when we adjust the alpha to account for inflated error when ...
3
votes
1answer
101 views

Question regarding Bonferroni correction

Prove the following version of the Bonferroni inequality- $$P\left(\bigcap_{i=1}^kA_i\right)\ge1-\sum_{i=1}^kP(A_i^c)$$ When creating simultaneous confidence interval, what are $A_i$ and $A_i^c$? ...
1
vote
1answer
2k views

Two questions about Bonferroni adjustment

I have two questions about Bonferroni adjustments: 1). Can one use the Bonferroni method to compare independent groups? The reason why I ask this is it seems that many examples I've encountered ...
1
vote
1answer
284 views

Determining number of tests to correct for using Bonferroni

I am doing final revisions to my dissertation, which includes a large number of regressions. I have six outcome variables (DVs) and I am testing the effect of four primary (correlated) IVs, but I am ...
2
votes
0answers
44 views

Is this a valid method to control the FWER?

I have a huge number, say $M$, of hypotheses that are potentially correlated. I have a dataset $D$ of random samples from an unknown distribution and I want to do test the hypotheses for significance ...
3
votes
1answer
1k views

Anova repeated measures is significant, but all the multiple comparisons with Bonferroni correction are not?

I counducted a mixed model ANOVA with time as the within subjects variable (3 levels) and group as the between subjects variable (2 levels). All is significant (time, group, and timeXgroup). To test ...
5
votes
2answers
4k views

Post hoc test in a 2x3 mixed design ANOVA using SPSS?

I have two groups of 10 participants who were assessed three times during an experiment. To test for differences between groups and across the three assessments, I ran a 2x3 mixed design ANOVA with ...
1
vote
0answers
738 views

Non parameteric tests for 2*2 mixed design

I have 2*2 mixed design, where the two factors are time and group. The time levels are pre- and post-intervention, and the group levels are control and intervention. If I were using a parameteric ...
1
vote
1answer
287 views

Strict Bonferroni - looking for p value

I run 18 exploratory regressions and would like to know what the strict Bonferroni p-value would be. Nine regressions are negative binomial regressions with 4 predictor variables, the other 9 are WLS ...
0
votes
0answers
35 views

Bonferroni post hoc test does not reveal significant differences between groups [duplicate]

Possible Duplicate: How can I get a significant overall ANOVA but no significant pairwise differences with Tukey's procedure? I used a repeated measures ANOVA for my dissertation, with ...
2
votes
2answers
145 views

Best way to go about correcting for multiple comparisons (or should one even) for a small dataset?

Disclaimer: I have a very minimal stats background and am just getting into this stuff. Corrections to any terminology I use or an improved rephrasing would be super appreciated. I have a simple ...
2
votes
0answers
366 views

Bonferroni correction in generalised linear model

I am running a generalised linear model in R. I have a single response variable and a maximum of 4 possible explanatory variables. I am adding each explanatory variable to the model sequentially, ...
1
vote
1answer
951 views

Permutation tests in R for correlations

What's the easiest way to run permutation tests for a bivariate Pearson correlation matrix in R? I have tried running MPT.Corr, Multivariate Permutation Test for Correlations by Urbano Blackford et ...
4
votes
2answers
3k views

Bonferroni and Greenhouse-Geisser corrections with repeated-measures ANOVA

I guess my question comes in two parts: Following a 1x7 repeated-measures ANOVA, I carried out 3 pre-planned tests. Two were t-tests between 2 of the conditions. The third is a 1x6 ANOVA (to test ...
1
vote
0answers
497 views

Multiple comparisons process using Kruskal-Wallis, G-Test and corrections

I want to know if changing a button color has a positive effect on different measures (see the 4 measures below), using the following process: There's a default button, which is the fixed control ...
8
votes
3answers
7k views

Sidak or Bonferroni?

I am using a generalized linear model in SPSS to look at the differences in average number of caterpillars (non-normal, using Tweedie distribution) on 16 different species of plants. I want to run ...