1
$\begingroup$

I'm doing some statistics work at my company, and we often do subgroup comparisons. For example, compare groups A, B, C, and D with each other.

Typically we do a 95% t-test of A vs B, A vs C, etc. However, isn't this technically incorrect, at least without some sort of Bonferroni correction? What is a simple way of testing multiple means against each other like this?

$\endgroup$
1
  • $\begingroup$ Hello, and welcome to CrossValidated! This sounds like a job for... the ANOVA: en.wikipedia.org/wiki/Analysis_of_variance Depending on your method of testing significance, you may need to implement Bonferroni correction. $\endgroup$ Aug 6, 2015 at 17:33

1 Answer 1

1
$\begingroup$

You're right that you need to do some kind of multiple testing correction. Whether or not you should use Bonferroni depends on the application.

If you're always going to be doing a $T$-test for control group $X_1$ versus treatment group $X_i$ for groups $i$ of $n$, you should look into Dunnett's test.

If you're going to be examining all possible combinations of $X_1$ to $X_n$ (i.e. $X_1$ vs $X_2$, $X_2$ vs $X_3$, $X_1$ vs $X_3$), then you're going to want to look into the different kinds of multiple testing correction available. There are two families of corrections generally: family wise error correction (FWER) which controls the chance of even one false positive at $\leq \alpha$, and false discovery rate control, which controls the proportion of false discoveries $ \leq \alpha$. The second gives more power, while the first gives more stringent results. You might have to do some reading to see which is best for your company (Bonferroni is generally too stringent, but I'm sure people here would have different opinions about that).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.