6
$\begingroup$

I have 3 groups, one control group and two treatment groups that have nothing to do with each other. I just want to see if any of those 2 treatment are different from the control group.

Textbooks say 3 groups or more, use ANOVA to avoid type-1 errors.

But I just don't understand why that applies here. On the one hand, it's only 2 t-tests so doesn't seem like the chance of error is greatly increased. And on the other hand, being that have no interest in differences between the treatment groups, by doing an ANOVA can it not be misleading? (if the two treatments are different from each other but not from the control).

So, should I do an ANOVA and if the means are different perform a "Dunnett's"? Or just do the 2 t-tests?

$\endgroup$
6
  • 1
    $\begingroup$ The null hypothesis of the global F-test from ANOVA is of no importance here, so there is no reason to run an F-test first and then, depending on its "answer", dig deeper. If you want to test the two working hypotheses "First treatment is systematically different from control" and "Second treatment is systematically different from control", then the two t-tests might be the best way to go (depending on groups sizes etc.) $\endgroup$
    – Michael M
    Dec 2, 2013 at 18:33
  • $\begingroup$ By "systematically" I mean "on average" here. Wasn't too clear. $\endgroup$
    – Michael M
    Dec 2, 2013 at 18:39
  • 2
    $\begingroup$ You can go straight to Dunnett's test if you want to control the family-wise error rate; or, as @Michael says, do two t-tests, if you want to control the individual error rates for each. $\endgroup$ Dec 2, 2013 at 20:21
  • 1
    $\begingroup$ Will you be making any inferences of the kind, "A-control was significant while B-control was not significant and therefore there is a difference between A and B"? If so, then Dunnett's or two t-test are definitely not sufficient for your analysis. $\endgroup$
    – John
    Dec 2, 2013 at 20:22
  • 2
    $\begingroup$ Note that the two treatments cannot be "different from each other but not from the control", although you could find a pattern 'significant' contrasts that logically necessitates there are some type I or type II errors (more here, if you're interested). $\endgroup$ Dec 2, 2013 at 22:39

2 Answers 2

4
$\begingroup$

You don't have to run an ANOVA first, but most people do out of habit. (Whether reviewers will give you a hard time about not having done so is a separate issue.) Note that the original Dunnett's test required that the conditions have equal $n$s. The test has since been generalized, so it is fine if you do not have equal $n$s, just be sure you are running the right test (and citing it properly). You can also run two t-tests instead of either an ANOVA or Dunnett's test, but if you want to control for type I error inflation, you will need to use the Bonferroni correction as your tests would not be independent.

$\endgroup$
1
  • 3
    $\begingroup$ Perhaps worth emphasizing that using the Bonferroni correction on the two pre-selected t-tests would be a conservative way to control family-wise Type I error, & that Dunnett's test has higher power. $\endgroup$ Dec 3, 2013 at 9:44
-1
$\begingroup$

If you have three groups you should do an ANOVA (after checking assumptions of normality etc of course) which will test if the three groups differ overall. If that is the case you can then either do contrasts or post-hoc tests to test your hypotheses directly, e.g. does group 1 differ from group 2. How to do contrasts or post-hoc tests depends on the software you use (e.g. R, SPSS etc).

$\endgroup$
12
  • $\begingroup$ Hello JolJols, thank you for your comment. I can do post-hocs on SPSS and Dunnett's on GraphPad. So assuming you are correct, I now am sure of what to do. But, to be honest, I still don't understand why it is more correct to use ANOVA instead of 2 t-tests in this case. I mean, I don't want to sound stubborn or plain stupid, I'd really like to understand this. Because, if I test 3groups for any kind o difference between them,I can see why the ANOVA. If I test many treatments vs a control, I can also see why the ANOVA, but 2 treatmentsvs1 contrl it still seems like the t-test could be the way. $\endgroup$
    – user34963
    Dec 2, 2013 at 18:27
  • $\begingroup$ Although advisable, it is not necessary to do the ANOVA first as long as when one uses a Bonferroni correction on the t-tests to control for the type I error inflation, or use the less conservative Dunnett's test. Nevertheless, I would suggest still doing the ANOVA first as this is how most people would use in the case of three or more groups. $\endgroup$
    – crazjo
    Dec 3, 2013 at 11:24
  • $\begingroup$ @JolJols: Just doing it as a kind of warm up? And then using Dunnett's test for the analysis you're really interested in? $\endgroup$ Dec 3, 2013 at 11:32
  • 1
    $\begingroup$ The point is that the composite test - reject either null hypothesis only after a significant ANOVA and a significant Dunnett's - is unnecessarily conservative; Dunnett's test already controls the family-wise error rate for the pre-specified comparisons of treatments with a control. $\endgroup$ Dec 3, 2013 at 12:41
  • $\begingroup$ I agree @Scortchi, thanks for pointing this out again. $\endgroup$
    – crazjo
    Dec 3, 2013 at 14:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.