# ANOVA vs. t-tests (yet again)

I am aware that there are many questions on this site with basically the exact same title. I have read them all and still want to formulate my question as follows:

I am trying to understand the benefits of running an ANOVA followed by post-hoc t-tests, vs. just running a sequence of t-tests to start with.

I have data for a study where there are three treatment groups for people receiving different kinds of psychotherapy, with three different measurement periods (pre-treatment, post-treatment, six-month follow-up). (The measure is a number resulting from a depression questionnaire.)

My understanding is I can set this up as a repeated measures ANOVA. If I run that, and see a significant result (p < some pre-determined value), my understanding is that all I know at that point is that there was some significant change across one or more time intervals for one or more of the treatment groups, and to find out which time/treatment combination(s) actually showed a significant difference I have to run a post-hoc t-test for each combination -- and to counteract the increased probability of getting a type I (false positive) error by running so many t-tests I can adjust the p-value limit to make it more strict.

If all of that is right, I'm having trouble understanding why (at least in my particular 3x3 case) it's better to use the ANOVA at all, instead of just doing the multiple t-tests with stricter p-values in the first place.

One thing I can understand is that if the result of the ANOVA is not significant, it's easier to find that out with the ANOVA instead of having to do the multiple t-tests. But is that the only reason? Or is the ANOVA doing something more for me?

There are a lot of paired comparisons among 3 treatments and 3 time periods. Depending on your level of curiosity, perhaps as many as $${9\choose 2}= 36$$ comparisons altogether.
Suppose there are no true differences anywhere in the experimental design, then each paired test at the usual 5% significance level has probability $$0.05$$ of rejecting $$H_0$$ to make a 'false discovery'. At worst, that might mean
• An average of $$36(.05) = 1.8$$ false discoveries per experiment and
• A probability $$1 - (.95)^{36} \approx 0.84$$ of at least one false discovery.
The advantage of doing an ANOVA first is that that you can do one test at the 5% level to see whether there are any differences at all. Then if you reject the overall $$H_0$$ at the 5% level, you can take an approach towards finding true differences that keeps the probability of false discovery low.