1
$\begingroup$

I have a group of subjects which did tests under 3 different conditions.

If I now want to compare variables with a ANOVA, is it right that I get the same p-values as if I would run 3 t-tests comparing each of the conditions with each other?

$\endgroup$
  • $\begingroup$ The basic problem with the three t-tests is that their results are not independent: each compares a group to the same reference group. If you instead had three independent sets of reference data (six groups in all), then three t-tests would work. But even then it would be an inferior procedure, because you could do better by combining the three reference groups and running an ANOVA. $\endgroup$ – whuber Jun 21 '15 at 16:31
2
$\begingroup$

Well, think about it. You get one p value from anova, three p values from the t tests. How are you going to compare one apple with three oranges? So the answer is NO.

$\endgroup$
  • $\begingroup$ I believe the OP may be asking about the difference between performing three post hoc ANOVA comparisons and three separate t-tests. $\endgroup$ – whuber Jun 21 '15 at 16:32
  • 1
    $\begingroup$ Partly in response to @whuber. The ANOVA tells you whether the means differ, the t tests have hope of showing you how they differ. The latter is more informative, but you should correct the p values for multiplicity, e.g. via the Tukey procedure. The ANOVA is completely optional, in my opinion. You can do the comparisons pre-hoc if you like. $\endgroup$ – Russ Lenth Jun 21 '15 at 19:52
1
$\begingroup$

No. Running three t-tests is not the same as running a single ANOVA. The critical difference is that each time you run a t-test, you'll inherit a Type-I error. Your overall error rate is proportional to the number of individual t-test that you run. The more you do, the more likely you will get a false positive.

We don't want to do this. Instead, we partition the variance in ANOVA. The single F-value will tell us whether any group is differ. We will have only a single p-value instead of three p-values.

$\endgroup$
0
$\begingroup$

Short answer: no, they are nothing alike.

We use an ANOVA to tell if one or more of the 3+ groups is different from the others. If that p value is significant, we then use a Tukey post-hoc test to determine the differences between groups. A Student T-test is not appropriate for comparing more than two groups.

The ANOVA, like other statistic tests, attempts to separate signal from noise -- it tests if there is any variation between groups that is not likely to be caused by variation within each group. By partitioning the sources of variation and comparing them, we can approximate a signal to noise ratio. High F-values are more likely with small sample sizes; the computer will spit out a useful p-value based on the F value and the degrees of freedom.

Comparing three T-tests just does not work very well. The T-test is a basic (though generally robust) tool for comparing two sample groups, but that is the extent of its purpose. The ANOVA was designed to better assess datasets of 3+ sample groups.

$\endgroup$
  • $\begingroup$ Welcome to our site! Although all this is correct, it doesn't really explain why ANOVA is better than three t-tests. Would you be able to share your insight about that? $\endgroup$ – whuber Jun 21 '15 at 16:33
  • $\begingroup$ The OP didn't ask if/why one was better than the other. I can write an explanation of why ANOVA is more appropriate than T-tests, but I do not think it fits in the scope of this question. $\endgroup$ – Jonathan Voss Jun 21 '15 at 17:31
  • $\begingroup$ Although they did not ask explicitly why, this site favors explanations that go beyond the mere "what" of a situation. When only the "what" is appropriate, without further explanation, then we seek references to authoritative sources to back up an answer. $\endgroup$ – whuber Jun 21 '15 at 19:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.