I have a group of subjects which did tests under 3 different conditions.
If I now want to compare variables with a ANOVA, is it right that I get the same p-values as if I would run 3 t-tests comparing each of the conditions with each other?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
No. Running three t-tests is not the same as running a single ANOVA. The critical difference is that each time you run a t-test, you'll inherit a Type-I error. Your overall error rate is proportional to the number of individual t-test that you run. The more you do, the more likely you will get a false positive.
We don't want to do this. Instead, we partition the variance in ANOVA. The single F-value will tell us whether any group is differ. We will have only a single p-value instead of three p-values.
Short answer: no, they are nothing alike.
We use an ANOVA to tell if one or more of the 3+ groups is different from the others. If that p value is significant, we then use a Tukey post-hoc test to determine the differences between groups. A Student T-test is not appropriate for comparing more than two groups.
The ANOVA, like other statistic tests, attempts to separate signal from noise -- it tests if there is any variation between groups that is not likely to be caused by variation within each group. By partitioning the sources of variation and comparing them, we can approximate a signal to noise ratio. High F-values are more likely with small sample sizes; the computer will spit out a useful p-value based on the F value and the degrees of freedom.
Comparing three T-tests just does not work very well. The T-test is a basic (though generally robust) tool for comparing two sample groups, but that is the extent of its purpose. The ANOVA was designed to better assess datasets of 3+ sample groups.