# What's the value of ANOVA/LME?

I have some data I want to analyze, for which I was curious whether one or more conditions lead to measurements significantly different from the rest. I started out by making a table of t-test p-values comparing the condition which looked different in my graphic plot with all the others.

I was told by people in my lab that that was wrong (because it constitutes data mining) and instead I should use ANOVA. So I started reading about how to use that.

Then I came here and I was told that ANOVA is generally (and especially in my case where I am doing repeated measures) inferior to linear models and that I should use linear models. Sounded good.

When finally writing up my results I used both ANOVA and LME, and got results which I found conflicting (ANOVA had a p value of 0.09, whereas LME found 3 significantly different categories) - and I asked about that in my other thread here. There I was told the following about ANOVA:

Him: "It is a well-known fact that a non-significant omnibus test does NOT imply that none of the comparisons subsumed under that omnibus test are significant."

Me: Wouldn't what you are saying amount to "it is normal for omnibus tests (here ANOVA) to lead to false negatives"?

Him: Yes.

Plus

ANOVA is, as is stated elsewhere there, sensitive to the average effect and thus can generate significant F's due to multiple differences averaging to significant with no significant paired test.

Me: so if I run lme I would have to define a priori which conditions interest me, and simply picking the most significant ones a posteriori means I'm data mining?

Him: Yes.

So my questions are:

• If both false negatives and false positives are "normal" for ANOVA - what use are ANOVA results in my case (or in any case)?

• If determining significantly different conditions (a posteriori) from LME output constitutes data mining (as would my initial table of t-test p-values) - what use are LME results in my case?

• ANOVA is a linear model. For repeated measures, people may have been suggesting a multi-level model. – Peter Flom - Reinstate Monica Nov 18 '13 at 14:05
• Then why do for instance R's stats::aov and nlme::lme give completely differently formatted results? – TheChymera Nov 18 '13 at 14:20
• Because the two methods evolved in parallel and unknown to each other. The output looks different but means the same thing. ANOVA largely evolved in agriculture and multiple regression in geography. – Peter Flom - Reinstate Monica Nov 18 '13 at 14:23
• Then why do people recommend linear models over anova? – TheChymera Nov 18 '13 at 14:39
• That thread is full of comments by me; I did not recommend a linear model over ANOVA, I recommended a multi-level model over repeated measures ANOVA – Peter Flom - Reinstate Monica Nov 18 '13 at 14:48