I have some data I want to analyze, for which I was curious whether one or more conditions lead to measurements significantly different from the rest. I started out by making a table of t-test p-values comparing the condition which looked different in my graphic plot with all the others.
I was told by people in my lab that that was wrong (because it constitutes data mining) and instead I should use ANOVA. So I started reading about how to use that.
Then I came here and I was told that ANOVA is generally (and especially in my case where I am doing repeated measures) inferior to linear models and that I should use linear models. Sounded good.
When finally writing up my results I used both ANOVA and LME, and got results which I found conflicting (ANOVA had a p value of 0.09, whereas LME found 3 significantly different categories) - and I asked about that in my other thread here. There I was told the following about ANOVA:
Him: "It is a well-known fact that a non-significant omnibus test does NOT imply that none of the comparisons subsumed under that omnibus test are significant."
Me: Wouldn't what you are saying amount to "it is normal for omnibus tests (here ANOVA) to lead to false negatives"?
ANOVA is, as is stated elsewhere there, sensitive to the average effect and thus can generate significant F's due to multiple differences averaging to significant with no significant paired test.
And the following about LME:
Me: so if I run lme I would have to define a priori which conditions interest me, and simply picking the most significant ones a posteriori means I'm data mining?
So my questions are:
If both false negatives and false positives are "normal" for ANOVA - what use are ANOVA results in my case (or in any case)?
If determining significantly different conditions (a posteriori) from LME output constitutes data mining (as would my initial table of t-test p-values) - what use are LME results in my case?