# R's stats::aov and nlme::lme produce conflicting results

I want to use both anova and linear models to test the assumption that my some of my categories have different means than the rest.

I am using stats::aov for anova and nlme::lme for linear modelling. The full code is available in this notebook.

Basically I end up getting:

Error: ID
Df  Sum Sq  Mean Sq F value Pr(>F)
Residuals  6 0.01198 0.001997

Error: Within
Df  Sum Sq  Mean Sq F value Pr(>F)
COI        7 0.01926 0.002752    1.92 0.0903 .
Residuals 42 0.06020 0.001433


and

Fixed effects: ER ~ COI
Value  Std.Error DF   t-value p-value
(Intercept)      0.00200000 0.01465712 42 0.1364525  0.8921
COIemotion-hard  0.05600000 0.02023699 42 2.7672098  0.0084
COIscrambling-06 0.04218045 0.02023699 42 2.0843242  0.0433
COIscrambling-10 0.02094737 0.02023699 42 1.0351029  0.3065
COIscrambling-14 0.00685714 0.02023699 42 0.3388420  0.7364
COIscrambling-18 0.02085714 0.02023699 42 1.0306445  0.3086
COIscrambling-22 0.04885714 0.02023699 42 2.4142493  0.0202
COIscrambling-26 0.02742857 0.02023699 42 1.3553681  0.1825


Which means that lme is telling me 3 groups may significantly lie outside the range of the others, while aov is telling me the probability of ANY GROUP AT ALL being different from the others is not significant (using a p=0.05 cutoff).

What am I to make of this? Do I understand these results correctly?

• That's probably because you're testing the combined effect (7 degrees of freedom) at .05 in the first case, whereas you're testing each difference with the reference category separately at .05, 1 degree of freedom at a time (more power, but also higher Type I error rate) Nov 17, 2013 at 18:49
• It is a well-known fact that a non-significant omnibus test does NOT imply that none of the comparisons subsumed under that omnibus test are significant. This has nothing to do with aov() vs. lme(). Nov 17, 2013 at 18:50
• Is CDIscrambling actually a continuous variable that you've made categorical? To be clear, it's still continuous if, even though you've collected values 06, 10, 14, etc., other values are possible.
– John
Nov 17, 2013 at 19:28
• so what should I trust? lme or aov? Are the inflated lme significances because of multiple comparison bias? is lme making the same error I would make by doing pairwise t-tests? Nov 17, 2013 at 19:57
• You don't have to choose because the two methods do not contradict each other in any way. One presents the omnibus test while the other tests the individual contrasts. As I already said: "It is a well-known fact that a non-significant omnibus test does NOT imply that none of the comparisons subsumed under that omnibus test are significant." The following document I found online may be useful to you: www4.uwsp.edu/psych/cw/statmanual/… Nov 17, 2013 at 22:45

## 2 Answers

The analyses being presented aren't being fairly compared so there's no way to know if they're actually saying different things. They could be perfectly compatible. You could use lm (what underlies aov) and get the treatment effects just as you have for lme. You can also find omnibus F's using lme. Then you could compare things a little better but that's not really how you decide between mixed models and repeated measures ANOVA for a particular case. Regardless, the omnibus F in an ANOVA is not about simple effects so what you're comparing is apples and pizza.

The lme and aov commands use very different kinds of analysis methods. Baayen (2008) and others have demonstrated that the mixed models used in lme are generally more powerful compared to conventional repeated measures ANOVA and they also have fewer assumptions (e.g. sphericity). That said, there is the issue that mixed models are more sensitive to normality in residuals partially because with repeated measures ANOVA you benefit from the CLT to generate normality. There are many differences and the modelling methods function differently. You should be selecting one based on theoretical grounds and not particular results. A few years ago I was pretty much told by every statistician I met that you just should never use repeated measures ANOVA. I don't think that's as universal advice as they thought it was but perhaps that will help you.

It is possible that lme is finding effects not found in your simple ANOVA using aov. It's not practically possible to get truly comparable results to decide what to believe because they are doing different things and any tests mean (slightly) different things. That said, you could at least try anova(mylmeModel) to get omnibus F tests of the mixed effects model. Then you've got something substantially more comparable than what you're presenting here.

NOTE: I also asked if CDIscrambling was a continuous variable because it's a terrible habit of those who typically use repeated measures ANOVA to turn continuous variables into multiple categories. When using mixed models don't do that. You'll get much more power out of being able to treat it as a continuous variable. If it's not linear that's a different question and you can deal with that separately.

• The continuousness is a bit complicated, emotion is continuous and scrambling is continuous. And I plan to do a ER~scrambling*emotion later. But right now I just want to see if any one category is special and cannot be compared to the others by these continuous metrics because at 100% emotion something different (qualitatively) happens than at 40% emotion and all scrambling steps. Nov 18, 2013 at 10:54
• Perhaps somethings special happens but if that's your question you should really make a new one about that. Nothing you've presented here can come close to answering that. In the new question try to define what makes one "special" and add a plot or plots. Also mention why your categorization is theoretically important for the specialness. Aside, you seem to be taking the effects in the regression as the effects and imbuing them with extra meaning. They're just each level compared to the first in the sorted factor. That's unlikely a good way to define special.
– John
Nov 18, 2013 at 12:22
• I actually already did ask the question, though I did fail to include a figure (tell me if you still think the question needs one after reading it) and that is where I got the suggestion of using a linear model instead of anova. You seem to hold that linear models aren't the right way of going at this - what is? Nov 18, 2013 at 14:15
• And here is actually an other very similar question of mine with a figure reflecting what my data looks like. I'm actually sorry for asking similar questions over and over again, but with every new method I try it turns out that's not right either :-/ Nov 18, 2013 at 14:18
• Also, by your initial answer do you mean to say that linear models are not an alternative to anova but rather a sub-calss which anova uses - and that lme4/nlme ARE in essence ANOVAS but they simply use a linear model which is superior to aov's? could I get the same kid of output nlme::lme gives me from stats::aov Nov 18, 2013 at 14:44

(Converted from a set of comments.)

You don't have to choose because the two methods do not contradict each other in any way. One presents the omnibus test while the other tests the individual contrasts. It is a well-known fact that a non-significant omnibus test does NOT imply that none of the comparisons subsumed under that omnibus test are significant. This has nothing to do with aov() vs. lme().

The following document I found online may be useful to you:

The Meaning of the Omnibus F

wouldn't what you are saying amount to "it is normal for omnibus tests (here ANOVA) to lead to false negatives"?

Yes.

so if I run lme I would have to define a priori which conditions interest me, and simply picking the most significant ones a posteriori means I'm data mining?

Yes.

• All hypothesis tests tend to find a preponderance of false negatives. There's very little effort put out to control the beta rate.
– John
Nov 18, 2013 at 3:08
• @John Hear hear! Nov 18, 2013 at 3:13
• Just to be clear: I choose the "COI" as the measure of interest (a priori). In the plot I see that one category (A) in "COI" is always null. To test whether it truly is outside the range of all the others I run aov. It kind of isn't (p=0.09). I have heard linear models are better than anova, so I do lme. I find out that lme says the most likely categories to be out of range are B, C, and G - and in fact, not A. What conclusion can I draw from this? Also here 3 of 8 categories are out of range .Can 4 of 8 be out of range? If so which ones will lme anotate as significant? Nov 18, 2013 at 11:01