The lmerTest package provides an anova() function for linear mixed models with optionally Satterthwaite's (default) or Kenward-Roger's approximation of the degrees of freedom (df). What is the difference between these two approaches? When to choose which?
$\begingroup$
$\endgroup$
5
-
5$\begingroup$ See the companion paper Kuznetsova et al, 2017, lmerTest Package: Tests in Linear Mixed Effects Models. $\endgroup$– amoebaJan 7, 2018 at 15:02
-
3$\begingroup$ In the Discussion they say "From our practice, we observed that the p values that the approximation methods provide are generally very close to each other. Schaalje, McBride, and Fellingham (2002) performed a number of simulations in order to investigate the appropriateness of the approximation methods. They discovered that complexity of the covariance structures, sample size and imbalance affect the performance of both approximations. However, these factors affect the Satterthwaite’s method more than the Kenward-Roger’s." $\endgroup$– amoebaJan 7, 2018 at 15:20
-
1$\begingroup$ Two examples where KR gives more appropriate dfs than Satterthwaite: stats.stackexchange.com/questions/320895 and stats.stackexchange.com/questions/84268. $\endgroup$– amoebaJan 13, 2018 at 17:45
-
1$\begingroup$ Another example: stats.stackexchange.com/questions/331541. $\endgroup$– amoebaMar 4, 2018 at 17:47
-
2$\begingroup$ The article Evaluating significance in linear mixed-effects models in R by Steven G. Luke has some nice comparisons of these methods. It concludes that both KR and Satterthwaite derived from REML models produce acceptable Type I error rates even for smaller samples. $\endgroup$– cbrnrJun 14, 2018 at 9:06
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
I'm also interested in figuring out what the difference might be. The best I can offer you, for now, is that this blog post suggests that the Kenward-Roger approximation is slightly, but probably not significantly, more conservative than the Satterthwaite approximation. The author also notes that they are both more conservative than the normal approximation, but again, not by much if the sample size is high enough. I'm not sure whether or not this was a generalizable conclusion of the author's or not though.
Edit: I will add that the article "A comparison of denominator degrees of freedom approximation methods in the unbalanced two-way factorial mixed model" by K.B. Gregory seems to indicate that neither method is typically a better method, although there are apparently occasions where the Kenward-Roger approximation loses some level of conservativeness.
-
3$\begingroup$ it's Kenward-Roger (no "s") ... Kenward-Roger's if you insist ... but usually expressed without the 's ... see also link.springer.com/article/10.1198/108571102726 $\endgroup$ Jan 4, 2015 at 23:00
$\begingroup$
$\endgroup$
Another difference between the two methods is described in Luke (2017):
Both the Kenward-Roger (Kenward & Roger, 1997) and Satterthwaite (1941) approaches are used to estimate denominator degrees of freedom for F statistics or degrees of freedom for t statistics. SAS PROC MIXED uses the Satterthwaite approximation (SAS Institute, 2008). While the Satterthwaite approximation can be applied to ML or REML models, the Kenward-Roger approximation is applied to REML models only.
(my bold)
- Luke, S.G. (2017). Evaluating significance in linear mixed-effects models in R. Behavior Research Methods, 49:4, 1494-1502. https://doi.org/10.3758/s13428-016-0809-y
$\begingroup$
$\endgroup$
1
"This latest result uses the Satterthwaite method, which is implemented in the lmerTest package. Note that, with this method, not only are the degrees of freedom slightly different, but so are the standard errors. That is because the Kenward-Roger method also entails making a bias adjustment to the covariance matrix of the fixed effects"
https://cran.r-project.org/web/packages/emmeans/vignettes/sophisticated.html
This is about degrees of freedom in emmeans, but I think might be useful.
-
$\begingroup$ Not totally about df in emmeans. When K-R is selected, you also get the adjusted covariances. $\endgroup$ Mar 14, 2022 at 23:17