Skip to main content
Bumped by Community user
Bumped by Community user
Tweeted twitter.com/StackStats/status/1560778841391513607
Clarify my main question with some elaboration
Source Link
mjandrews
  • 283
  • 1
  • 9

So what exactly does $\mathit{SS}_i = \hat{\beta}^\top \mathbf{R}_i^\top \mathbf{R}_i \hat{\beta}$ represent, and how does it relate to the numerator sum of squares in normal linear models? Is there, for example, a way of writing the numerator sums of squares representin normal linear models in an analogous matrix way to $\mathit{SS}_i = \hat{\beta}^\top \mathbf{R}_i^\top \mathbf{R}_i \hat{\beta}$?

So what exactly does the sums of squares represent?

So what exactly does $\mathit{SS}_i = \hat{\beta}^\top \mathbf{R}_i^\top \mathbf{R}_i \hat{\beta}$ represent, and how does it relate to the numerator sum of squares in normal linear models? Is there, for example, a way of writing the numerator sums of squares in normal linear models in an analogous matrix way to $\mathit{SS}_i = \hat{\beta}^\top \mathbf{R}_i^\top \mathbf{R}_i \hat{\beta}$?

Source Link
mjandrews
  • 283
  • 1
  • 9

What exactly do Anova sums of squares represent in lmer models?

In a lme4::lmer model, the anova function returns an sequential analysis of variance table. As explained on page 34 of http://cran.nexr.com/web/packages/lme4/vignettes/lmer.pdf the numerator sum of squares for each F statistic is calculated by the following: $$ \mathit{SS}_i = \hat{\beta}^\top \mathbf{R}_i^\top \mathbf{R}_i \hat{\beta}, $$ where $\mathbf{R}_i^T$ is obtained from decompositions of matrices in the likelihood function, and the some of the key lme4 code that does this calculation can be found at https://github.com/lme4/lme4/blob/master/R/lmer.R#L590 for example.

Here's an example:

> library(lme4)
> M <- lmer(Reaction ~ factor(Days) + (1|Subject), data = sleepstudy)
> anova(M)
Analysis of Variance Table
             npar Sum Sq Mean Sq F value
factor(Days)    9 166235   18471  18.703

My question is what the denominator sums of squares, i.e. with the value 166235 above represents?

To be clearer about what exactly I am asking, consider the anova in a non-mixed-effects normal linear model, e.g. an lm model. There, what the numerator sums of squares represents is very clear: it represents the difference in the residual sums of squares of two nested models (see, for example, Maxwell, Delaney, & Kelley (2018) Designing Experiments and Analyzing Data: A Model Comparison Perspective for elaboration), which is also equivalent in this example to the total sums of squares minus the residual sums of squares. For example,

> M1 <- lm(Reaction ~ factor(Days), data = sleepstudy)
> anova(M1)
Analysis of Variance Table

Response: Reaction
              Df Sum Sq Mean Sq F value    Pr(>F)    
factor(Days)   9 166235 18470.6  7.8164 1.317e-09 ***
Residuals    170 401719  2363.1                      
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
> M0 <- update(M, . ~ . - factor(Days))
> sum(M0$residuals^2) - sum(M$residuals^2)
[1] 166235.1

(While the values of the relevant sums of squares in the lm and the lmer model in these examples are the same, that will not necessarily be the case).

In the lmer model, the Anova numerator sums of squares don't represent the difference of the residual sums of squares of the two relevant nested models. For example,

> M <- lmer(Reaction ~ factor(Days) + (1|Subject), data = sleepstudy)
> M3 <- update(M, . ~ . - factor(Days))
> sum(residuals(M3)^2) - sum(residuals(M)^2)
[1] 169535.2

So what exactly does the sums of squares represent?

More generally, what does the F statistic in a lmer model Anova represent? Again, in a non-mixed-effect normal linear model, the F statistic is transformation the likelihood ratio of the two nested models, specifically. $$ \frac{L_1}{L_0} = \left[ F \frac{\mathit{df}_0 - \mathit{df}_1}{\mathit{df_1}} + 1 \right]^{n/2} $$ For example,

> f <- anova(M1)[1,4]
> n <- nrow(sleepstudy)
> (n/2) * log(f * (M0$df.residual - M1$df.residual)/M1$df.residual + 1)
[1] 31.16589
> logLik(M1)-logLik(M0)
'log Lik.' 31.16589 (df=11)

But in a lmer model, it is not as simple as that:

> f <- anova(M)[1,4]
> n <- nrow(sleepstudy)
> (n/2) * log(f * (df.residual(M0) - df.residual(M1))/df.residual(M1) + 1)
[1] 61.93858
> logLik(M)-logLik(M3)
'log Lik.' 87.41698 (df=12)

In general it seems that the meaning of sums of squares and F statistics etc in normal linear models does not carry over to linear mixed effects model. But presumably there is still some important relationship between the concepts in these different contexts? But I am just not sure exactly what that is, and this question is an attempt to gain more clarity and insight.

For the record, this question What does the anova() command do with a lmer model object? is relevant and I found it and the accepted answer to be very useful. However, I do not think my question now is a duplicate, nor is the answer to that question an answer to what $\mathit{SS}_i = \hat{\beta}^\top \mathbf{R}_i^\top \mathbf{R}_i \hat{\beta}$ means and how it relates to numerator sums of squares in normal linear models.