5
$\begingroup$

In short

I recently had a little conversation on the lme4 project's GitHub on how to properly test the significance of effects in a mixed-effect model, which made me realise that I really didn't know/understand much about what I was doing, namely a type I sum of squares.

So I read a bit more about the different types of sum of squares (e.g. this page), and I started wondering: since it doesn't really make sense to test an interaction without including the corresponding main effects (let's say testing Trial:Condition without having Condition in the model), does it make sense to including a random slope for a term that is absent from the fixed effects?

Full example (in case it's not clear)

Let's consider the following mixed-effect model:

Prop ~ Trial + Condition + Trial:Condition + (1 + Trial | Participant)

Now let's say that I want to do a type II sum of squares (correct me if I'm wrong, by the way), I'll need to run these model comparisons:

order1           <- Prop ~ Trial + Condition + (1 + Trial | Participant)
order1.condition <- Prop ~         Condition + (1 + Trial | Participant)
order1.trial     <- Prop ~ Trial             + (1 + Trial | Participant)
order2           <- Prop ~ Trial + Condition + Trial:Condition +
                             (1 + Trial | Participant)

anova(order1.condition, order1.full) # Test effect of Trial
anova(order1.trial, order1.full) # Test effect of Condition
anova(order1.full, order2) # Test effect of Trial:Condition

Now, in my model order1.condition above, Trial is deleted from the fixed-effect term, but I still tell the model to compute a random slope for Trial by Participant. Does it matter? Should I also delete Trial from my random effects?

P.S.: I just write in pseudo-code, to give you an idea of what I'm talking about.

P.P.S.: the page I linked to about types of SS does say that you can do a type III SS violating the marginality principle, but I gathered from my conversation on GitHub that this was a really bad and nonsensical thing to do

$\endgroup$

1 Answer 1

9
+25
$\begingroup$

As a rule for lme4 and other packages with a similar parameterization (at least at the level of the user interface), it does not make sense to have random slopes for terms not present in the fixed effects.

The reason for this is straightforward: the random effects (or more precisely, the BLUPs / conditional modes) are computed as offsets from the population-level / fixed effects. So if a given fixed effect is missing, then this is equivalent to assuming that the population-level effect is zero. This is a rather strong assumption and not one we generally want. It will also mess up the estimation of the variance (the actual critical part for random effects, which are in other contexts called variance components), because the variance is calculated as the mean squared distance to the mean and if your assumed mean doesn't match the actual one, then your variance will be wrong. (Note that this is part of the reason for calculating random effects as offsets from the population mean: it means that the random effects have mean equal to 0, so that part of the formula just cancels out.)

As an example of the repercussions of this, consider the following two models:

m <- lmer(Reaction ~ 1 + (1|Subject), sleepstudy)
m.0 <- lmer(Reaction ~ 0 + (1|Subject), sleepstudy)

(I'm using the intercept term here for simplicity instead of dealing with slopes, but the same ideas hold equally.)

The random effects show that the models actually use the offsets :

> ranef(m)
$Subject
    (Intercept)
308   37.829172
309  -72.209815
310  -58.536726
330    4.087222
331    9.476087
...

> ranef(m.0)
$Subject
    (Intercept)
308    341.3933
309    214.7671
310    230.5013
330    302.5651
331    308.7663
...

The first set include negative values because some subjects are faster than the population average, while the second set includes only positive values because all subjects had positive reaction times.

We can also extract the individual predictions by combining the offset and the population mean, lme4 will helpfully do this for you:

> coef(m)
$Subject
    (Intercept)
308    336.3371
309    226.2981
310    239.9712
330    302.5951
331    307.9840
...

(For m.0, this is of course identical to the random effects.)

Note that these values do not match up with the random effects from m.0. This is important -- the random effects for both models are shrunk towards 0, but for m this corresponds shrinking just the offsets, i.e shrinking the individual predictions towards the (grand) mean. For m.0, this corresponds towards shrinking the individual predictions towards 0. This will of course yield different results -- all the individual predictions in m.0 become smaller, but the individual predictions in m can become bigger or smaller, depending on whether an individual subject was faster or slower than the (grand) mean reaction time.

The variance estimates also differ:

> VarCorr(m)
 Groups   Name        Std.Dev.
 Subject  (Intercept) 35.754  
 Residual             44.259  
> VarCorr(m.0)
 Groups   Name        Std.Dev.
 Subject  (Intercept) 300.505 
 Residual              44.259

Clearly m.0 is wrong in some rather fundamental sense: the standard deviation between subjects is not 300.505! Now, overall m.0 does a decent job of fitting the data (with a similar log likelihood to m), but it does so less efficiently (because the computational assumptions of the model is not met) and with parameter estimates that are incorrect/misleading.

Now, it is possible to parameterize mixed models such that random effects aren't centered (or "spherical") in this way, and indeed I believe brms uses a non-centered parameterization for its Stan code (there's something about the way the centered parameterization creates weird chokepoints in the critical set for Hamiltonian MCMC), but the formula interface for the most popular packages -- nlme, lme4, brms, rstanarm -- nonetheless requires a centered specification.

Since you've recently discovered the different types of sums of squares, make sure to check out

  • Venables' Exegeses on Linear Models, which is often mentioned in such discussions, especially with regards to whether Type-III SS even examine interesting hypothesis (instead of the usual rant about whether they make "sense").

  • John Fox's excellent book Applied Regression Analysis & Generalized Linear Models. The index coveniently has an entry "Marginality, principle of" with references to many different points in the text whether issues related to this (and thus Type II vs III SS) come to play.

  • car::Anova() which can compute both Type II and III SS for lmer models, either using the $\chi^2$ distribution (i.e. treating the $F$ denominator degrees of freedom as infinite, analogous to treating $t$ values as $z$ values) or using $F$ distribution with Kenward-Roger approximated denominator degrees of freedom. (car is an abbreviation for "Companion to Applied Regression".)

  • lmerTest::anova() which will compute Type I, II and III sums of squares for lmer models using with options for using the Satterthwaite or Kenward-Roger approximations for the denominator degrees of freedom. Note that as of this writing, there is a major package rewrite in beta which generally improves computational efficiency (by caching the ddf approximations) compared to the current version on CRAN.

$\endgroup$
8
  • 2
    $\begingroup$ This demonstrates why one should be suspicios of models that have random slopes without the corresponding fixed effect. However it seems too strong to say that they don't make sense at all: they are necessary if for example one wants to test the null hypothesis for a given fixed effect using a likelihood ratio test (of course there may be better ways to test an effect). Such test would make no sense if the reduced model also had a different random effect structure. $\endgroup$
    – matteo
    Commented Apr 16, 2018 at 9:11
  • 1
    $\begingroup$ @MatteoLisi I didn't say that didn't make sense at all, in fact I spent a long time explaining the rule of thumb I put forth at the top. However, your own take is a bit self-contradictory: why would you want to compare a model to a suspicious (reduced fixed but same random effects) model? I don't have a problem with that per se, but it does answer a different inferential question than comparing the full model with a reduced (both in FE and RE) model. Which one is the appropriate comparison very much depends on your hypothesis and the assumptions you're willing to make about the data. $\endgroup$
    – Livius
    Commented Apr 16, 2018 at 12:30
  • $\begingroup$ Comparing the full model with one reduced both in FE and RE does not make much sense because you are mixing and confusing two different null hypotheses: one that the FE might be zero, and the other that the variance of the RE might be zero. Any difference in model likelihood could be due to either of the two, making the comparison useless. $\endgroup$
    – matteo
    Commented Apr 16, 2018 at 13:10
  • $\begingroup$ Reducing the both the FE and RE is testing the hypothesis that there is (simultaneously) neither population nor intergroup ("[systematic] interindividual differences") effects, which is a useful comparison for my work. Again, each of these tests different hypothesis and only the experimenter/analyst actually knows which one they care about. Also, note that the RE variance is somewhat confounded with the FE, as shown above in the text, so if you keep a RE that is absent from the FE, you're still not testing the FE alone, at least not with a centered parameterization. $\endgroup$
    – Livius
    Commented Apr 16, 2018 at 15:17
  • $\begingroup$ Unless of course, the fixed effect is truly zero, but it seems odd to the make the test of something depend on that something being true. "begging the question" and all that. $\endgroup$
    – Livius
    Commented Apr 16, 2018 at 15:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.