6
$\begingroup$

My question is about the best way to estimate the effect of a predictor on a dependent variable, while accounting for several other predictors that may correlate with the predictor of interest. I'm using a linear mixed-effects model, using the lmer function from the R lme4 package. (Warning: I'm fairly new at this, so their may be some misunderstandings woven through my question.)

The problem

To make things a bit more specific, I'll just explain the actual data that I'm working with. I have eye-movement data of participants freely viewing natural scenes. I want to determine whether pupil size predicts the 'visual saliency' (i.e. the conspicuity) of the locations in the image that participants are looking at. But there are many other things that correlate with pupil size, such as luminosity, and this makes the analysis tricky (or does it?).

Option 1 (simple): Looking at fixed effects

One option would be to simply create a linear mixed-effects model that has all predictors of saliency that I can think of, including the predictor of interest (pupil_size), as fixed effects and subject and scene as random effects. (To keep things manageable, I'm using a purely additive model, although I suppose that this is a whole topic in itself.)

my_lmer = lmer(saliency ~ brightness + (.. lots of predictors ...)
    + pupil_size + (1|subject) + (1|scene))

This will give me a t-value for the fixed effect pupil_size. From what I understand, this fixed effect will already be partial, so it's the unique predictive power of pupil size, with any correlations between fixed effects already taken into account. Is my understanding correct?

Option 2 (complex): Using model comparison

An alternative approach, which I have from Baayen et al. (2008), is to compare a model without pupil size as fixed effect (simple_model) to a model with pupil size as fixed effect (complex_model).

simple_model = lmer(saliency ~ brightness + (.. lots of predictors ...)
    + (1|subject) + (1|scene))
complex_model = lmer(saliency ~ brightness + (.. lots of predictors ...)
    pupil_size + (1|subject) + (1|scene))

Now I can use the anova function to compare these two models (see Baayen's paper for an example). This will give me a Chisq Chi value, and I can use this to determine whether adding pupil_size as fixed effect is a justified addition to the model.

Clearly, this model comparison approach is more complex than simply looking at the t-values for fixed effects in a single model. And it seems to me that if pupil_size is a significant predictor (per Option 1), then it must also be a significant addition to the model (per Option 2).

In sum, my question is: Is there any reason to do a model comparison (Option 2), or am I better off just creating a single linear mixed-effects model and seeing whether the t-value associated with pupil_size as fixed effect is sufficiently high (Option 1)?

$\endgroup$
3
  • 1
    $\begingroup$ Note that for option 2 you should set REML = FALSE, i.e., optimize the likelihood for the comparisons and only for parameter estimation use the default REML = TRUE. FWIW, I usually use option 2 (but I'm sure others have a more fundamental understanding of mixed effects models). $\endgroup$
    – Roland
    Commented Jan 27, 2014 at 10:22
  • $\begingroup$ Thank you, I was actually not aware that this makes a difference. Also, Baayen doesn't appear to do this in the paper that I'm referring to in the question. (Not that he's necessarily the authority on everything, but you have to rely on someone, right?) Do you happen to have a link/ paper/ resource that describes this point in a bit more detail? $\endgroup$
    – Sebastiaan
    Commented Jan 28, 2014 at 13:56
  • $\begingroup$ For a start you should study this. Also, see this CV question. $\endgroup$
    – Roland
    Commented Jan 28, 2014 at 14:18

1 Answer 1

1
$\begingroup$

I'm not sure if this is what Baayen mean, but one advantage of the model comparison approach is that you get to see the effect of adding pupil size on the other parameters, model fit statistics and so on.

This isn't specific to multi-level models; often people build up models, first using a simple model, then a more complex one, to see these changes.

$\endgroup$
1
  • $\begingroup$ Thank you for your answer, I hadn't it considered it from this angle, but it makes sense. Let me rephrase, to make sure I understand this point correctly: The model-comparison approach does not only allow you to test the effectiveness of the newly added predictor, but also whether the predictors that were in the model are affected by the addition. $\endgroup$
    – Sebastiaan
    Commented Jan 28, 2014 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.