# Running a mixed effects model in R with lmer()

I am currently trying to run a mixed effect model in R for a study I am conducting.

I am looking at the effect of target ethnicity on objectification and whether affinity mediates this relationship.

Each participant rated the warmth and competence of three targets (Asian, African American and Caucasian target) and a composite objectification score was computed by averaging the z-scored index for each item, per target.

The data look something like this:

participant <- c(1,1,1,2,2,2)

target_ethnicity <- ("caucasian", "africanamerican", "asian", "caucasian", "africanamerican", "asian")

objectification <- c(1.00, 0.90, 0.97, 0.78, 0.76, 0.89)

affinity_level <- c(5, 7, 7, 4, 6, 7)

df <- data.frame(participant, target_ethnicity, objectification, affinity_level)


So for each participant, there are three scores for objectification (one for each target) and three scores for affinity (one for each target).

When computing the mixed effect model, I used the code:

mod1 <- lmer(objectification ~ target + affinity + (1|participant), data = df)


When looking at the summary with my full data set of 131 participants it shows the following:

asian: B = 0.15976, t(259.87)= 4.159, p <.05
black: B = 0.07963, t(261.30)= 2.054, p<.05
affinity: B = 0.13647, t(338.11) = 3.169, p<.05


(1) I am confused why the degrees of freedom are all different?

(2) I want to test the assumptions of this model, but functions like qqPlot() do not work as I get the error message "Error in x[good]: object of type 'S4' is not subsettable In addition: Warning message: In is.na(x) : is.na() applied to non-(list or vector) of type 'S4'"

(3) When I look for outlying data points is it necessary to use the influence function before looking at dfbetas and cook's distance like I have done below? How do you choose what the cut off point is to remove outliers in mixed effect models?

outliers <- influence(mod1, group="participant")
dfbeta <- dfbetas.estex(outliers)
cooks <- cooks.distance.estex(outliers)


(4) How do I interpret the mediator affinity in this case? How would I know whether it is mediating? Is it necessary to or useful to do a simple slopes plot?

(5) How do I extract the R^2 and adjusted R^2 values for this model as well as the F statistic?

I know this is a lot of questions, but I am very new to mixed effect models and am quite confused.

I am confused why the degrees of freedom are all different?

Degrees of freedom for what ?

I want to test the assumptions of this model, but functions like qqPlot() do not work as I get the error message "Error in x[good]: object of type 'S4' is not subsettable In addition: Warning message: In is.na(x) : is.na() applied to non-(list or vector) of type 'S4'"

You can obtain a QQ plot of standardized residuals with:

> qqmath(mod1)


(3) When I look for outlying data points is it necessary to use the influence function before looking at dfbetas and cook's distance like I have done below? How do you choose what the cut off point is to remove outliers in mixed effect models?

Yes, you can identify possible outliers in this way, however, it is a very bad idea to simply use an arbitrary threshold and then delete those points that are above it. There are no statistical tests or rules of thumb that should be used as a basis for excluding observations in a model - this must come from domain knowledge. Note that this applies to statistical models in general, not just mixed effects models.

How do I interpret the mediator affinity in this case? How would I know whether it is mediating? Is it necessary to or useful to do a simple slopes plot?

If it is a potential mediator - that is, if it is on the causal pathway from ethnicity to objectification, then it should not be included in the model as this could invoke a reversal paradox - see Tu et al (2008). Note, again, that this applies to regression models in general, not just mixed effects models.

How do I extract the R^2 and adjusted R^2 values for this model as well as the F statistic?

R^2 is not defined for a linear mixed effects model. R^2 is a beautifully simple statistic that applies well to linear regression models, but does not generalize to GLMs and (G)LMMs

There have been many attempts to define a quantity that behaves similarly to R^2 for mixed models. See the excellent GLMM FAQ here for more detail.

References:

Tu, Y.K., Gunnell, D. and Gilthorpe, M.S., 2008. Simpson's Paradox, Lord's Paradox, and Suppression Effects are the same phenomenon–the reversal paradox. Emerging themes in epidemiology, 5(1), p.2.

• Thank you for your advice! For the degrees of freedom, I am confused why the df for the t-tests were all different? – Imogen Campbell Mar 8 at 11:50
• lme4::lmer() does not output degrees of freedom (for good reasons). Perhaps you are using a different package such as lmerTest, and if so these df will be using Satterthwaite's method for denominator degrees of freedom. – Robert Long Mar 8 at 13:32
• Yes, I am using the package lmerTest(). What would the reason be for lme4 not showing degrees of freedom? Also, what is most important to report in a mixed-effect model as I thought I should be reporting the Beta coefficients, t value, degrees of freedom, p-value, and F-statistic? – Imogen Campbell Mar 8 at 16:27
• @ImogenCampbell you can read all about this, from the main author of the lme4 package here – Robert Long Mar 8 at 16:33
• @ImogenCampbell as for what is imporant to report, I would say: the point estimates for all fixed effects and their corresponding SEs and CIs along with the variance of the random effects and the residuals. You may find that others suggest trying to compute the types of statistics you mention above, and that is why packages such as lmerTest exist (note that the authors of lme4 have nothing to do with lmerTest – Robert Long Mar 8 at 16:52