# ANOVA or Linear Mixed Model?

I have been running several linear mixed effects models for some data of my current project, and now I'm moving on to different data I have. I say that because I'm in a mindset to use LME, and didn't think about good ole ANOVA, though I don't think it's appropriate here.

Here's my design:

For this specific group, there's 30 subjects. Each subject reads 80 sentences in Spanish, and 80 sentences in English.

My first instinct was to run lmer, thinking that Sentence Language (Spa or Eng) is nested within Subject.

mod1 = lmer(totreadtime ~ langcode + (1|subject), REML=F, data=data)

> anova(mod1)
Type III Analysis of Variance Table with Satterthwaite's method
Sum Sq  Mean Sq NumDF DenDF F value  Pr(>F)
langcode 19268687 19268687     1  4740  3.8996 0.04835 *
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


Great, there's a difference between Spanish Sentences and English sentences!

Am I correct in using lmer? Just want to verify.

• Yes, this is perfectly fine. You might want to consider looking at emmeans to marginal means too. Commented Jul 18, 2019 at 17:13
• As @Isabella said: a fully crossed design probably is more appropriate. Most Phonetics journal would expect it. Commented Jul 18, 2019 at 19:45

Nice answer from @Noah! However, I wonder if a better way to conceptualize the model of interest is to treat both subject and sentence as fully crossed random grouping factors (since one could view the subjects and the sentences included in this study as being representative of a larger universe of subjects and sentences, respectively).

Subject and sentence are fully crossed because each subject reads each of the 80 + 80 = 160 sentences. According to Section 2.1.1 The Penicillin Data of the document available at http://lme4.r-forge.r-project.org/book/Ch2.pdf, two random grouping factors are fully (or completely) crossed provided that we have at least one observation of the outcome variable (i.e., totreadtime) for each combination of levels of the two factors:

xtabs(~ subject + sentence, data)


It seems that we have exactly one observation for each combination of subject and sentence in the current case.

In the proposed model conceptualization, the predictor variable langcode is a sentence-specific predictor variable. (The model could also include predictor variables that are subject-specific.)

With this conceptualization, the model of interest could be specified as:

mod2 = lmer(totreadtime ~ langcode + (1|subject) + (1|sentence), REML=F, data=data)


where sentence is a numerical sentence identifier (e.g., 1, 2, ..., 160) converted to a factor in R.

The reason I suggest this model conceptualization is because it is common in linguistics settings where a sample of subjects would be expected to rate a sample of items (with at least one rating per subject and item combination), in which case subject and item would be treated as fully crossed random grouping factors. See this article on Mixed-effects modeling with crossed random effects for subjects and items by Baayen et al. for more details: https://www.sciencedirect.com/science/article/pii/S0749596X07001398.

• This is the right idea! (+1) In my comment I focused on the two language comparison but yes, item should be a factor in itself. (I feel a bit thick for not mentioning it to begin with...) Commented Jul 18, 2019 at 19:43
• I agree with this. Thank you for bringing in your expertise in linguistics. This seems like a fuller model than original proposed.
– Noah
Commented Jul 18, 2019 at 21:18
• Cool! That's what I love about this forum - we're all here to help each other. 😊 Commented Jul 19, 2019 at 2:58
• @IsabellaGhement thank you so much! I love this idea and I will try it. I really appreciate the linguistic input Commented Jul 23, 2019 at 21:14
• @IsabellaGhement would you still recommend adding (1|sentence) even if all 160 sentences are unique? The 80 sentences they read in Spanish are not the same as the 80 sentences they read in Spanish Commented Aug 21, 2019 at 20:57

Your use of a multilevel model looks fine here. You could add a subject-language interaction to see if the difference between Spanish and English sentences varies across individuals (as this might initiate a future research question aimed at explaining that variability, if there is any). In any case, the coefficient on langcode should equal the difference in means.

An analysis that is common in psychology is a mixed ANOVA, with langcode as a within-subjects factor. Ideally you would get the same results. I'm not sure how to run such a model in R.