4
$\begingroup$

Can you specify one level of a factor as fixed and the other as random in a linear mixed model (with lmer)?

Some background information first: A set of speakers who vary in their proficiency levels (or dominance levels) in English and another language are asked to interpret as many words as possible once in English and once in another language within five minutes. The self-reported proficiency score is gradual and higher when participants are more proficient in English, near zero when they are proficient in both English and another language, and negative when they are more proficient in another language.

The hypothesis is that those who have higher positive proficiency scores will do better on the interpreting task in English than in the other language. Participants with negative proficiency scores will perform better in the task in their other language. One simple method to investigate correlations is to find Pearson’s r with two continuous variables; proficiency scores and the proportion of words interpreted in the English task (English/(English+another language).


measure_another_lang <- c(74, 120, 41, 79, 56, 45, 43, 56, 72, 53, 54, 64, 47, 90, 75, 54, 57, 37, 51, 63, 39, 44, 68, 32, 51, 58, 34, 48, 50, 67, 36, 50, 44, 80, 29, 53, 41, 46, 56, 50)

proficiency_scores <- c(87.08024, 111.3099, -55.53908, 86.68562, -2.42164, 31.64922, -12.33076, 13.32078, 37.69716, 66.44704, -33.88776, 24.32318, -111.3784, 27.18608, -37.84434, 84.1292, 86.96463, -83.71868, -3.848974, 43.27664, -71.236, 82.99149, 26.83826, 52.12666, -21.94731, 73.82172, -9.716439, 58.2229, -25.71618, -18.66557, -116.2157, -52.30922, 21.82515, 47.30964, 12.90472, 11.13121, -129.6731, 7.642772, -5.015314, -89.4537)

proportion_english <- measure_english/(measure_english+measure_another_lang)

cor(proficiency_scores, proportion_english)

However, Pearson’s r does not include the fact that the experiment includes repeated measures; each participant was measured twice. I assume that a linear mixed model would give more conservative results and I’ve included a suggestion in the r code below.

The task factor has two levels, English + any other language, and both levels are considered as fixed. But "any other language" sounds like a random factor to me. The only criterion for the participants is knowing English and any other language. Some participants are native English speakers, while other participants are non-native English speakers or exposed to both English and another language from birth. Language 1 consists of 15 different languages, with 1 to 5 speakers for each language. Is this an appropriate approach or do I need to take into consideration that languages were sampled randomly in one of the levels of the task factor?

data_frame_lmm <- data.frame(speaker_id = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40),
                             proficiency = c(87.08024, 111.3099, -55.53908, 86.68562, -2.42164, 31.64922, -12.33076, 13.32078, 37.69716, 66.44704, -33.88776, 24.32318, -111.3784, 27.18608, -37.84434, 84.1292, 86.96463, -83.71868, -3.848974, 43.27664, -71.236, 82.99149, 26.83826, 52.12666, -21.94731, 73.82172, -9.716439, 58.2229, -25.71618, -18.66557, -116.2157, -52.30922, 21.82515, 47.30964, 12.90472, 11.13121, -129.6731, 7.642772, -5.015314, -89.4537, 87.08024, 111.3099, -55.53908, 86.68562, -2.42164, 31.64922, -12.33076, 13.32078, 37.69716, 66.44704, -33.88776, 24.32318, -111.3784, 27.18608, -37.84434, 84.1292, 86.96463, -83.71868, -3.848974, 43.27664, -71.236, 82.99149, 26.83826, 52.12666, -21.94731, 73.82172, -9.716439, 58.2229, -25.71618, -18.66557, -116.2157, -52.30922, 21.82515, 47.30964, 12.90472, 11.13121, -129.6731, 7.642772, -5.015314, -89.4537),
                             task = c(2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1),
                             measure = c(74, 120, 41, 79, 56, 45, 43, 56, 72, 53, 54, 64, 47, 90, 75, 54, 57, 37, 51, 63, 39, 44, 68, 32, 51, 58, 34, 48, 50, 67, 36, 50, 44, 80, 29, 53, 41, 46, 56, 50, 59, 39, 43, 46, 38, 30, 40, 46, 37, 34, 47, 41, 67, 92, 73, 34, 39, 43, 72, 49, 82, 28, 49, 16, 44, 43, 39, 24, 40, 86, 39, 39, 48, 32, 22, 33, 72, 53, 54, 56))

summary(lmer(measure ~ proficiency + task + proficiency:task + (1|speaker_id), data = data_frame_lmm, REML=T))

This question is edited for clarification, adding the hypothesis and a description of other languages (thanks to EdM and Robert Long). I've also made the r code simpler and cleaner to run, including a data frame.

$\endgroup$
8
  • $\begingroup$ How was language 1 chosen for or assigned to each individual? For example, might these be non-native English speakers for whom language 1 is the native language? How many different languages are represented in language 1? Are any of those non-English languages used by only 1 person? $\endgroup$
    – EdM
    Aug 1, 2020 at 22:12
  • $\begingroup$ The only criterion is knowing English and any other language. Some participants are native English speakers, while other participants are non-native English speakers or exposed to both English and another language from birth. Language 1 consists of 15 different languages, with 1 to 5 speakers for each language. $\endgroup$
    – BlueMarlin
    Aug 2, 2020 at 0:07
  • 1
    $\begingroup$ So what is the specific hypothesis you wish to test with this study? $\endgroup$
    – EdM
    Aug 2, 2020 at 1:49
  • 1
    $\begingroup$ What is your research question ? $\endgroup$ Aug 2, 2020 at 12:34
  • $\begingroup$ The hypothesis (or research question) is that those who have higher positive proficiency scores will do better on the interpreting task in English than in the other language. Participants with negative proficiency scores will perform better in the task in their other language. I've added the hypothesis to the question. $\endgroup$
    – BlueMarlin
    Aug 2, 2020 at 15:38

1 Answer 1

2
$\begingroup$

I think this example (which strips out most of the stuff except the languages and the division into tasks) shows that lmer can estimate the difference between language groups (English vs. non-English) even when English is included as a group in the random effect. In your case there would be interactions between task and proficiency, but that shouldn't change the random effects structure.

This should work fine with variable numbers of speakers, as few as one per language (in a more extreme situation where most of the levels of the grouping variable have only one observation and a few have two, this is still theoretically possible but I wouldn't expect it to work well), and with a random effect of speaker as well.

You should think carefully about the maximal model that applies to this design, but not necessarily try to fit it ... for example, (task|speaker_id) is theoretically estimable (because each speaker sees each task). In principle, you could use (task|language/speaker_id) (because as I understand it, speakers are nested within languages in this case).

Make up a data frame with 10 observations each of 15 languages: language 1 is English, corresponding to task 2

dd <- data.frame(lang=factor(rep(1:15,each=10)))
dd$task <- factor(ifelse(as.numeric(dd$lang)>1,1,2))

Assign uniform values to each language, with 0 for English (the model will assume these values are normally distributed, not uniform, but it doesn't change much)

lval <- (0:14)*0.1

Make up a response with an added value of 0.5 for non-English languages

set.seed(101)
dd$y <- rnorm(nrow(dd),sd=0.02)+lval[dd$lang]+0.5*(as.numeric(dd$task=="1"))

For more complicated designs, I'd suggest using ?simulate.formula from lme4.

Fit the model:

m <- lmer(y~task+(1|lang),data=dd)
lattice::dotplot(ranef(m))  ## view random-effect 'estimates'

The estimated effect of 'task' is -1.24, which is nearly equal to the true difference of 0.5 between English (0) and the average of the non-English languages (the non-English values range from 0.1 to 1.4, so 1.5/2 + 0.5)

Predicted vs observed values by language:

pp <- predict(m,
              newdata=data.frame(lang=levels(dd$lang),
                                 task=c("2",rep("1",14))))    
boxplot(y~lang,data=dd)
points(pp,col=2,pch=1,cex=5)

I'd suggest experimenting with some more complex simulated examples that match your experimental questions more closely and reassuring yourself that the model can reliably extract the effects you're interested in ...

enter image description here

$\endgroup$
5
  • 1
    $\begingroup$ (+1) for demonstrating once again how useful mixed models can be. Would this work in the present situation, with some non-English languages only represented by a single individual and a desire to treat the individuals as random effects? That information is in a comment on the question. $\endgroup$
    – EdM
    Aug 2, 2020 at 18:14
  • 1
    $\begingroup$ see edits ..... $\endgroup$
    – Ben Bolker
    Aug 2, 2020 at 18:25
  • $\begingroup$ Would this work, @BenBolker? This is a suggestion based on your answer, but I am new to mixed models. 17 corresponds to English: data_frame_lmm$language <- c(17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 15, 6, 1, 2, 7, 12, 7, 16, 8, 5, 5, 6, 5, 9, 15, 11, 11, 7, 11, 11, 14, 2, 11, 11, 11, 6, 13, 6, 6, 13, 10, 6, 13, 12, 4, 6, 3, 7, 7, 13) followed by lmer(measure ~ proficiency + task + proficiency:task + (1|speaker_id) + (1|language), data = data_frame_lmm, REML=T) $\endgroup$
    – BlueMarlin
    Aug 3, 2020 at 2:02
  • $\begingroup$ I don't see why not. Have you tried it? You should definitely consider my comments about maximal models (e.g. allow for among-individual variation in task effects, not just in intercept ...) $\endgroup$
    – Ben Bolker
    Aug 3, 2020 at 2:05
  • $\begingroup$ Yes, I tried it and the summary() output looks fine. The proportion of the residual in random effects and the AIC/BIC decreased, which I guess is good. I tried to add among-individual variation with (task|speaker_id) and similar input, but I keep getting errors like "Error: number of observations (=80) <= number of random effects (=80) for term (task | speaker_id); the random-effects parameters and the residual variance (or scale parameter) are probably unidentifiable." I've not understood those errors completely.It seems that my data only can be used with "(1|group)" constructions. @BenBolker $\endgroup$
    – BlueMarlin
    Aug 3, 2020 at 2:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.