6
$\begingroup$

I'm new to mixed effects models and am trying to use the lmer() function from the lme4 R package to specify a random effects structure.

In my experiment, subjects are spread out over 11 non-overlapping groups. Groups were fairly big (hundreds of subjects) and were tested on at least 4 successive days. On each day, subject performance was measured under two experimental conditions (hungry, satiated) and during each of these conditions, each subject contributed zero to potentially many data points. Groups were tested sequentially, i.e., on each day only one group was tested.

After reading through lectures, tutorials, and posts on here, I think my model should look like this

response_time ~ experimental_condition + (1 | group_id/day) + (1 | subject_id)

and the subject_id needs to be unique across the entire data set.

Does this look okay? And under which (hypothetical) circumstances would one nest experimental_condition within day?

Thanks for any help!

Edit: I should have mentioned that day is currently coded as the day of the experiment. In other words, it is not unique across groups (i.e., there is a day 1, day 2, ... for each group).

$\endgroup$
12
  • $\begingroup$ What is your research question ? Presumably your primary interest is in estimating the effect of experimental_condition, while accounting for the non-independence of observations due to the random structure your experiment has. Please confirm or provide further detail. $\endgroup$ Nov 13, 2023 at 11:53
  • $\begingroup$ So each subject has been measured first while hungry, and then while satiated ? Do the groups represent treatment levels, or they just happened because of the experimental setup ? $\endgroup$
    – CaroZ
    Nov 13, 2023 at 12:30
  • $\begingroup$ @Robert Long: That's correct, I'm interested in the effect of experimental condition on response_time. Sorry for the omission. $\endgroup$
    – Tim
    Nov 13, 2023 at 13:28
  • $\begingroup$ @CaroZ: Yes, the subjects were first measured when hungry and then measured again when satiated. And the groups represent honey bee colonies, not treatment levels. $\endgroup$
    – Tim
    Nov 13, 2023 at 13:47
  • $\begingroup$ For the nested random effects, you have specified (1 | group_id/day) which implies that day is nested within group_id. This means that each day is unique within a group but not across groups so for example, day 1 for group 1 is different from day 1 for group 2). Is that the case ? $\endgroup$ Nov 13, 2023 at 15:24

3 Answers 3

5
$\begingroup$

Does this look okay?

Based on your description, and given your research question of estimating the effect of experimental_condition, while accounting for the non-independence of observations due to the random structure your experiment has, this does not look OK to me. The issue is with the random structure, and how to handle the day variable.

It appears that each and every subject belongs to one and only one group. Thus, subjects are nested within groups, so you need the term:

... + (1 | group_id / subject_id) + ...

which will fit random intercepts for each group and each subject within a group.

This leaves the question of how to treat the day variable: fixed or random. There isn't necessarily a black and white answer to this, but see the list of threads at the end of my answer for help on how to choose. The first thing to note is that day has only 4 levels. This isn't necessarily a problem if day is nested within group_id, since there will then be $n_{day} \times n_{group} = 44$ intercepts.

So, if treating day as random and nested within group we would have:

response_time ~ experimental_condition + (1|group_id/subject_id) + (1|group_id/day)

which expands to

response_time ~ experimental_condition + (1|group_id) + (1|group_id:subject_id) + (1|group_id)+ (1|group_id:day)

which then simplifies to:

response_time ~ experimental_condition + (1|group_id) + (1|group_id:subject_id) + (1|group_id:day)

Alternatively if day is not nested within group we wouldn't fit random intercepts with only 4 levels, so treating day as fixed would make more sense in that scenario:

response_time ~ experimental_condition + day + (1|group_id/subject_id)

In the this latter model you should consider whether to fit an interaction term in the fixed part if the effect of the experimental condition differs by day:

response_time ~ experimental_condition * day + (1|group_id/subject_id)

And under which (hypothetical) circumstances would one nest experimental_condition within day?

Nesting experimental_condition within day makes sense if each experimental_condition belongs to one and only one day. That does not seem to be the case with your design. This would also bring up the problem of whether to fit a factor as random or variable. See the following threads for much discussion on that topic:

What is the difference between fixed effect, random effect and mixed effect models?

How to determine random effects in mixed model

Understanding Random Effects in Linear Mixed Models

Can a variable be included in a mixed model as a fixed effect and as a random effect at the same time?

Choosing Random Effects to Include in a Linear Mixed Model

$\endgroup$
11
  • $\begingroup$ I completely disagree. If subject_id is already unique and nested in group_id then (1|group_id / subject_id) = (1|group_id) + (1|subject_id), so your first point is wrong. Source(last bullet point): bbolker.github.io/mixedmodels-misc/… $\endgroup$ Nov 13, 2023 at 21:05
  • $\begingroup$ inversely, since day is probably not uniquely specified and it makes perfect sense that different beehives are in different places and get different weather, etc... on the same day, it should be nested in group $\endgroup$ Nov 13, 2023 at 21:10
  • $\begingroup$ Well the question says "and the subject_id needs to be unique across the entire data set", so... Also is that link right? It just points to here $\endgroup$ Nov 13, 2023 at 21:35
  • $\begingroup$ Oh, you deleted your comment. I do want to walk back the "completely": I agree with you about nesting experimental_condition and if they only measured 1 group a day, day is also automatically nested. $\endgroup$ Nov 13, 2023 at 22:46
  • $\begingroup$ @Lukas Lohse : Were you referring to my previous comment ? I put it back under my answer. $\endgroup$
    – CaroZ
    Nov 13, 2023 at 22:57
2
$\begingroup$

I think your model is fine. If subject_id is unique between groups, then you don't have to specify as nested in the model formula. See last bullet point here: https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#nested-or-crossed

Also a quick demonstration in R:

library(lme4)
n <- 1000
set.seed("0815")
subject_id <- sample(1:100, size = n, replace = T)
subject_random_effect <- rnorm(100)
# randomly generate groups for the subjects
group_id <- sample(1:10, size = length(unique(subject_id)), replace = T)
group_random_effect <- rnorm(10)
# simluate response, by exploiting how R does indexing
y <- subject_random_effect[subject_id] + group_random_effect[group_id[subject_id]] + rnorm(n, sd = 0.5)

df <- data.frame(y,subject_id,  group_id = group_id[subject_id])
# perfectly identical models
summary(lmer(y ~ (1|group_id/subject_id), data = df))
summary(lmer(y ~ (1|group_id) + (1|subject_id), data = df))

I also agree with you that it makes perfect sense that different beehives are in different places and get different weather, etc... on the same day, day should be nested in group, although with only one group getting measured per day it doesn't actually matter. Optimally you'd have weather data and include adjustments based on strong domain knowledge, but that is likely unrealistic.

Spatial correlation, i.e. groups that are closer together are more similar/experience more similar weather is something you might consider, but the technical challenge is high: https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#spatial-and-temporal-correlation-models-heteroscedasticity-r-side-models

With 11 groups you could also consider treating group as a fixed effect.

And under which (hypothetical) circumstances would one nest experimental_condition within day?

Nesting it would be weird, but you might want to look at effect variation, i.e. whether the difference in response time between hungry and satiated bees changes.

A formula that both teats group as fixed effect and looks at effect variation for experimental condition would be:

response_time ~ experimental_condition + group_id + (experimental_condition|group_id:day) + (1|subject_id)
$\endgroup$
5
  • $\begingroup$ I could obtain weather data and have considered adding adjustments to the model based on it it. However, from the feeding apparatus, I also know how much food the bees consumed (e.g. 152 ml in group 2 on day 11), which is probably a more relevant adjustment than the weather, as it pertains to the satiated experimental_condition more directly. What do you think? $\endgroup$
    – Tim
    Nov 14, 2023 at 9:03
  • $\begingroup$ I'll probably forgo making adjustments based on spatial correlations as the groups where tested in roughly the same place and the technical challenges seem high. $\endgroup$
    – Tim
    Nov 14, 2023 at 9:05
  • $\begingroup$ I sounds like you and Robert Long agree that nesting experimental_condition within day doesn't make much sense with respect to the research question. But thanks for explaining what that would do and for showing the corresponding model. That was educational. $\endgroup$
    – Tim
    Nov 14, 2023 at 9:17
  • $\begingroup$ your 1st comment: Effects on reaction time are more relevant and only variables that have an effect on both or the difference between the groups need to be involved. There exists extensive theory about this: en.wikipedia.org/wiki/Bayesian_network $\endgroup$ Nov 14, 2023 at 15:30
  • $\begingroup$ 2nd comment, sure, 3rd comment: What i defined is more of a random slopes model, which in this case is only subtlety different, but nevertheless i wouldn't mind it. $\endgroup$ Nov 14, 2023 at 15:32
0
$\begingroup$

Since individuals belong to colonies, I would nest the subject ID in the group :

response_time ~ experimental_condition + (1|group_id/subject_id)

Since on each group has its own unique days where it was tested, I would not further include "day" as a random factor.

$\endgroup$
4
  • $\begingroup$ I would ave thought that you need the day variable in the model somewhere ? $\endgroup$ Nov 13, 2023 at 16:12
  • $\begingroup$ @Lukas Lohse: were you referring to my previous comment here ? It was : "Since each day contains measurements of only 1 group, then group and day are completely colinear. I thought the variance caused by the day cannot be discriminated from the variance caused by the group." Actually I am wondering if +(group_id/subject_id)+(1|group_id/day) would be a good thing. $\endgroup$
    – CaroZ
    Nov 13, 2023 at 22:57
  • $\begingroup$ @CaroZ: I thought day needs to be in the model because I measure each group on more than one day, and there is uncontrolled day to day variability in subject performance. $\endgroup$
    – Tim
    Nov 14, 2023 at 8:25
  • $\begingroup$ Yes group is repeated across day, which is why I was proposing +(group_id/subject_id)+(1|group_id/day) , but I wonder if it is correct, and then if it is an overkill, because the variance caused by the days where you sampled is contained in the variance caused by the group. but I suppose you can make the same argument for (group_id/subject_id). $\endgroup$
    – CaroZ
    Nov 14, 2023 at 11:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.