I've read a lot of threads on stack exchange but haven't exactly found what I'm looking for. Everyone seems to have a slightly different problem/issue.

First, lets have a look at my data:

  • 120 Users
  • 80 Items per User
  • Between factor with 4 levels
  • Within factor with 4 levels
  • Binary response variable

Now, usually one would perform a logistic regression. However, there are several issues with that:

  • Categorical predictors have to be dummy coded?
  • How do deal with the within-subject fator? -> lmm?
  • How would you plot this data?

As I think (but I'm not sure? correct me if I'm wrong) that one assumption of binary logistic regression is the independence of errors which is - again not sure - violated with within subject factor I'm trying to perform a linear-mixed effects model for my data. Now here begins the actual problem:

First: I know, that I want to model the between-subjects and within-subject factors as fixed effects. However: which random effects should I implement?

  • Random intercept of each subject as it can be assumed that they differ in they're apriori knowledge of the items (it's a performance test).
  • Random intercept of the within-subject factor - as this is the reason for performing lmm in the first place?
    • My data is actually not nested, so there is no sense in creating a random effect like all the examples "school", "county" and so on...
    • other suggestions?

Okay my suggestion is to assume random intercepts of subjects (the first one):

lmm3 <- glmer(y ~ between * within + (1|user), data, family = binomial(link = "logit"))

But, first I would have to calculate the ICC 1 and ICC2 to support the use uf lmm.

for the ICC 1 I use the nullmodel:

lmm0 <- glmer(y ~ (1|user), data, family = binomial(link = "logit"))
icc1 <- tau2/(tau2+pi^2/3)

No, again two questions arise:

  • How do I calculate ICC2 for the logistic model? I know that there is a function for linear mixed models, but this is not the case here.
  • However, my ICC1 is only 0.03760069 so it seems that this above model doesn't make a lot of sense. What kind of model should I try then?

I thank you a lot for your inputs. You need more specific information I would be willing to prepare some data for you. I know that this is a rather theoretical issue so I'm looking forward to a discussion.

Kind regards, David

  • $\begingroup$ The reason why you want to use random effects is to account for correlations, i.e., responses on the outcome y coming from the same user will be expected to be correlated. Now, why do you want to calculate ICCs? What is really your purpose? $\endgroup$ Oct 4, 2018 at 21:03
  • $\begingroup$ I want to calculate the ICC1 to check how much of the total variance of my model is explained by the variance of the caused by the users just because they differ in their intercepts. This would justify the use of LMMs $\endgroup$ Oct 5, 2018 at 8:47
  • $\begingroup$ There is a lot of controversy in using ICCs for mixed models, especially for categorical data; see, e.g., this post by Douglas Bates the main developer of lme4: stat.ethz.ch/pipermail/r-sig-mixed-models/2010q1/003363.html . If you want to see if you need to use the mixed model, you can compare with a likelihood ratio test (i.e., the anova() function) the logistic regression model without random effects, fitted by glm(..., family = binomial()) with the mixed effects logistic regression fitted by glmer(..., family = binomial()). $\endgroup$ Oct 5, 2018 at 8:53
  • $\begingroup$ Well, thank you Dimitris. This is quiet interesting as I wasn't aware that I'm allowed to compar a glm with a glmer. Seems to be a really straight forward solution for a topic that causes a lot of discussions^^ $\endgroup$ Oct 5, 2018 at 10:57

1 Answer 1


Since you have 120 users and 80 items per user, your model would have to treat user and item as random grouping factors.

Are the 80 items the same for each of your users? If yes, the random grouping factors user and item will be fully crossed, in which case the glmer syntax would include terms like

(1|user) + (1|item)

See https://nlp.stanford.edu/manning/courses/ling289/GLMM.pdf for more ideas on how to proceed.

  • 1
    $\begingroup$ Actually, the 80 items are basically the same for all of the users, but the users are supported by an aid to a different extent depending on which between-subject factor they belong. Does this mean I should only add the (1|user) as a random effect to the model? However, thanks for the paper, this seems to be exactly what I want to do. $\endgroup$ Oct 5, 2018 at 8:42
  • $\begingroup$ Can't you include (1|user) and (1|item), as well as a predictor variable which quantifies the extent of aid received by each user? (You need to capture the between-item variability somehow in your model.) Also, I'm assuming that you have multiple binary outcome values collected for each user by item combination. $\endgroup$ Oct 5, 2018 at 19:56
  • $\begingroup$ yes I can, but the random effect of (1|item) is does not enhnace the model. $\endgroup$ Oct 15, 2018 at 13:52
  • $\begingroup$ Sometimes we leave terms in the model because of substantive - not statistical - reasons. $\endgroup$ Oct 15, 2018 at 14:08
  • $\begingroup$ Hi Isabella, we had this discussion back in October and now another question arose. What if the items are not the same for the between subject variables? Should am I still allowed to include the (1|Item) random effect? I suppose not or am I wrong? $\endgroup$ Jan 7, 2019 at 12:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.