My dataset (long format) contains of data collected in 4 studies. Although the variables in the studies were identical, I want to account for the substantial heterogeneity of the populations between these studies and thus treat "study" as a random effect. In each study, participants were randomly presented with 25 binary choices (0/1). I want to test whether attitudes and trait self-control predict making choice 1.

Thus, my logistic regression model should account for the fact that each subject made 25 decisions and that the subjects were nested within one study.

To test this, I used R:

model <-glmer(
  depvar ~ attitude + selfcontrol + (1 | study/subject),
  data = df,
  family = binomial("logit")

and I tried the same with stata. After having defined the panel with xtset, I tried:

xtmelogit depvar attitude selfcontrol || study:

The coefficients are extremely different (and thus also the plots differ)! Is one of the models wrongly defined?
Even if I try to change things like laplace approximation, the results just does not even get close to similar.


1 Answer 1


Those are different models. In the case of lmer, you are estimating a 3-level model of decisions nested within subject within study. By the way this is different than the syntax you described in your other post. In that post, you treated study as a non-nested random intercept. I'm not sure why you have changed it, but that is your choice.

In comparison, the model you estimated with xtmelogit in Stata is a 2-level model that depends on which variables you mentioned when you xtset the data. If you wanted to run a comparable model with Stata, you should use melogit, specifying the random effects similarly as you did in lmer. For example, for a three-level model, the random effects structure in meologit would be || study: || subject:.

  • $\begingroup$ Thank you very much again for your answer! I changed it because I thought that (1 | study/subject) would be more suitable for my data, as I try to tell the model that subjects were nested within study - but I may be totally wrong. How would you do it? Reg. Stata: I did xtset subject decisionnumber. Which model would you chose given the data structure I described? Wouldn't a 3-level model be correct (~25 decisions nested within one person, who themselves are nested within one study)? Your help means a lot to me, thank you very much!! :-) $\endgroup$
    – annwy
    Feb 28, 2020 at 13:57
  • $\begingroup$ I would probably treat study as a fixed effect predictor rather than a random effect. You only have 3 studies, right? In Stata, that would mean adding it as i.study to your list of predictors. If you do that, then you can use xtlogit given that you only have two levels - observations within subject and you used xtset to define the cluster variable (subject). $\endgroup$
    – Erik Ruzek
    Feb 28, 2020 at 14:24
  • $\begingroup$ I have 4 studies (Mturk with n=200, Mturk with n=600, Prolific with n=600 and student sample with n = 70) and the means/variance of self-control and attitudes as well as their effects on the behavior varies a lot between them. Now I am confused in what whay the stata commands are different from each other: xtmelogit depvar attitude selfcontrol || study: (with subject defined as cluster var) and melogit depvar attitude selfcontrol || study: || subject in R this would be the exact same code, no? Thank you, I promise this will be my last question :-) $\endgroup$
    – annwy
    Feb 28, 2020 at 15:07
  • 1
    $\begingroup$ What version of Stata are you using? If a newer 14+, you should be using melogit, not xtmelogit. Either way, you need to define the random effects structure irrespective of what you did with xtset. So I would propose the following model: melogit DV IV1 IV2 i.study || subject: In glmer: glmer(DV ~ IV1 + IV2 + as.factor(study) + (1|subject). $\endgroup$
    – Erik Ruzek
    Feb 28, 2020 at 15:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.