I am trying to fit a multilevel model with complex survey design data in Stata. However, my model levels do not correspond to my survey design stages.

My survey design is based on sal primary sampling units, with participant sample weights and finite population correction based on sal_tot. Strata of this complex survey are according to region.

Stata specifies this on their website: "What if we want to fit a multilevel model to data collected using a complex survey design rather than a simple random sample? We need to take into account characteristics of the survey design—clustering, stratification, sampling weights, and finite-population corrections—to obtain appropriate point estimates and standard errors. Adjusting for survey design in multilevel models is unique in that we need weights for each level of the model, assuming those levels correspond to stages of the sampling design."

**How can I use a multilevel model to explore nested variables at the individual-, interpersonal-, and community-level, when this hierarchy is not about my survey design based on region - sal - participant stages? Is this possible?


1 Answer 1


Executive summary: this is much harder than you would expect, and there is neither a standard implementation, nor even an accepted estimator.

Let's fix some terminology. In a survey, you have primary sampling units, secondary sampling units, and so on for potentially multiple stages. In a mixed model you have top-level clusters, possibly clusters at intermediate levels, and first-level individual observations.

Stata's -mixed-, and the gllamm package before it, have a method for the (fairly common) setting where the clusters and the sampling units are the same (or at least nested).

You're talking about a setting where they aren't. When the first complex-survey multilevel models were described in 1998, Jon Rao pointed this issue out in the discussion

The authors assume that the sample is selected according to the hierarchical structure of the model, but in multipurpose surveys the hierarchical structure of the sample could be quite different. For example, an educational multilevel model may include, as levels, students, classes, schools and school boards. However, the sample of students may be chosen by a multistage design where the stages are geographical areas, then households and then students; for example the Canadian National Longitudinal Survey of Children and Youth follows such a design. Since the design clusters cut across the model clusters for such surveys, the theory proposed may not be applicable. We are currently studying this problem.

He didn't get very far; it's a hard problem.

One approach to this problem is to use pairwise likelihood. I'm working on software for this, but it's not ready yet; Xudong Huang worked on the theory in his PhD thesis, based on earlier work by Jon Rao and Grace Li and co-workers.

On top of the lack of user-friendly software, a straightforward implementation of pairwise likelihood gives quite inefficient estimators for the variance components -- and if you aren't interested in the variance components, you might as well be doing ordinary regression. On top of that, there is currently no approach to computing BLUPs of the random effects, which are needed for adaptive Gaussian quadrature to fit generalised linear mixed models.

Overall, I have some hopes for an implementation of this being available in R by the end of 2020, but it isn't yet.

  • 1
    $\begingroup$ Any updates? Currently grappling with this now. My intuition is to at least explore the distribution of weights within my analysis clusters (which are NOT design clusters) and proceed on examining whether the survey design is ignorable $\endgroup$
    – Rick Hass
    Commented Sep 17, 2022 at 18:26
  • 1
    $\begingroup$ No updates, sorry. Looking to see if the design might be ignorable is probably a good start. $\endgroup$ Commented Oct 1, 2022 at 23:55
  • $\begingroup$ Ok, update: the svylme package for R is looking reasonably ok now, according to simulations: github.com/tslumley/svylme $\endgroup$ Commented Jun 29, 2023 at 7:12
  • $\begingroup$ Excellent!! Looking forward to trying it $\endgroup$
    – Rick Hass
    Commented Jun 30, 2023 at 12:42
  • 1
    $\begingroup$ And there's now a preprint: link here notstatschat.rbind.io/2023/11/24/svy2lme-the-preprint $\endgroup$ Commented Dec 19, 2023 at 22:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.