1
$\begingroup$

I am having trouble determining which method to use to analyze my data. Here is the info: -575 observations nested within 292 groups -some groups only have one observation, the max number is 23 in a group with a mean of 2.0

I am trying to control for the within group effects for my dependent variable, and am thinking about using multilevel modeling to partition this variance. However, I am concerned about having groups with only one or few observations. My dependent and all but one independent variable are all at the observation level. I only have one variable at the group level. Should I used a random effects or fixed effects model? Some insight would be great.

$\endgroup$
1
$\begingroup$

Welcome to the site, Brad. If I were in your situation, I would use a mixed (a.k.a. random) effects model to analyze your data. These models easily handle uneven group sizes and have no problem with singleton clusters (with just 1 member) provided that such clusters are (ideally) outnumbered by non-singleton clusters.

In a 2-level mixed effects model,

$$y_{ij} = \beta_0 + u_{0j} + \epsilon_{ij}$$

the outcome $y_{ij}$ is a linear function of the overall intercept $\beta_0$, between-cluster error $u_{0j}$ (the random intercept), and within-cluster error $\epsilon_{ij}$ (the residual). The model thus splits variation in $y_{ij}$ into two error components, each associated with their own variance terms $-$ $\theta$ for within-cluster variation and $\psi$ for between-cluster variation.

Because singleton clusters have 0 within-cluster variation they do not contribute to the model-estimated $\theta$ nor to the determination of how variance is partitioned across $\psi$ and $\theta$. However, they do contribute to the estimation of any slopes ($\beta$) and the estimation of $\psi$ + $\theta$, which is the total variance in the outcome.

Another reason to favor mixed effects models over "fixed" effects models comes into play when you want to predict each cluster's value on the outcome (call it $\tilde{y_j}$). In a fixed effect model a singleton cluster's $\tilde{y_j}$ is completely based on information from that single individual, which is why these models are sometimes referred to as "no-pooling" models. In contrast, the mixed effects model assigns a empirical prior distribution to the random intercepts $u_{0j}$:

$$u_{0j} \sim N(0, \psi)$$

The mean of this normal distribution is 0, which is understood in relation to the intercept ($\beta_0$) from your model with variance $\psi$ to be estimated by the model. Using empirical Bayes prediction, a singleton cluster's $\widetilde u^{EB}_{0j}$ is pulled back toward the overall intercept $\beta_0$. These models are sometimes called "partial pooling" models for these reasons. The degree of partial pooling is based not only on the within-cluster sample size, but other factors as well. See this answer for information on the empirical Bayes correction factor that helps determine how much pooling occurs.

$\endgroup$
1
  • $\begingroup$ Thanks for the info. That makes a lot of sense. I am using Stata to run my analyses and have set it up with the xtmixed function grouping by the group id. Would this be the correct method to run these analyses? Similarly, what types of diagnostics exist for these models? $\endgroup$ – Brad Aug 12 '20 at 18:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.