4
$\begingroup$

Imagine a dataset where there are 3 nested grouping variables: study $>$ group $>$ outcome (read outcome is nested in group which in turn is nested in study.)

We have two predictors. ktype is a categorical factor (0=direct,1=indirect) that can vary between studys, between groups, and between outcomes. And, treat is a continuous variable only varying between studys.

Question: If I fit 3 models (see below) that only differ in their random-effects specification, then, how does the interpretation or meaning of any of the 3 fixed-effect coefficients (A, B, C) for each model change?

         Estimate
ktype0     A
ktype1     B
treat      C
# Syntax in R's `lme4` package (DON'T RUN)
1) lmer(y ~ 0 + ktype + treat + (ktype |study))

2) lmer(y ~ 0 + ktype + treat + (ktype |study) + (treat |study))

3) lmer(y ~ 0 + ktype + treat + (ktype |study/group/outcome) + (treat |study))

# STRUCTURE OF GROUPING VARIABLES:
"
study  group outcome
1       1    1
1       1    2
1       1    1
1       1    2
1       1    1
1       1    2
1       2    1
1       2    2
1       2    1
1       2    2
1       2    1
1       2    2
2       1    1
2       1    2
2       2    1
2       2    2"
```
$\endgroup$

1 Answer 1

3
$\begingroup$

In my experience, fixed effects are usually well-estimated when the number of clusters is large enough so that the assumption of multivariate normal random effects can be justified, even when the random structure is mis-specified.

When the number of clusters is small, then parameter estimates can be biased. The extent of the bias is hard to know in advance, since it depends very much on the structure of the data and th underlying data generation process. Simulation studies are the best way to approach this, using specific data structures and a range of values for various options:

  • number of levels of each grouping factor
  • correlations among covariates
  • types of covariates
  • values for the fixed effects in the model
  • variance of random intercepts for each grouping variable
  • variance of random slopes where applicable
  • correlations between random intercepts and slopes

The idea would be to simulate data for such a model, and then run a Monte-Carlo simulation to assess whether the fitted values are biased. I have posted code on here which could be easily adapted for this. eg:
Why is this linear mixed model singular?
If I consider the fixed factor as a random slope, the p-value changes from p<0,05 to p>0,05

$\endgroup$
16
  • $\begingroup$ Dear Robert, thank you so much. I believe my question might not have been clear enough. My real intent was to see if the actual interpretation/meaning of fixed effects coefficients (A, B, C) might differ across the 3 models I showed in my question. For example, what is the difference in the interpretation/meaning of treat in model 1 vs. model 2 vs. model 3. And what is the difference in the interpretation/meaning of ktype0 in model 1 vs. model 2 vs. model 3. My own understanding is that the interpretation/meaning of treat in . . . $\endgroup$
    – rnorouzian
    Aug 12, 2021 at 15:05
  • $\begingroup$ . . .model 1 is: change in y for each unit of increase in treat when ktype == 0. However, I wonder how this interpretation (or meaning of treat as a fixed-effect coefficient) changes when treat is itself taken as random-effect as in model 2 or model 3? The same question applies to ktype. Given that in model 1, ktype is only taken as random at the study level, and then in model 3 taken to be random at the study, group, and outcome levels, I wonder how this interpretation (or meaning of ktype0 as a fixed-effect coefficient) changes when ktype0 used in . . . $\endgroup$
    – rnorouzian
    Aug 12, 2021 at 15:06
  • $\begingroup$ . . . model 1 vs. in model 3? $\endgroup$
    – rnorouzian
    Aug 12, 2021 at 15:06
  • $\begingroup$ As I explained in my answer, it all depends on the structure and details of your actual data as to whether the parameters will be biased. I don't know what you mean by "My real intent was to see if the actual interpretation/meaning of fixed effects coefficients (A, B, C) might differ across the 3 models I showed in my question". The interpretation of a fixed effect in a linear mixed model is exactly the same as a non-mixed model - the estimated change in the response for a unit change in the predictor. This has nothing to do with the random structure (correctly specified or not) $\endgroup$ Aug 12, 2021 at 15:09
  • $\begingroup$ But Roberts, when we allow say ktype0 and ktype1 in model 3 to vary across, group, outcome, and study levels, won't the fixed effect of these two coefficients be each a coefficient that has been averaged across these three levels? By contrast, wouldn't ktype0 and ktype1 in model 1 that are only allowed to vary across study each represent a coefficient that has been averaged across study levels? $\endgroup$
    – rnorouzian
    Aug 12, 2021 at 15:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.