I'm interested in assessing the performance of a multi-level model (aka hierarchical linear model, aka linear mixed effects model) when examining time-lagged associations. My interest is in making a more robust causal inference about the effect of a lagged IV on a DV by adjusting for the previous level of the DV in some way.
(related CV questions: Confusion over Lagged Dependent and HAC Standard Errors, Lagged dependent variable or handling residual as AR process?, Residual autocorrelation versus lagged dependent variable)
I vary models in two ways:
- Lagged dependent variable versus autoregressive residual error structure
- Centered lagged independent variable versus within- (and between-)person centered lagged variables.
I generate data from a simple cross-lagged path model where the autoregressive path is set to .7, the within-wave correlation is set to .5, and the cross-lagged regression paths are set to 0.
Bias is simply the coefficient estimate for the effect of the lagged independent variable, and for error control I'm looking at the significant tests of that coefficient.
In my test models, $y_t$ is the dependent variable at time $t$, $x_t$ is the independent variable, and the lagged variables are always at $t-1$. Unless specified, I use the default covariance structure for the residuals ($\epsilon$), and random intercept ($\nu_0$).
The models are all fit with lme
in the nlme
(version 3.1) package in R (3.5.1), except for a couple where I try to fix the poor error control with Satterthwaite's method for determining correct degrees of freedom.
Lagged DV
(I do this even though I guess I shouldn't according to this blog post: https://statisticalhorizons.com/lagged-dependent-variables.)
When I simulate data and evaluate the model $y_{it} = \beta_{0}y_{i,t-1} + \beta_{1}x_{i,t-1} + \epsilon_{it} + \nu_{0i}$, I see no bias, and error control at the nominal 5%.
AR(1) structure
If I use an AR(1) structure for the residuals, I see positive bias (and as a result, poor error control).
Lagged DV, within-person centering
At this point, using the lagged DV seems to be the best way to proceed. So I decompose the predictor variables into within and between-person variables. I use the within-person means for each person as a time invariant between-person predictor (e.g., $x^\text{bw}_{t-1}$), and the within-person deviations (e.g., $x^\text{wi}_{t-1}$) from those means as the time-varying, within-person predictors. I do the same thing with the lagged dependent variable, but leave out the within-person means (which are redundant with the random intercept): $y_{it} = \beta_{0}y^\text{wi}_{i,t-1} + \beta_{1}x^\text{bw}_{i,t-1} + \beta_{2}x^\text{wi}_{i,t-1} + \epsilon_{it} + \nu_{0i}$.
This shows no bias, but bad error control.
What's going on?
I imagine that I've simply taken a wrong turn somewhere, but I'm not sure where. It sounds like it's a bad idea to use lagged DVs in your model, but for my toy example this doesn't seem like a problem (until I use within-person centering). The AR(1) fails in both the within-person decomposition and non-decomposed cases (more detail here: https://jflournoy.github.io/lagged_mlm/ar_stuff.html).
There are possibly two questions:
- Why doesn't the AR(1) residual structure fully adjust for the stability of the DV? (this may be answered in part already by the related questions I note above, though if so, I've not made the connection).
- Why are the significance tests of the effect of the within-person-centered IV $x^\text{wi}_{t-1}$ anti-conservative?