What is the difference between autocorrelated residuals and controlling for the previous time point in mixed effects models?

Question

I have several dozen observations from about 100 people who participated in an ecological momentary assessment study. I am using mixed effects models to estimate the effect of $X_{t-1}$ on $Y$.

Approach 1 is to include $Y_{t-1}$ as a covariate, e.g.:

y ~ y(t-1) + x + x(t-1)

Approach 2 is not to include $Y_{t-1}$ as a covatiate, but to include an autoregressive error structure in the model, e.g.:

y ~ x + x(t-1) + corAR1(form = ~ 1 | id)

---

Question

What is the difference between these two models? I am specifically curious about the difference in interpretation of the effect of x(t-1).

Answer 1

Lesa Hoffman (2015) in her Longitudinal Analysis book has a very useful way to think about regression models that can help you understand the difference in the two approaches. On the one hand, you have the structural or fixed effects part of the model, which she calls the model for the means. This "states how the expected (or predicted) outcome for each person varies as a function of their predictor values" (page 9). Relatedly, Andrew Gelman urges researchers to interpret structural regression coefficients in terms of differences in means (i.e., the predicted mean outcome difference for individuals who differ on x by 1 unit, adjusting for other variables).

On the other hand, you have the stochastic or error part of the model, which Hoffman calls the model for the variance. This part of the model "describes how the residuals of the Y outcome (i.e., the differences between the Y values observed in the data and the Y values predicted by the model for the means) are distributed and related across observations" (page 9 in Hoffman, 2015). In the model for the variance you are predicting the pattern of variance and covariance in the residuals of the Y outcomes. This matters in part because the standard errors of the tests for the fixed effect coefficients rely on you getting the model for the variances right.

Approach 1 is using the model for the means to deal with the "problem" of the prior value of y. In this approach, your predictor of interest x(t-1) is interpreted as the mean difference in y between individuals who differ by 1-unit on the prior observation of x, adjusting for the prior outcome value y(t-1) and the present value of x. You might use this if you believe that y(t-1) is a common cause (or confounder) of the effect of x(t-1) on y.

Approach 2 is more consistent with multilevel or mixed effects modeling, which allows for a more complex model for the variance. These models decompose variance in the outcome into different sources. In your model, outcome variance in y is due to three sources$^1$:

1. Persistent person variance (the random intercept: $\tau^2_{U_0}$)
2. Residual within-person variance ($\sigma^2_e$)
3. Residual covariance, or autocorrelation (r$_T$)

The coefficient of x(t-1) is interpreted as the mean difference in y between individuals who differ by 1-unit on the prior observation of x, adjusting for the present value of x and accounting for the pattern of variances and covariances in the residuals. Here you assume that the covariance pattern is autoregressive, but you could consider and evaluate model fit for other types of covariance structures, including compound symmmetry, Topelitz, and heterogenous versions of autoregressive and Topelitz.

Is one of these better than the other? It depends on what you are trying to accomplish. You are not modeling the effect of time on the outcome explicitly in either of these models other than to say that there is some kind of association between prior and present value of y. That suggests that you are more interested in fluctuation in y rather than systematic differences in y that are due to some kind of developmental process. An example of the latter would be a growth curve model where you allow for linear or non-linear effects of time in the model for the mean and the model for the variance (i.e., fixed and random slopes for time). Approach 2 is more consistent with an interest in modeling fluctuation of the outcome, however Approach 1 might be important if you have some sort of causal theory about how y arises.

This is a complicated topic, and there has been a lot of interesting work on modeling residual variance in the structural equation (SEM) literature recently.

$^1$Note that Approach 1 decomposes outcome variance only into 1 and 2 of the list.

Answer 2

Both models are problematic and are not really used in my field (biostatistics with a focus on clinical trials) for the reasons mentioned below:

• Both models assume that the residual variance is the same across time, which is rarely the case (e.g. in a clinical trial where you recruit people according to some criteria that tend to constrain the distribution a little, have a baseline pre-treatment assessment and then look at values over time, residual SD or variance almost invariably goes up over time).

• Using previous observations as a covariate

  • It ignores that things are usually measured with error.
  • It (indirectly) induces a relatively rigid inflexible correlation structure that will usually not correctly capture the real correlation structures you would encounter in the wild (which is e.g. why multiple imputation with chained equations aka "MICE" often performs badly).
  • I think (but am not totally sure) that it might cause problems with causal inference across the timepoints (i.e. if you wanted to estimate a causal effect of an intervention across two timepoints, but you condition on the second timepoint on the observed value at the first, I suspect this is problematic).

• Using AR(1) correlated residuals

  • AR(1) is a terrible correlation structure for any real data I've ever worked with (usually correlation declines over time, but not as fast towards zero as AR(1) assumes).
  • While it may not be so bad that it overestimates correlation for times that are close together, the biggest problem comes from how it treats observations that are far enough apart as effectively independent. This latter point will typically invalidate lots of things (e.g. type I error for hypothesis tests, coverage for confidence intervals etc.).

What are alternatives?

• If you have fixed timepoints, something like mixed models for repeated measures (see here for a Bayesian version) with an unstructured (or at least more flexible covariance structure - you might be willing to make more assumptions than that, which would result in a gain in precision on your inference).
• If you don't have fixed timepoints:
  • There's Gaussian process models to capture the correlation, but these have a reputation to be hard to fit.
  • A more mechanistic/generative model that describes underlying process through e.g. differential equations (while your measurements are only realizations from this possibly unobserved underlying process). This tends to induce reasonably complex correlation structures of a sufficiently capacity to fit a lot of data, but you may not know enough to specify/setup such a model.
• If we are not really talking about an inference task, but rather about prediction:
  • Options could include various time series types of neural networks (e.g. LSTMs).
  • Anything that reflects what you would really know when you'd make a prediction. So, models with (possibly multiple) lags can often do okay for pure prediction tasks, although you'd often want to be more flexible than a linear model (e.g. gradient boosted decision trees with multiple lags as inputs).