There have been a number of papers over the past 40 years supporting either change score or regressor analysis for longitudinal observational studies.

However, recent papers (for example, Tennant et al. (2022) and Shahar & Shahar (2010)) argue change scores are not meaningful quantities and not suitable for causal inference of observational data.

Without getting into the technical details of these articles, it's not difficult to create simple case-control simulations where regressing a $$Y_1-Y_0$$ change score on a binary treatment/control group variable $$X$$ produces (seemingly) misleading results compared with a standard regression of $$Y_1$$ on $$Y_0$$ and $$X$$.

In both examples below, what appears to be a significant treatment effect in the first data set, and a non-significant treatment effect in the second data set are completely reversed in the change score analysis.

## Significant treatment effect (X=1) for regressor method, not significant for change score method
set.seed(123)
df_trmt <- data.frame(x=c(rep(1,100)), y1 = c(rnorm(100, 20, 8)))
df_trmt$$y2 = c(rnorm(100, 15 + .25*df_trmt$$y1, 2))
df_ctrl <- data.frame(x=c(rep(0,100)), y1 = c(rnorm(100, 40, 8)))
df_ctrl$$y2 = c(rnorm(100, 30 + .25*df_ctrl$$y1, 2))
df <- rbind(df_trmt, df_ctrl)
df$$z = as.factor(df$$x)

plot(df$$y1, df$$y2, col=c("black","gray50")[df$z], xlim=c(0,100), ylim=c(0,100)) a = seq(0, 100, 1) b = seq(0, 100, 1) lines(a, b, col="blue") df$$diff = df$$y2 - df$$y1 boxplot(df$$diff ~ df$x)

# Regressor method
summary(lm(df$$y2 ~ df$$y1 + df$x)) # Change score method summary(lm(df$$diff ~ df$$x)) ## No significant treatment effect (X=1) for regressor method, significant for change score method set.seed(234) df_trmt <- data.frame(x=c(rep(1,100)), y1 = c(rnorm(100, 20, 6))) df_trmt$$y2 = c(rnorm(100, 30 + .25*df_trmt$$y1, 2)) df_ctrl <- data.frame(x=c(rep(0,100)), y1 = c(rnorm(100, 60, 6))) df_ctrl$$y2 = c(rnorm(100, 30 + .25*df_ctrl$$y1, 2)) df <- rbind(df_trmt, df_ctrl) df$$z = as.factor(df$$x) plot(df$$y1, df$$y2, col=c("black","gray50")[df$z], xlim=c(0,100), ylim=c(0,100))
a = seq(0, 100, 1)
b = seq(0, 100, 1)
lines(a, b, col="blue")

# Regressor method
summary(lm(df$$y2 ~ df$$y1 + df\$x))

# Change score method
summary(lm(df$$diff ~ df$$x))


Can we finally resolve the debate in favor of the regressor method?

• The simplest heuristic argument in favor of the ANCOVA approach is that it is a generalization of a change score analysis. That is IF the change score is right, you simply estime the baseline (y0) coefficient as 1. For they and other reasons I'm in the "it's dead" camp. Jan 7 at 5:24
• Case-control designs are weak sauce for causal inference, so not sure where your argument is coming from there. Also: without measuring some kind of change score (DID, RCT, cohort design, etc.) it is not clear how you can measure change which is necessary for causal inference. Jan 7 at 5:24
• Difference in difference, interestingly, is a whole other topic. You can conduct a difference in difference analysis with the untransformed, raw response as the outcome. The language around what, exactly, DiD is is weak, but it is most easily understood as an interaction. Jan 7 at 5:25
• @Alexis As I understand it, the position in the Tennant paper is that a change score is not a random variable. Instead it's a variable derived from Y0 and Y1 & therefore contains no new information. In causal diagrams for DiD models does it make sense to include Y0, Y1, and Y1-Y0? I'm leaning in the No direction. There's also the issue of change scores producing contradictory results (see my examples). Jan 7 at 15:15
• You might be interested in my question and answer: How to represent difference variables in DAGs. Jan 7 at 18:59