Is it valid to include a baseline measure as control variable when testing the effect of an independent variable on change scores?

Question

I am attempting to run an OLS regression:

• DV: Change in weight over a year (initial weight - end weight)

• IV: Whether or not you exercise.

However, it seems reasonable that heavier people will lose more weight per unit of exercise than thinner people. Thus, I wanted to include a control variable:

• CV: Initial starting weight.

However, now initial weight is used BOTH to calculate the dependent variable AND as a control variable.

Is this okay? Does this violate an assumption of OLS?

Answer 1

To answer your literal question, "Is it valid to include a baseline measure as control variable when testing the effect of an independent variable on change scores?", the answer is no. The answer is no, because by construction the baseline score is correlated with the error term when the change score is used as the dependent variable, hence the estimated effect of the baseline on the change score is uninterpretable.

Using

• $Y_1$ as the initial weight
• $Y_2$ as the end weight
• $\Delta{Y}$ as the change in weight (i.e. $\Delta{Y} = Y_2 - Y_1$)
• $T$ as a randomly assigned treatment, and
• $X$ as other exogenous factors that affect weight (e.g. other control variables that are related to the outcome but should be uncorrelated with treatment due to random assignment)

One then has a model regressing $\Delta{Y}$ on $T$ and $X$;

$$\Delta{Y} = \beta_1T + \beta_2X + e$$

Which by definition is equivalent to;

$$Y_2 - Y_1 = \beta_1T + \beta_2X + e$$

Now, if you include the baseline as a covariate, one should see a problem, in that you have the $Y_1$ term on both sides of the equation. This shows that $\beta_3Y_1$ is uninterpretable, because it is inherently correlated with the error term.

$$\begin{align*}Y_2 - Y_1 &= \beta_1T + \beta_2X + \beta_3Y_1 + e \\ Y_2 &= \beta_1T + \beta_2X + \beta_3Y_1 + (e + Y_1) \end{align*}$$

Now, part of the confusion in the various answers seems to stem from the fact that different models will yield identical results for the treatment effect, $\beta_1T$ in my above formulation. So, if one were to compare the treatment effect for the model using change scores as the dependent variable to the model using the "levels" (with each model including the baseline $Y_1$ as a covariate), ones interpretation of the treatment effect would be the same. In the two models that follow $\beta_1T$ will be the same, and so will the inferences based on them (Bruce Weaver has some SPSS code posted demonstrating the equivalence as well).

$$\begin{align*} Change\ Score\ Model&: Y_2 - Y_1 = \beta_1T + \beta_2X + \beta_3Y_1 + e \\ Levels\ Model&: Y_2 = \beta_1T + \beta_2X + \beta_3Y_1 + e \end{align*}$$

So some will argue (as Felix has in this thread, and as Bruce Weaver has done on some discussions over on the SPSS google group) that since the models result in the same estimated treatment effect, it does not matter which one you choose. I disagree, because the baseline covariate in the change score model can not be interpreted, you should never include the baseline as a covariate (regardless of whether the estimated treatment effect is the same or not). So this brings up another question, what is the point in using the change scores as dependent variables? As Felix already noted as well, the model using the change score as the dependent variable excluding the baseline as a covariate is different than the model using the levels. To clarify, the subsequent models will give different treatment effects (especially in the case that the treatment is correlated with baseline);

$$\begin{align*} Change\ Score\ Model\ Without\ Baseline&: Y_2 - Y_1 = \beta_1T + \beta_2X + e \\ Levels\ Model&: Y_2 = \beta_1T + \beta_2X + \beta_3Y_1 + e \end{align*}$$

This has been noted in prior literature as "Lord's Paradox". So which model is right? Well, in the case of randomized experiments, I would say the Levels model is preferable (although if you did a good job randomizing, the average treatment effect should be very close between the models). Other's have noted reasons why the levels model is preferable, Charlie's answer makes a good point in that you can estimate interaction effects with the baseline in the levels model (but you can't in the change score model). Whuber in this response to a very similar question demonstrates how the change scores induce correlations between different treatments.

In situations in which the treatment is not randomly assigned, the model using change scores as the dependent variable should be given more consideration. The main benefit of the change score model, is that any time invariant predictors of the outcome are controlled for. So say in the above formulation, $X$ is constant throughout time (for example say a genetic predisposition to be at a certain weight), and that $X$ is correlated with whether an individual chooses to exercise (and $X$ is unobserved). In that instance, the change score model is preferable. Also in instances in which selection into treatment is correlated with the baseline value, the change score model may be preferable. Paul Allison in his paper, Change Scores as Dependent Variables in Regression Analysis, gives these same examples (and largely influenced my perspective on the topic, so I highly suggest to read it).

This isn't to say that change scores are always preferable in non-randomized settings. In the case that you expect the baseline to have an actual causal effect on the post weight, you should use the levels model. In the case that you expect the baseline to have a causal effect, and the selection into treatment is correlated with the baseline, the treatment effect is confounded with the baseline effect.

I've ignored the note by Charlie that the logarithm of the weight could be used as the dependent variable. While I don't doubt that could be a possibility, it is somewhat non sequitur to the initial question. Another question has discussed when it is appropriate to use the logarithms of the variable (and those still apply in this case). There is probably prior literature on the subject that would help guide you as to whether using the logged weight is appropriate as well.

---

Citation

Allison, Paul D. 1990. Change scores as dependent variables in regression analysis. Sociological Methodology 20: 93-114. Public PDF version.

Answer 2

Andy's answer seems to be the economist's view of things. It is accepted practice in clinical trials to almost always adjust for the baseline version of the response variable, to greatly increase power. Since we condition on the baseline variables there is no 'error term' for them to be confused with the overall error term. The only problem would be if measurement errors in the baseline covariate are confounded with another X, distorting that other X's effect. The overall preferred method is to adjust for baseline and to model the response variable, not computing the change. One reason for this is that change is heavily dependent on getting the transformation of Y correct, and that change does not apply to regression models in general. E.g. if Y is ordinal, the difference between two ordinal variables is no longer ordinal. Concerning logging or not logging, that just depends on model and overall residual distribution assumptions.

Answer 3

We can alter @ocram's reasoning slightly to have $$\begin{align*} \text{E}[w_1 - w_0 \mid X, w_0] &= \beta_0 + x \beta + w_0 \gamma \\ \text{E}[w_1 \mid X, w_0] &= \beta_0 + x \beta + w_0 (\gamma + 1) \end{align*} $$

So, if this is the right model, saying that the difference depends upon the weight implies that the end value depends upon the initial value with a coefficient that could be anything. Running a regression of the difference on $x$ and $w_0$ or the end weight on the same variables should give you the same coefficients on everything but $w_0$. But, if this model isn't exactly correct, these regressions will give different results on the other coefficients as well.

Note that this set up implies that the starting weight predicts the difference in weights, not the impact of treatment. This would require an interaction term, perhaps $$\begin{align*} \text{E}[w_1 - w_0 \mid X, w_0] &= \beta_0 + (x * w_0) \beta + w_0 \gamma. \end{align*} $$

Another approach would be to calculate $$\begin{align*} \log (w_1) - \log (w_0) \approx r; \end{align*}$$ here, $r$ is the growth rate of weight. This could be your outcome. Your coefficients on $x$ would be telling you how these predictors are related to proportion changes in weight. This "controls for" initial weight by saying that, for example, an exercise regime that reduces weight by 10% (a coefficient of 0.1 multiplied by 100%) for someone that weights 130 pounds reduces weight by 13 pounds, while the program reduces the weight of a 200 pound participant by 20 pounds. In this case, you might not need to include the initial weight (or its log) on the right hand side.

An interaction term may still be necessary if you believe that the impact of the program depends upon the starting weight. If you use $w_0$ in the interaction term, then the program would be associated with a $w_0 \beta_1$ change in the growth rate of weight. Every pound heavier that a person was at the start of the program leads to a $\beta_1$ increase in the change in the growth rate (this is the cross-partial derivative of the expected value with respect to both treatment and starting weight).

If you use $\log (w_0)$ in the interaction term, the impact of the program increases by $\beta_1/w_0$ for each additional pound heavier the participant was at the start of the program.

As you can see, the cross-partials on interaction terms can become a bit tricky to interpret, but they may capture an impact that you are interested in.

Answer 4

EDIT: Andy W's argument convinced me to drop Model C. I added another possibility: Analyzing change with Random Coefficient Models (aka Multilevel Models or Mixed Effect Models

There has been a lot of scientific debate about the use of difference scores. My favorite texts are Rogosa (1982, [1]) and Fitzmaurice, Laird, & Ware (2004, [2])

In general, you have three possibilities of analyzing your data:

• A) Only take the interindividual difference score (the change score)
• B) Treat the post measurement as DV and control it for the baseline
• C) Take the difference score as DV and control it for the baseline (that's the model you suggested)._{Due to Andy W's arguments, I dropped this alternative}
• D) Using a multilevel / mixed-effect-model approach, where the regression line is modeled for each participant and participant are treated as Level-2 units.

Models A and B can produce very different results if the baseline is correlated with the change score (e.g., heavier people have more weight loss), and/or treatment assignment is correlated with the baseline.

If you want to know more about these issues, see the cited papers, or here and here.

There has also been a recent simulation study [3] which empirically compares the conditions under which A or B are preferable.

For completely balanced designs with no missing values, Model D should be equivalent to Model A. However, it gives you more information about between person variability, it is easily extended to more measurement points, and it has nice properties in the presence of unbalanced data and/ or missing values.

As a bottom line: In your case, I would analyze post-measures controlled for baseline (Model B).

[1] Rogosa, D., Brandt, D., & Zimowski, M. (1982). A growth curve approach to the measurement of change. Psychological Bulletin, 92, 726-748.

[2] Fitzmaurice, G. M., Laird, N. M., & Ware, J. H. (2004). Applied longitudinal analysis. Hoboken, NJ: Wiley.

[3] Petscher, Y., & Schatschneider, C., 2011. A Simulation Study on the Performance of the Simple Difference and Covariance‐Adjusted Scores in Randomized Experimental Designs. Journal of Educational Measurement, 48, 31-43.

Answer 5

Glymour et al. (2005) addressed using baseline adjustment when analyzing a change score. If change in health status preceded baseline assessment or there is large measurement error in the dependent variable, they find that a bias can arise if the regression model using change score as the dependent variable includes a baseline covariate. Frank Harrell's answer "The only problem would be if measurement errors in the baseline covariate are confounded with another X, distorting that other X's effect." may be reflecting the same bias as Glymour addresses.

Glymour (2005) "When is Baseline Adjustment Useful in the Analysis of Change? An Example with Education and Cognitive Change. American Journal of Epidemiology 162:267-278

Answer 6

See Josh Angrist on exactly this question: http://www.mostlyharmlesseconometrics.com/2009/10/adding-lagged-dependent-vars-to-differenced-models/. He comes down largely against including the lagged DV in your model. There is nothing in his response that is not in the responses above, but a further succinct answer to your question may help.

Answer 7

Just an addition to the entire discussion (I did not know where to put a comment, so I put it as an answer): if you plan to do a longitudinal analysis in the pharmaceutical industry, then the adjustment for baseline is mentioned by the major guidelines (FDA and EMA) also in case of the analysis of change (change score). This topic gives more details and references to the industry guidelines: Is there any formal guideline that would indicate the necessity for adjustment for baseline when analysing change from baseline? and also this blog post: http://onbiostatistics.blogspot.com/2019/05/fda-and-ema-guidance-on-adjusting-for.html

Also:

Answer 8

Ocram is not correct. The difference in weights does not take the initial weight into account. Specifically, the intial weight is kind of taken out by subtracting the end weight from it.

Therefore, I would argue that it does not violate any assumptions if you control for the initial weight.

(The same logic applies if you take the difference of the BMI and the initial BMI.)

---

Update
After Andy W's critic let me be more formal on why I am right and Ocram wrong (at least from my point).

There is some absolute level of weight each person has (e.g., around 100 pound as opposed to 200 pounds). Let $a_w$ be this absoulte weight.
Then, the initial weight can be formalized as $i_w = a_w$ and the end weight as $e_w = a_w + \Delta_w$

The dv the OP wants to use is thus $\Delta_w = i_w - e_w = a_w - a_w + \Delta_w = \Delta_w$

In other words, the absolute level of weight (formalized as $a_w$) drops out from the equation representing the dv and, hence, does not contaminate it (which disagrees with Andy W's claim).

If you want to take it into account you need to incorporate it into your model separately (as an ordinary parameter and/or as an interaction term).

Obviosuly this same logic applies to $\Delta_{BMJ}$ and can be easily accommodated to proportions where one would say e.g.: $e_w = a_w * prop_{\Delta w}$

Answer 9

Observe that

$\underbrace{\textrm{end weight} - \textrm{initial weight}}_{Y} = \beta_{0} + \beta^{T}x$

is equivalent to

$\textrm{end weight} = \textrm{initial weight} + \beta_{0} + \beta^{T}x$

In words, using the change in weight (instead of the end weight itself) as DV already accounts for the initial weight.