I am attempting to run an OLS regression:

  • DV: Change in weight over a year (initial weight - end weight)

  • IV: Whether or not you exercise.

However, it seems reasonable that heavier people will lose more weight per unit of exercise than thinner people. Thus, I wanted to include a control variable:

  • CV: Initial starting weight.

However, now initial weight is used BOTH to calculate the dependent variable AND as a control variable.

Is this okay? Does this violate an assumption of OLS?

  • 4
    Was the treatment randomly assigned? – Andy W Sep 19 '11 at 2:48
  • 1
    Note that another very similar was recently asked as well, stats.stackexchange.com/q/15104/1036. The answer to that question is applicable to this question (in fact, I would say they are duplicate questions). – Andy W Sep 19 '11 at 12:31
  • 3
    @Andy Actually, the two questions are different enough that I would give a different answer to this one than I gave to the other. Charlie has already given a nice analysis here. – whuber Sep 19 '11 at 19:09
  • 3
    Note that using difference scores is typically associated with a substantial reduction in reliability, although this is somewhat debated – Behacad Jul 29 '13 at 0:49

To answer your literal question, "Is it valid to include a baseline measure as control variable when testing the effect of an independent variable on change scores?", the answer is no. The answer is no, because by construction the baseline score is correlated with the error term when the change score is used as the dependent variable, hence the estimated effect of the baseline on the change score is uninterpretable.

Using

  • $Y_1$ as the initial weight
  • $Y_2$ as the end weight
  • $\Delta{Y}$ as the change in weight (i.e. $\Delta{Y} = Y_2 - Y_1$)
  • $T$ as a randomly assigned treatment, and
  • $X$ as other exogenous factors that affect weight (e.g. other control variables that are related to the outcome but should be uncorrelated with treatment due to random assignment)

One then has a model regressing $\Delta{Y}$ on $T$ and $X$;

$$\Delta{Y} = \beta_1T + \beta_2X + e$$

Which by definition is equivalent to;

$$Y_2 - Y_1 = \beta_1T + \beta_2X + e$$

Now, if you include the baseline as a covariate, one should see a problem, in that you have the $Y_1$ term on both sides of the equation. This shows that $\beta_3Y_1$ is uninterpretable, because it is inherently correlated with the error term.

$$\begin{align*}Y_2 - Y_1 &= \beta_1T + \beta_2X + \beta_3Y_1 + e \\ Y_2 &= \beta_1T + \beta_2X + \beta_3Y_1 + (e + Y_1) \end{align*}$$

Now, part of the confusion in the various answers seems to stem from the fact that different models will yield identical results for the treatment effect, $\beta_1T$ in my above formulation. So, if one were to compare the treatment effect for the model using change scores as the dependent variable to the model using the "levels" (with each model including the baseline $Y_1$ as a covariate), ones interpretation of the treatment effect would be the same. In the two models that follow $\beta_1T$ will be the same, and so will the inferences based on them (Bruce Weaver has some SPSS code posted demonstrating the equivalence as well).

$$\begin{align*} Change\ Score\ Model&: Y_2 - Y_1 = \beta_1T + \beta_2X + \beta_3Y_1 + e \\ Levels\ Model&: Y_2 = \beta_1T + \beta_2X + \beta_3Y_1 + e \end{align*}$$

So some will argue (as Felix has in this thread, and as Bruce Weaver has done on some discussions over on the SPSS google group) that since the models result in the same estimated treatment effect, it does not matter which one you choose. I disagree, because the baseline covariate in the change score model can not be interpreted, you should never include the baseline as a covariate (regardless of whether the estimated treatment effect is the same or not). So this brings up another question, what is the point in using the change scores as dependent variables? As Felix already noted as well, the model using the change score as the dependent variable excluding the baseline as a covariate is different than the model using the levels. To clarify, the subsequent models will give different treatment effects (especially in the case that the treatment is correlated with baseline);

$$\begin{align*} Change\ Score\ Model\ Without\ Baseline&: Y_2 - Y_1 = \beta_1T + \beta_2X + e \\ Levels\ Model&: Y_2 = \beta_1T + \beta_2X + \beta_3Y_1 + e \end{align*}$$

This has been noted in prior literature as "Lord's Paradox". So which model is right? Well, in the case of randomized experiments, I would say the Levels model is preferable (although if you did a good job randomizing, the average treatment effect should be very close between the models). Other's have noted reasons why the levels model is preferable, Charlie's answer makes a good point in that you can estimate interaction effects with the baseline in the levels model (but you can't in the change score model). Whuber in this response to a very similar question demonstrates how the change scores induce correlations between different treatments.

In situations in which the treatment is not randomly assigned, the model using change scores as the dependent variable should be given more consideration. The main benefit of the change score model, is that any time invariant predictors of the outcome are controlled for. So say in the above formulation, $X$ is constant throughout time (for example say a genetic predisposition to be at a certain weight), and that $X$ is correlated with whether an individual chooses to exercise (and $X$ is unobserved). In that instance, the change score model is preferable. Also in instances in which selection into treatment is correlated with the baseline value, the change score model may be preferable. Paul Allison in his paper, Change Scores as Dependent Variables in Regression Analysis, gives these same examples (and largely influenced my perspective on the topic, so I highly suggest to read it).

This isn't to say that change scores are always preferable in non-randomized settings. In the case that you expect the baseline to have an actual causal effect on the post weight, you should use the levels model. In the case that you expect the baseline to have a causal effect, and the selection into treatment is correlated with the baseline, the treatment effect is confounded with the baseline effect.

I've ignored the note by Charlie that the logarithm of the weight could be used as the dependent variable. While I don't doubt that could be a possibility, it is somewhat non sequitur to the initial question. Another question has discussed when it is appropriate to use the logarithms of the variable (and those still apply in this case). There is probably prior literature on the subject that would help guide you as to whether using the logged weight is appropriate as well.


Citation

Allison, Paul D. 1990. Change scores as dependent variables in regression analysis. Sociological Methodology 20: 93-114. Public PDF version.

  • 3
    In the equation $Y_2 = \beta_1T + \beta_2X + \beta_3Y_1 + (e + Y_1)$ if, as is standard practice, we assume all the covariates are not random variables, then $Y_1$ is not correlated with $e + Y_1$. Thus I think there is only a problem if you view $Y_1$ as random, in which case (again just my opinion) you should model $(Y_1,Y_2)$ jointly but without $Y_1$ as a covariate. In this respect without missing data I have been informed that this approach is equivalent to $Y_1$ being a fixed covariate (I will try and find some references for this). – dandar Jan 12 '15 at 9:30
  • 1
    @dandar, that statement does not make sense to me. Note that $Y_1$ is the pre-treatment value of the outcome, it is not the variable being manipulated in an experiment. Are you saying if I have the baseline value of $Y_1$, then I conduct an experiment, and then measure $Y_2$, I should model both $Y_1$ and $Y_2$ as a function of the experimental intervention? – Andy W Jan 12 '15 at 12:38
  • 1
    The model I am talking about does indeed imply $Y_{1}$ is a function of treatment, but only from the viewpoint that despite randomization there will always be slight differences between the treatment and control group with respect to their baseline means. Thus $\beta_{1}$ will capture this difference as well as the effect of treatment. The reference for this is ("Longitudinal Data Analysis of Continuous and Discrete Responses for Pre-Post Designs" by Zeger and Liang, 2000). – dandar Jan 12 '15 at 15:03
  • 1
    A clear discussion of this paper can be found in (“Should baseline be a covariate or dependent variable in analyses of change from baseline in clinical trials?” by Liu, Mogg, Mallick and Mehrotra 2009). They refer to this model as an unconditional model (i.e. it does not condition on baseline response). In the Liu (2009) paper they discuss the main results of the Zeger (2000) paper. These are firstly that with no missing data the point estimates of $B_{1}$ from the unconditional model are the same as those from the conditional approach of ANCOVA using the post-baseline – dandar Jan 12 '15 at 15:04
  • 1
    measurement as a response, and conditioning on a fixed baseline value, and secondly that the point estimate variance from the ANCOVA model is always greater than or equal to that from the unconditional one. It turns out this variance difference will typically be small due to randomization ensuring baseline mean responses between the groups are small. The authors conclude the unconditional model is appropriate for modelling baseline as a random variable, but ANCOVA as appropriate when viewing it as fixed. – dandar Jan 12 '15 at 15:05

Andy's answer seems to be the economist's view of things. It is accepted practice in clinical trials to almost always adjust for the baseline version of the response variable, to greatly increase power. Since we condition on the baseline variables there is no 'error term' for them to be confused with the overall error term. The only problem would be if measurement errors in the baseline covariate are confounded with another X, distorting that other X's effect. The overall preferred method is to adjust for baseline and to model the response variable, not computing the change. One reason for this is that change is heavily dependent on getting the transformation of Y correct, and that change does not apply to regression models in general. E.g. if Y is ordinal, the difference between two ordinal variables is no longer ordinal. Concerning logging or not logging, that just depends on model and overall residual distribution assumptions.

  • I do not fully understand this answer. What do you mean with "adjust for baseline"? Take the difference, or control for it? – Henrik Sep 20 '11 at 15:44
  • 3
    By 'adjust for baseline' I meant including baseline as a covariate. It is also common to use change scores but you can't use them without also adjusting for baseline as a covariate (hence why bother with change scores?). – Frank Harrell Sep 20 '11 at 19:58
  • 6
    Actually nothing you say here (or in response to Felix's comments) directly conflicts with what I say. Using change scores doesn't 'adjust for baseline', it controls for any time invariant ommitted variables (or if selection into treatment is highly correlated with baseline). If baseline is non-neglible (i.e. it has a direct causal effect on the outcome or it has an interaction with the treatment) change scores don't solve the problem. – Andy W Sep 21 '11 at 12:06
  • 2
    @Frank Harrell Thanks for joining this discussion and clarifying this. (+1) – Henrik Sep 21 '11 at 14:02

We can alter @ocram's reasoning slightly to have $$\begin{align*} \text{E}[w_1 - w_0 \mid X, w_0] &= \beta_0 + x \beta + w_0 \gamma \\ \text{E}[w_1 \mid X, w_0] &= \beta_0 + x \beta + w_0 (\gamma + 1) \end{align*} $$

So, if this is the right model, saying that the difference depends upon the weight implies that the end value depends upon the initial value with a coefficient that could be anything. Running a regression of the difference on $x$ and $w_0$ or the end weight on the same variables should give you the same coefficients on everything but $w_0$. But, if this model isn't exactly correct, these regressions will give different results on the other coefficients as well.

Note that this set up implies that the starting weight predicts the difference in weights, not the impact of treatment. This would require an interaction term, perhaps $$\begin{align*} \text{E}[w_1 - w_0 \mid X, w_0] &= \beta_0 + (x * w_0) \beta + w_0 \gamma. \end{align*} $$

Another approach would be to calculate $$\begin{align*} \log (w_1) - \log (w_0) \approx r; \end{align*}$$ here, $r$ is the growth rate of weight. This could be your outcome. Your coefficients on $x$ would be telling you how these predictors are related to proportion changes in weight. This "controls for" initial weight by saying that, for example, an exercise regime that reduces weight by 10% (a coefficient of 0.1 multiplied by 100%) for someone that weights 130 pounds reduces weight by 13 pounds, while the program reduces the weight of a 200 pound participant by 20 pounds. In this case, you might not need to include the initial weight (or its log) on the right hand side.

An interaction term may still be necessary if you believe that the impact of the program depends upon the starting weight. If you use $w_0$ in the interaction term, then the program would be associated with a $w_0 \beta_1$ change in the growth rate of weight. Every pound heavier that a person was at the start of the program leads to a $\beta_1$ increase in the change in the growth rate (this is the cross-partial derivative of the expected value with respect to both treatment and starting weight).

If you use $\log (w_0)$ in the interaction term, the impact of the program increases by $\beta_1/w_0$ for each additional pound heavier the participant was at the start of the program.

As you can see, the cross-partials on interaction terms can become a bit tricky to interpret, but they may capture an impact that you are interested in.

  • Hi Charlie, I see the advantage of using proportion change, however why do you find the difference in the logged variables as opposed to just dividing w1 over w0. – ChrisStata Sep 18 '11 at 21:54
  • I like the idea of proportional change. The question remains however if the expected interaction is literally proportional or not. If not, you would still need to include the initial weight as a covariate. Or would you be sure that it is of same difficulty to loose 10% of your weight for a 100 or 200 pounds person?? – Henrik Sep 18 '11 at 22:18
  • @ChrisStata, you could do that, too. I'm an economist and we do love our logs (and differencing, too). If you had a time series (i.e., multiple observations) for each person (making a panel data set), I could argue that my way is better, but that's not relevant here. Henrik, you're right; I added a bit about that to my answer. – Charlie Sep 19 '11 at 1:26

EDIT: Andy W's argument convinced me to drop Model C. I added another possibility: Analyzing change with Random Coefficient Models (aka Multilevel Models or Mixed Effect Models

There has been a lot of scientific debate about the use of difference scores. My favorite texts are Rogosa (1982, [1]) and Fitzmaurice, Laird, & Ware (2004, [2])

In general, you have three possibilities of analyzing your data:

  • A) Only take the interindividual difference score (the change score)
  • B) Treat the post measurement as DV and control it for the baseline
  • C) Take the difference score as DV and control it for the baseline (that's the model you suggested).Due to Andy W's arguments, I dropped this alternative
  • D) Using a multilevel / mixed-effect-model approach, where the regression line is modeled for each participant and participant are treated as Level-2 units.

Models A and B can produce very different results if the baseline is correlated with the change score (e.g., heavier people have more weight loss), and/or treatment assignment is correlated with the baseline.

If you want to know more about these issues, see the cited papers, or here and here.

There has also been a recent simulation study [3] which empirically compares the conditions under which A or B are preferable.

For completely balanced designs with no missing values, Model D should be equivalent to Model A. However, it gives you more information about between person variability, it is easily extended to more measurement points, and it has nice properties in the presence of unbalanced data and/ or missing values.

As a bottom line: In your case, I would analyze post-measures controlled for baseline (Model B).

[1] Rogosa, D., Brandt, D., & Zimowski, M. (1982). A growth curve approach to the measurement of change. Psychological Bulletin, 92, 726-748.

[2] Fitzmaurice, G. M., Laird, N. M., & Ware, J. H. (2004). Applied longitudinal analysis. Hoboken, NJ: Wiley.

[3] Petscher, Y., & Schatschneider, C., 2011. A Simulation Study on the Performance of the Simple Difference and Covariance‐Adjusted Scores in Randomized Experimental Designs. Journal of Educational Measurement, 48, 31-43.

  • I've downvoted this answer, and you can see my response to why I believe the change scores with baseline as a covariate should not be done. To sum it up, even though Models B and C in your formulation produce equivalent treatment effects, it does not mean that Model C is preferable. In fact, the baseline effect in Model C is uninterpretable, so I argue it should not be used. – Andy W Sep 19 '11 at 15:41
  • @AndyW: Your argument convinced me; although the most relevant estimate of the treatment effect is the same in both models, Model B should be prefered over Model C. I adjusted my answer accordingly. But what do you say to Laird, N. (1983). Further Comparative Analyses of Pretest-Posttest Research Designs. The American Statistician, 37, 329-330.?, who shows a equivalence of B and C? – Felix S Sep 20 '11 at 9:03
  • I don't think anything I said conflicts with the Laird article. Basically all my rant was that (in Laird's notation) $\bar b$ is uninterpretable, so why report it (the equivalence was not in question). Laird does make other comments about how the baseline covariate effect may be interpreted as a hypothesis of if individual treatment groups do not change (although still is critical of it). Feel free to counter my point with situations in which $\bar b$ is useful (it most certainly isn't useful in the normal ways we interpret regression coefficients). – Andy W Sep 20 '11 at 13:51
  • One point for model D. I'm wondering why not considering only model D. It is the most consistent (baseline value is a random variable and has not be be forced to a dependent variable), it is simple, very flexible (interaction can be added) and delivers also the standard deviation of the population. – giordano Apr 29 at 17:37

See Josh Angrist on exactly this question: http://www.mostlyharmlesseconometrics.com/2009/10/adding-lagged-dependent-vars-to-differenced-models/. He comes down largely against including the lagged DV in your model. There is nothing in his response that is not in the responses above, but a further succinct answer to your question may help.

Glymour et al. (2005) addressed using baseline adjustment when analyzing a change score. If change in health status preceded baseline assessment or there is large measurement error in the dependent variable, they find that a bias can arise if the regression model using change score as the dependent variable includes a baseline covariate. Frank Harrell's answer "The only problem would be if measurement errors in the baseline covariate are confounded with another X, distorting that other X's effect." may be reflecting the same bias as Glymour addresses.

Glymour (2005) "When is Baseline Adjustment Useful in the Analysis of Change? An Example with Education and Cognitive Change. American Journal of Epidemiology 162:267-278

Ocram is not correct. The difference in weights does not take the initial weight into account. Specifically, the intial weight is kind of taken out by subtracting the end weight from it.

Therefore, I would argue that it does not violate any assumptions if you control for the initial weight.

(The same logic applies if you take the difference of the BMI and the initial BMI.)


Update
After Andy W's critic let me be more formal on why I am right and Ocram wrong (at least from my point).

There is some absolute level of weight each person has (e.g., around 100 pound as opposed to 200 pounds). Let $a_w$ be this absoulte weight.
Then, the initial weight can be formalized as $i_w = a_w$ and the end weight as $e_w = a_w + \Delta_w$

The dv the OP wants to use is thus $\Delta_w = i_w - e_w = a_w - a_w + \Delta_w = \Delta_w$

In other words, the absolute level of weight (formalized as $a_w$) drops out from the equation representing the dv and, hence, does not contaminate it (which disagrees with Andy W's claim).

If you want to take it into account you need to incorporate it into your model separately (as an ordinary parameter and/or as an interaction term).

Obviosuly this same logic applies to $\Delta_{BMJ}$ and can be easily accommodated to proportions where one would say e.g.: $e_w = a_w * prop_{\Delta w}$

  • When I said that the difference takes the initial weight into account, this is what I actually meant. Now, specifically, what would you write ? final weight - initial weight = ... ? – ocram Sep 18 '11 at 12:04
  • As I wrote, your argumentation seems false to me. I would argue that in fact the end weight takes the initial weight more into account as it is on the same "scale", whereas the diffeence is "rescaled" (as the end weight, hence some absolute value is subtracted from anoher absoulte value. – Henrik Sep 18 '11 at 22:15
  • (-1) This is not correct. In general, you shouldn't include the same variable on both the right hand side and left hand side of the equation (as it results in the independent variable being correlated with the error term). So if you use differences for the dependent variable, you should not include the baseline as a covariate. – Andy W Sep 19 '11 at 2:44
  • @Andy W: I know that your argument is in principal correct. But my argument is that you kind of partial out the absolute value (by subtracting te end value with the baseline) thereby eliminating this correlation. Hence, adding it as a covariate does not inroduce that kind of spurious error correlation. – Henrik Sep 19 '11 at 6:51
  • @Henrik, see my response to this question, and why I still believe this sentiment is misguided. – Andy W Sep 19 '11 at 15:42

Observe that

$\underbrace{\textrm{end weight} - \textrm{initial weight}}_{Y} = \beta_{0} + \beta^{T}x$

is equivalent to

$\textrm{end weight} = \textrm{initial weight} + \beta_{0} + \beta^{T}x$

In words, using the change in weight (instead of the end weight itself) as DV already accounts for the initial weight.

  • 1
    But I guess there could be an interaction between initial weight and weight loss given training. Let's say an adult of 1,90m height and 70kg body mass and an adult of 1,60m height and 90kg body mass take part in the same training exercises. I'd bet that the latter loses more weight. On a second thought: maybe body mass index is a better CV than just weight. – xmjx Sep 18 '11 at 9:13
  • 1
    @xmjx: If you think that the initial weight will impact the final weight - and you are probably right - then it is a good idea to introduce it as an offset in the model as it is done here... – ocram Sep 18 '11 at 9:51
  • 3
    Not correct in general. If the slope of baseline weight is not 1.0 then analysis of change will not be equivalent to analysis of final weight unless initial weight is in both models and you are using ordinary regression. If baseline weight is in two places the model is actually more difficult to explain, so reasons for persisting with this approach are unclear. – Frank Harrell Sep 20 '11 at 14:53

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.