Is it valid to use a difference score as an independent variable in a regression analysis

Question

I would like to see if the difference in the number of BPD symptoms from baseline to follow-up two years later can predict psychosocial functioning at the 2 year follow-up. So I wanted to do a linear regression using a difference score (BPD score T1- BPD score T2) as the IV and different measures of functioning at T2 as the DV's (e.g. SOFAS score as global functioning; peer rating scale as interpersnal functioning; and some binary measures such as work or no work at T2; so will need to complete a series of regressions).

Is it valid to use the difference score. I have read a lot about using a difference score as a dependent variable but am unsure if the same information would correspond to using a difference score as a IV?

I guess an additional question would be is it more approriate to use T1 measures of the outcome variables as covariates in the regression equation in order to measure change in that variable also? Or is that a seperate question?

I appreciate any comments as I am quiet unsure of how to proceed and am going around in circles in how best to get some clarity.

Answer 1

Difference scores as independent variables are fine, but they impose a functionally more restrictive form on the equation. Consider;

$y = \beta_{11}(X_2) - \beta_{21}(X_1) + e_1$

Versus the equation;

$y = \beta_{12}(\Delta X) + e_2$

Where $\Delta X = X_2 - X_1$. You can see the second equation is a special case of the first when $\beta_{11} = \beta_{21}$. Only when you have very good reason to believe the more functionally restrictive form is reasonable, should you use the change scores.

You could actually have situations in which $\beta_{11}$ and $\beta_{21}$ have countervaling effects (e.g. $\beta_{11} = -\beta_{21}$, and the change score would appear to be inconsequential when in reality the two individual components contribute to the outcome.

Answer 2

As far as I know, there is no reason you can't use a difference score as an independent variable in a regression. It violates no assumptions.

Your second question is more complex. Your idea of using T1 measures as covariates is often done. People also sometimes use difference scores as a DV (as you probably know from your reading).

There are some problems with pre- post- testing when the variables are measured with error (as all psychological variables inevitably are). If I recall correctly, there are details in Collins and Horn.

Answer 3

Using a difference score as a predictor in multiple regression will usually lead to some loss of model fit, i.e. R-squared will be less than what it could be if you leave both variables in the difference score to be their own predictors with their own slopes. That is, if you have a model like this: $$ y' = a + b_1 d, \text{where}~~~ d = (x_1 - x_2) $$ It is the same as forcing the two slopes to be equal in magnitude but opposite in sign. That is via multiplication, $$ y' = a + b_1 (x_1 -x_2) $$ and $y' = a +b_ 1x_1 - b_1 x_2$.

So, in a sense it is forcing the linear restriction that you use +1 and $-1$ coefficients, or at least equal but opposite in sign slopes.

To maximize the fit of the model, use this approach instead, allowing the slopes for both variables to just be estimated freely (and if they happen to be equal but opposite in sign, then the difference score is okay): $$ y' = a + b 1 x_1 + b_2 x_2 $$ The loss of R-square or model fit depends on how different the freely estimated slopes would be from the linear restriction of the difference score. Running a simulation with different population slopes, we've found that the loss in R-square (predictable variance in the DV) ranges from zero to about .83, so it can be small or drastic.

Bottom line - just use the regular model with the two variables with their own estimated slopes as the last model above. If the best fit results from time1 - time2 (difference score) then it will be estimated as such, and if not, then your model fit will be much better.

References:

Edwards, J. R. (2001). Ten difference score myths. Organizational Research Methods, 4, 264-286. Download

Difference Scores in Linear Regression: Model Fit with Correlated Predictors MICHAEL C. HELFORD, ADRIAN L. THOMAS, MARLAINA M. MONTOYA, LONG H. NGUYEN, AYESHA P. JAMASPI, ASHLEY Y. CHUNG, Roosevelt University; [email protected] A statistical simulation was used to estimate the loss of model fit in linear regression when using difference scores with correlated predictors compared to non-difference score models. Differences in model fit ranged from 0 to 0.84 across 9 simulated populations.