Skip to main content
added 54 characters in body
Source Link
kjetil b halvorsen
  • 82.8k
  • 32
  • 201
  • 663

Using a difference score as a predictor in multiple regression will usually lead to some loss of model fit, i.e. R-squared will be less than what it could be if you leave both variables in the difference score to be their own predictors with their own slopes. That is, if you have a model like this:

y' = a + b1d, where d = (x1 - x2)

It $$ y' = a + b_1 d, \text{where}~~~ d = (x_1 - x_2) $$ It is the same as forcing the two slopes to be equal in magnitude but opposite in sign. That is via multiplication,

y' = a + b1(x1 -x2)

and y' = a +b1x1 $$ y' = a + b_1 (x_1 -x_2) $$ and -b1x2$y' = a +b_ 1x_1 - b_1 x_2$.

So, in a sense it is forcing the linear restriction that you use +1 and -1$-1$ coefficients, or at least equal but opposite in sign slopes.

To maximize the fit of the model, use this approach instead, allowing the slopes for both variables to just be estimated freely (and if they happen to be equal but opposite in sign, then the difference score is okay):

y' = a + b1x1 + b2x2

The $$ y' = a + b 1 x_1 + b_2 x_2 $$ The loss of R-square or model fit depends on how different the freely estimated slopes would be from the linear restriction of the difference score. Running a simulation with different population slopes, we've found that the loss in R-square (predictable variance in the DV) ranges from zero to about .83, so it can be small or drastic.

Bottom line - just use the regular model with the two variables with their own estimated slopes as the last model above. If the best fit results from time1 - time2 (difference score) then it will be estimated as such, and if not, then your model fit will be much better.
References

References:

Edwards, J. R. (2001). Ten difference score myths. Organizational Research Methods, 4, 264-286. Download

Difference Scores in Linear Regression: Model Fit with Correlated Predictors MICHAEL C. HELFORD, ADRIAN L. THOMAS, MARLAINA M. MONTOYA, LONG H. NGUYEN, AYESHA P. JAMASPI, ASHLEY Y. CHUNG, Roosevelt University; [email protected] A statistical simulation was used to estimate the loss of model fit in linear regression when using difference scores with correlated predictors compared to non-difference score models. Differences in model fit ranged from 0 to 0.84 across 9 simulated populations.

Using a difference score as a predictor in multiple regression will usually lead to some loss of model fit, i.e. R-squared will be less than what it could be if you leave both variables in the difference score to be their own predictors with their own slopes. That is, if you have a model like this:

y' = a + b1d, where d = (x1 - x2)

It is the same as forcing the two slopes to be equal in magnitude but opposite in sign. That is via multiplication,

y' = a + b1(x1 -x2)

and y' = a +b1x1 -b1x2

So, in a sense it is forcing the linear restriction that you use +1 and -1 coefficients, or at least equal but opposite in sign slopes.

To maximize the fit of the model, use this approach instead, allowing the slopes for both variables to just be estimated freely (and if they happen to be equal but opposite in sign, then the difference score is okay):

y' = a + b1x1 + b2x2

The loss of R-square or model fit depends on how different the freely estimated slopes would be from the linear restriction of the difference score. Running a simulation with different population slopes, we've found that the loss in R-square (predictable variance in the DV) ranges from zero to about .83, so it can be small or drastic.

Bottom line - just use the regular model with the two variables with their own estimated slopes as the last model above. If the best fit results from time1 - time2 (difference score) then it will be estimated as such, and if not, then your model fit will be much better.
References:

Edwards, J. R. (2001). Ten difference score myths. Organizational Research Methods, 4, 264-286. Download

Difference Scores in Linear Regression: Model Fit with Correlated Predictors MICHAEL C. HELFORD, ADRIAN L. THOMAS, MARLAINA M. MONTOYA, LONG H. NGUYEN, AYESHA P. JAMASPI, ASHLEY Y. CHUNG, Roosevelt University; [email protected] A statistical simulation was used to estimate the loss of model fit in linear regression when using difference scores with correlated predictors compared to non-difference score models. Differences in model fit ranged from 0 to 0.84 across 9 simulated populations.

Using a difference score as a predictor in multiple regression will usually lead to some loss of model fit, i.e. R-squared will be less than what it could be if you leave both variables in the difference score to be their own predictors with their own slopes. That is, if you have a model like this: $$ y' = a + b_1 d, \text{where}~~~ d = (x_1 - x_2) $$ It is the same as forcing the two slopes to be equal in magnitude but opposite in sign. That is via multiplication, $$ y' = a + b_1 (x_1 -x_2) $$ and $y' = a +b_ 1x_1 - b_1 x_2$.

So, in a sense it is forcing the linear restriction that you use +1 and $-1$ coefficients, or at least equal but opposite in sign slopes.

To maximize the fit of the model, use this approach instead, allowing the slopes for both variables to just be estimated freely (and if they happen to be equal but opposite in sign, then the difference score is okay): $$ y' = a + b 1 x_1 + b_2 x_2 $$ The loss of R-square or model fit depends on how different the freely estimated slopes would be from the linear restriction of the difference score. Running a simulation with different population slopes, we've found that the loss in R-square (predictable variance in the DV) ranges from zero to about .83, so it can be small or drastic.

Bottom line - just use the regular model with the two variables with their own estimated slopes as the last model above. If the best fit results from time1 - time2 (difference score) then it will be estimated as such, and if not, then your model fit will be much better.

References:

Edwards, J. R. (2001). Ten difference score myths. Organizational Research Methods, 4, 264-286. Download

Difference Scores in Linear Regression: Model Fit with Correlated Predictors MICHAEL C. HELFORD, ADRIAN L. THOMAS, MARLAINA M. MONTOYA, LONG H. NGUYEN, AYESHA P. JAMASPI, ASHLEY Y. CHUNG, Roosevelt University; [email protected] A statistical simulation was used to estimate the loss of model fit in linear regression when using difference scores with correlated predictors compared to non-difference score models. Differences in model fit ranged from 0 to 0.84 across 9 simulated populations.

Source Link

Using a difference score as a predictor in multiple regression will usually lead to some loss of model fit, i.e. R-squared will be less than what it could be if you leave both variables in the difference score to be their own predictors with their own slopes. That is, if you have a model like this:

y' = a + b1d, where d = (x1 - x2)

It is the same as forcing the two slopes to be equal in magnitude but opposite in sign. That is via multiplication,

y' = a + b1(x1 -x2)

and y' = a +b1x1 -b1x2

So, in a sense it is forcing the linear restriction that you use +1 and -1 coefficients, or at least equal but opposite in sign slopes.

To maximize the fit of the model, use this approach instead, allowing the slopes for both variables to just be estimated freely (and if they happen to be equal but opposite in sign, then the difference score is okay):

y' = a + b1x1 + b2x2

The loss of R-square or model fit depends on how different the freely estimated slopes would be from the linear restriction of the difference score. Running a simulation with different population slopes, we've found that the loss in R-square (predictable variance in the DV) ranges from zero to about .83, so it can be small or drastic.

Bottom line - just use the regular model with the two variables with their own estimated slopes as the last model above. If the best fit results from time1 - time2 (difference score) then it will be estimated as such, and if not, then your model fit will be much better.
References:

Edwards, J. R. (2001). Ten difference score myths. Organizational Research Methods, 4, 264-286. Download

Difference Scores in Linear Regression: Model Fit with Correlated Predictors MICHAEL C. HELFORD, ADRIAN L. THOMAS, MARLAINA M. MONTOYA, LONG H. NGUYEN, AYESHA P. JAMASPI, ASHLEY Y. CHUNG, Roosevelt University; [email protected] A statistical simulation was used to estimate the loss of model fit in linear regression when using difference scores with correlated predictors compared to non-difference score models. Differences in model fit ranged from 0 to 0.84 across 9 simulated populations.