2
$\begingroup$

I have seen in many books that they will tell you to normalize independent variables in a linear regression before model fitting. My understanding was always one of making sure your betas would be interpretable on the same scale. However, if we normalize by taking the values of $X$, the independent ones, and subtract each from the smallest value of $X$, then divide by the range, wouldn't we be destroying the units of our predictor variables so that we cannot interpret them anymore?

What is the point of normalization when it leads to values that no longer have their units? Thanks.

$\endgroup$
2
  • $\begingroup$ Can you please tlle me the tile of one of the books " to tell you to normalize independent variables in a linear regression before model fitting"? I would like to read. I had thought it is only useful to compare coefficients between different predictors in one model. Thanks. $\endgroup$
    – Deep North
    Commented Oct 3, 2017 at 6:26
  • $\begingroup$ @Knarpie gave some excellent reasons for applying such kind of pre-transformations. But there is no general rule that you have to apply them. $\endgroup$
    – Michael M
    Commented Oct 3, 2017 at 9:54

1 Answer 1

7
$\begingroup$

1) Independent variables are sometimes centered to reduce collinearity between higher order terms, as in the equation $$E(Y_i) = \beta_0 + \beta_1 X_{i} + \beta_2 X_{i}^2$$ with Y the explained variable

2) Independent variables can be normalized by dividing by their standard deviations. The interpretation of the coefficients is then "the expected change in Y when the predictor X is augmented by one standard deviation, all other variables kept equal". This standard deviation is known to the researcher, so he can recover the units. This approach renders quantitative comparison of coefficients of different variables possible, in terms of how they react to changes of the magnitude of their standard deviation.

3) Independent variables can be standardized by their range as you describe it. In this case the interpretation of the coefficients should be made in terms of "the expected change in Y when the predictor X is augmented by one observed range of this variable, all other variables kept equal". Again this range is known to the researcher, together with its units. At its minimum value this variable's contribution to the expected value will be 0, at its maximum equal to the coefficient.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.