I am trying to learn a linear regression model. However, I have some confusion related to the normalization of the data. I have normalized the features/predictors to zero mean and unit variance. Do I need to do the same for the target. If so why?

  • 1
    $\begingroup$ Why did you normalize the features/predictors? $\endgroup$ – Peter Flom Apr 20 '13 at 21:39
  • 4
    $\begingroup$ BTW I think 'standardize' is a better term for that. $\endgroup$ – Scortchi - Reinstate Monica Apr 21 '13 at 0:56

Normalizing the target in linear regression doesn't matter. In linear regression, your fit will be of the form $$ \hat{y}_i = a_0 + a \cdot x_i. $$ When you predictors $x_i$ are centered, the constant term $a_0$ will always be the mean of the $y_i$. So if you center the $y_i$ before running a regression, you will just get $a_0 = 0$, but all your other coefficients will remain unchanged.

(That being said, normalizing the predictors---as you are currently doing---is a good idea.)

  • 1
    $\begingroup$ Why's normalizing the predictors a good idea? $\endgroup$ – Scortchi - Reinstate Monica Apr 21 '13 at 0:56
  • $\begingroup$ @Stefan. Yeah, when I center the predictors, I get the constant term $a_0$ to be the mean of y. But I didn't get how come it becomes the mean. Can you tell me the maths behind it? $\endgroup$ – user34790 Apr 21 '13 at 12:40
  • 2
    $\begingroup$ @Scortchi Normalizing the predictors is not necessary, but can make interpreting the coefficients from the regression easier: After normalization, big coefficients correspond to important predictors. Also, without normalization, the coefficients of interaction terms can be seriously misleading. That being said, normalization won't affect the predictions you get from your model, so normalization only matters if you intend to interpret the coefficients in the regression. $\endgroup$ – Stefan Wager Apr 24 '13 at 21:35
  • 1
    $\begingroup$ @user34790 The math is worked out at pmean.com/10/LeastSquares.html $\endgroup$ – Stefan Wager Apr 24 '13 at 21:35

I think theoretically it doesn't matter, but numerically it does matter. Take a look at this answer.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.