I am trying to learn a linear regression model. However, I have some confusion related to the normalization of the data. I have normalized the features/predictors to zero mean and unit variance. Do I need to do the same for the target. If so why?
$\begingroup$
$\endgroup$
2
-
1$\begingroup$ Why did you normalize the features/predictors? $\endgroup$– Peter FlomCommented Apr 20, 2013 at 21:39
-
4$\begingroup$ BTW I think 'standardize' is a better term for that. $\endgroup$– Scortchi - Reinstate Monica ♦Commented Apr 21, 2013 at 0:56
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
Normalizing the target in linear regression doesn't matter. In linear regression, your fit will be of the form $$ \hat{y}_i = a_0 + a \cdot x_i. $$ When you predictors $x_i$ are centered, the constant term $a_0$ will always be the mean of the $y_i$. So if you center the $y_i$ before running a regression, you will just get $a_0 = 0$, but all your other coefficients will remain unchanged.
(That being said, normalizing the predictors---as you are currently doing---is a good idea.)
-
1$\begingroup$ Why's normalizing the predictors a good idea? $\endgroup$ Commented Apr 21, 2013 at 0:56
-
$\begingroup$ @Stefan. Yeah, when I center the predictors, I get the constant term $a_0$ to be the mean of y. But I didn't get how come it becomes the mean. Can you tell me the maths behind it? $\endgroup$ Commented Apr 21, 2013 at 12:40
-
2$\begingroup$ @Scortchi Normalizing the predictors is not necessary, but can make interpreting the coefficients from the regression easier: After normalization, big coefficients correspond to important predictors. Also, without normalization, the coefficients of interaction terms can be seriously misleading. That being said, normalization won't affect the predictions you get from your model, so normalization only matters if you intend to interpret the coefficients in the regression. $\endgroup$ Commented Apr 24, 2013 at 21:35
-
1$\begingroup$ @user34790 The math is worked out at pmean.com/10/LeastSquares.html $\endgroup$ Commented Apr 24, 2013 at 21:35
$\begingroup$
$\endgroup$
I think theoretically it doesn't matter, but numerically it does matter. Take a look at this answer.
