I have been reading Elements of Statistical Learning. And I saw some example R codes written online.

I realise that very often the explanatory variables $X$ (the data of the predictors) are standardised to 'zero mean and unit variance', while the data of the response variable $y$ is scaled to have zero mean only.

Why don't we also standardise $y$ (response variable) to have not only zero mean, but also unit variance (of response variable) ?

  • 2
    $\begingroup$ The predictors may be standardized in order to make coefficients at slope parameters comparable. However, this does not affect the procedure and retaining the original values may be easier to interpret anyway. $\endgroup$ – Germaniawerks Feb 28 '14 at 9:49
  • 1
    $\begingroup$ What is often done differs a lot by (sub-)discipline. I have the impression that in psychology it is common to report standardized coefficients, sometimes called beta coefficients, which in effect standardize both the $x$s and the $y$. This fits with the comment of @Germaniawerks as they typically work with test scores that don't have a natural unit, so nothing is lost by standardizing those variables. On the other hand, in my discipline (sociology) one rarely standardizes the $x$s or the $y$, as the focus is typically less on comparing effects of different variables. $\endgroup$ – Maarten Buis Feb 28 '14 at 10:43
  • 2
    $\begingroup$ @Germaniawerks: In ESL, some of the methods, e.g. LASSO, ridge regression, involve shrinkage, so changing the predictors' scale will affect the model's fit. $\endgroup$ – Scortchi - Reinstate Monica Feb 28 '14 at 10:58

Part of the regression model entails modeling the conditional variance of Y given your predictors x; Var(Y|x). In order to scale the Y variable to unit variance before fitting the model you would have to use the marginal variance Var(Y). This marginal variance is of no importance to the regression model however.

In case you use an error function location-scale from the location scale family such as the Gaussian, and assume homoscedasticity, scaling the response variable does not affect the final model. However, when the variance depends on the mean, such as for count data, scaling the response variable prior to model fitting distorts the mean-variance relationship.

Scaling of the regressors may be useful to eliminate effects of the units used and enable direct comparison of their coefficients, but when using only univariate regression this argument does not hold for the response variable.

In conclusion, scaling the response variable only complicates interpretation, and is outright wrong if the variance of Y is not constant.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.