37
$\begingroup$

I have 2 simple questions about linear regression:

  1. When is it advised to standardize the explanatory variables?
  2. Once estimation is carried out with standardized values, how can one predict with new values (how one should standardize the new values)?

Some references would be helpful.

$\endgroup$
26
$\begingroup$

Although terminology is a contentious topic, I prefer to call "explanatory" variables, "predictor" variables.

When to standardise the predictors:

  • A lot of software for performing multiple linear regression will provide standardised coefficients which are equivalent to unstandardised coefficients where you manually standardise predictors and the response variable (of course, it sounds like you are talking about only standardising predictors).
  • My opinion is that standardisation is a useful tool for making regression equations more meaningful. This is particularly true in cases where the metric of the variable lacks meaning to the person interpreting the regression equation (e.g., a psychological scale on an arbitrary metric). It can also be used to facilitate comparability of the relative importance of predictor variables (although other more sophisticated approaches exist for assessing relative importance; see my post for a discussion). In cases where the metric does have meaning to the person interpreting the regression equation, unstandardised coefficients are often more informative.
  • I also think that relying on standardised variables may take attention away from the fact that we have not thought about how to make the metric of a variable more meaningful to the reader.

  • Andrew Gelman has a fair bit to say on the topic. See his page on standardisation for example and Gelman (2008, Stats Med, FREE PDF) in particular.

Prediction based on standarisation:

  • I would not use standardised regression coefficients for prediction.
  • You can always convert standardised coefficients to unstandardised coefficients if you know the mean and standard deviation of the predictor variable in the original sample.
$\endgroup$
  • 3
    $\begingroup$ +1, but why would you not use unstandardised regression coefficients for prediction? $\endgroup$ – onestop Feb 12 '11 at 9:43
  • 1
    $\begingroup$ (+1) About assessing variable importance, I think the relaimpo R package does a good job (but see Getting Started with a Modern Approach to Regression). There was also a nice paper by David V. Budescu on dominance analysis (freely available on request). $\endgroup$ – chl Feb 12 '11 at 10:09
  • $\begingroup$ @onestep oops. typo. It's changed now. $\endgroup$ – Jeromy Anglim Feb 12 '11 at 11:13
  • 1
    $\begingroup$ @Jeromy, Could you elaborate on why you wouldn't use standardized regression coefficients for prediction? $\endgroup$ – Michael Bishop Dec 1 '11 at 20:28
  • 3
    $\begingroup$ @MichaelBishop I'm thinking of contexts where you take your regression model and apply it to predict out of sample data. In general, you'd want unstandardised predictions. Also, means and standard deviations can change across samples; using unstandardised predictors should thus give more meaningful results. $\endgroup$ – Jeromy Anglim Dec 2 '11 at 4:06
-4
$\begingroup$

Let me reply with a short answer dough it might overlap with the excelent answer written before.

  1. Always standarize, that allows you to interpret the regression, specially the coefficients of the regression better.

  2. For the new data which is not standarize, I recomend you to store the values you used for each variable to be standarized, such as the maximum and the minimum, and then do the same transformation you did in the hole dataset before but just for this single instance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.