17
$\begingroup$

I am aware it is common practice to standardize the features for ridge and lasso regression, however, would it ever be more practical to normalize the features on a (0,1) scale as an alternative to z-score standardization for these regression methods?

$\endgroup$

1 Answer 1

1
$\begingroup$

Normalization is very important for methods with regularization. This is because the scale of the variables affect the how much regularization will be applies to specific variable.

For example, suppose one variable is in a very large scale, say order of millions and another variable is from 0 to 1. Then, we can think the regularization will have little effect on first variable.

As well as we do normalization, normalize it to 0 to 1 or standardize the features does not matter too much.

$\endgroup$
4
  • 3
    $\begingroup$ This answer is stating the obvious. By "normalisation" here it is meant squeezing all values in [0,1], it's not just another word for standardisation. The question is about the effects of normalization in [0,1] vs. standardization ~N(0,1) on model coefficients. $\endgroup$
    – Digio
    Jun 27, 2017 at 20:07
  • $\begingroup$ What does it mean to normalize to [0,1]? There are many ways to achieve that. What exactly is your recommendation for penalized regression? $\endgroup$ Jun 28, 2017 at 8:48
  • 2
    $\begingroup$ As the question states to "normalize the features on a (0,1)" scale, though maybe feature rescaling is a better term, is a general technique to produce coefficient estimates that express relative variable importance (similar to RF's purity measure). Yes, there are many ways to achieve this and it is not something specific to penalised regression but this question is about the effect of feature rescaling (not standardisation) on Lasso. $\endgroup$
    – Digio
    Jun 29, 2017 at 7:21
  • 1
    $\begingroup$ what do you mean by "normalize it to 0 to 1 or standardize the features does not matter too much"? In what sense does it not matter too much? Could you provide any intuition or references for this claim? $\endgroup$
    – user795305
    Oct 12, 2017 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.