LASSO - normalization of response variable needed? I wonder whether the response variable needs to be normalized before LASSO estimation (I am using the lars package in R to perform LASSO estimation). My guess is that only right-hand side variables need to be normalized, is this correct?
For the sake of interpretation, I would prefer to only normalize right-hand side variables and leave out the response variable. If this is fine, I guess I need to normalize the data myself as the argument 'normalize' in the lars package performs normalization on all variables including the response variable - at least I believe so.
Thanks for help!
 A: You don't need to normalize the response variable, as that just scales the error term that the procedure minimizes.
It doesn't seem that lars normalizes the response variable, anyway. Examine the code by loading the package and typing "lars" (without quotes or parentheses) at the R prompt. As I read the code, if "normalize=TRUE" then only the x (predictor/"right-hand") variables are normalized. The y (response) variable is untouched. I suspect that the statement in the documentation that "each variable is standardized to have unit L2 norm" was only meant to apply to the x values.
A: You don't 'need' to do anything, but it is highly recommended to normalize the variables.  This is because lasso is a technique that penalizes large coefficients.  If the variables are on different scales, the technique will tend to just penalize variables on the smallest scale as they will have the largest coefficient. For example if y~x and z, and w are noise then lasso will just give the (correct) model y~x. However if x1 = 10^{-100}x then the right model for predicting y in terms of x1,z,w would be y~(10^{100})x1.  However if the coefficients are constrained to be less than that in absolute value, lasso can't give the correct model. The more that the coefficients are constrained, the worse the model will be. You can imagine that if one is predicting y on two (equally important) variables, the same sort of issue will occur. 
