I have a linear regression problem for my car fleet data, where $y$ is the change in rental price and $X$ is a design matrix with around 30 columns (predictors).

Most of the predictors are continuous real values, while only of them, $x_{22}$ is an 'encoded' categorical feature, which has only 2 values, -1 and 1.

when I do a simple $lm(y = 0 + x_{22})$, I see a R-squre similar to other predictors. However, when I use LASSO to fit it, this feature is dropped -- unless I make the penalty term very small

I cannot make sense of this, because the correlation between this feature and other features is very low. In other words, this feature is very unique. It will be very nice to have it included in the final model.

Can someone explain to me what's the possible rationality for LASSO to drop it? Does this reveal some limitation/disadvantage of LASSO? or based on the result ($x_{22}$ is dropped) there is something I am not awared of yet about my features. One of my hypothesis is something like, although no exact correlation, there is some other form of relationshipt between $x_{22}$ and other predictors.


1 Answer 1


To have a valid analysis of change in rental price would require demonstration that the change is independent of the initial price.

To your lasso question, bootstrap the lasso analysis and examine the volatility of the features selected. In other words, first see if lasso is self-consistent before seeing if lasso is consistent with another method.

  • 2
    $\begingroup$ Thanks Frank. I am not from a statistics background, so could you elaborate a bit: " demonstration that the change is independent of the initial price" what dos this mean? $\endgroup$
    – user152503
    May 5, 2019 at 13:41
  • 1
    $\begingroup$ See the section on general problems with analyzing change at hbiostat.org/doc/bbr.pdf . It is almost always better to predict absolute levels adjusted for baseline level as a (sometimes nonlinear) covariate. $\endgroup$ May 5, 2019 at 18:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.