I have a linear regression problem for my car fleet data, where $y$ is the change in rental price and $X$ is a design matrix with around 30 columns (predictors).
Most of the predictors are continuous real values, while only of them, $x_{22}$ is an 'encoded' categorical feature, which has only 2 values, -1 and 1.
when I do a simple $lm(y = 0 + x_{22})$, I see a R-squre similar to other predictors. However, when I use LASSO to fit it, this feature is dropped -- unless I make the penalty term very small
I cannot make sense of this, because the correlation between this feature and other features is very low. In other words, this feature is very unique. It will be very nice to have it included in the final model.
Can someone explain to me what's the possible rationality for LASSO to drop it? Does this reveal some limitation/disadvantage of LASSO? or based on the result ($x_{22}$ is dropped) there is something I am not awared of yet about my features. One of my hypothesis is something like, although no exact correlation, there is some other form of relationshipt between $x_{22}$ and other predictors.