This post is related to another post I've made earlier, in order to not spam it with many additional questions I decided to make another post for the follow up questions that I had.
Say I have two datasets one containing categorical variables only and another containing a mix of categorical and continuous variables. Now I want to apply LASSO and Group LASSO to both datasets using glmnet and grplasso [1], as answered here:
if you estimate such models with regularization, for example ridge, lasso or the elastic net, then you should not leave out any columns. The regularization takes care of the singularities, and more important, the prediction obtained may depend on which columns you leave out. That will not happen when you do not use regularization.
When using LASSO we shouldn't drop any level after one-hot encoding the categorical variables, and the selected levels in the final model will be the most "important" ones. Now for the interpretations, how will the coefficients be interpreted when:
1) Fitting LASSO in glmnet to a dataset consisting of all the levels and setting the standardize argument to FALSE.
2) Fitting LASSO in glmnet to a dataset consisting of all the levels and setting the standardize argument to TRUE`.
3) Fitting LASSO in glmnet to a dataset consisting of all the levels in addition to continuous variables and setting the standardize argument to TRUE.
Regarding Group LASSO this time:
1) Will the interpretation of the coefficients be the same as with the standard logistic or linear regression models?
2) How different will the interpretation be if we standardize vs not standardize in case of using dummy variables only?
3) How will the interpretation be if we standardize in case of using dummy and continuous variables at the same time?
[1] https://cran.r-project.org/web/packages/grplasso/grplasso.pdf
