Sorry if this question comes across a little basic.
I am looking to use LASSO variable selection for a multiple linear regression model in R. I have 15 predictors, one of which is categorical(will that cause a problem?). After setting my $x$ and $y$ I use the following commands:
model = lars(x, y)
coef(model)
My problem is when I use coef(model)
. This returns a matrix with 15 rows, with one extra predictor added each time. However there is no suggestion as to which model to choose. Have I missed something? Is there a way I can get the lars package to return just one "best" model?
There are other posts suggesting using glmnet
instead but this seems more complicated. An attempt is as follows, using the same $x$ and $y$. Have I missed something here?:
cv = cv.glmnet(x, y)
model = glmnet(x, y, type.gaussian="covariance", lambda=cv$lambda.min)
predict(model, type="coefficients")
The final command returns a list of my variables, the majority with a coefficient although some are =0. Is this the correct choice of the "best" model selected by LASSO? If I then fit a linear model with all my variables which had coefficients not=0
I get very similar, but slightly different, coefficient estimates. Is there a reason for this difference? Would it be acceptable to refit the linear model with these variables chosen by LASSO and take that as my final model? Otherwise I cannot see any p-values for significance. Have I missed anything?
Does
type.gaussian="covariance"
ensure that that glmnet
uses multiple linear regression?
Does the automatic normalisation of the variables affect the coefficients at all? Is there any way to include interaction terms in a LASSO procedure?
I am looking to use this procedure more as a demonstration of how LASSO can be used than for any model that will actually be used for any important inference/prediction if that changes anything.
Thank you for taking the time to read this. Any general comments on LASSO/lars/glmnet would also be greatly appreciated.