I'm trying to fit a Lasso Regression Model to a dataset but I have been stucked in some issues. I understand that I have to fit the best model using the proper lambda. But every time I run the cv.glmnet function it gives different values for lambda.1se and lambda.min and also different coefficients, i.e, different models with differents MSE and R-Squared values. Which one should I use? Does it make sense to replicate this function 1000 times and use the average lambda or the average coefficients value?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ This seems more like a theoretical question than one about glmnet, so I am voting to keep it open. $\endgroup$– Peter FlomSep 29, 2017 at 12:17
-
$\begingroup$ There is some randomization in the cross-validation process. You can run "set.seed(some number)" before the cv.glmnet call to avoid it. $\endgroup$– Xiao ChenOct 9, 2018 at 7:59
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Unless your training set is small, the best way is to remove a validation set (at least 100 data points, or 10% of the training set if this is larger), and run the model with different parameters. The difference between the fit on the training data and the validation set is the overfitting. Choose parameters that minimise overfitting, not parameters that maximise the fit to the training data.
-
$\begingroup$ My training data is really small. Actually the entire dataset is small (about 40 observations). I'm trying to fit a LASSO regression because it's better for small datasets, but every time I do the cross-validation it returns a different lambda. What if I choose the lambda value that minimises the model MSE? $\endgroup$ Oct 3, 2017 at 15:54
-
$\begingroup$ WIth a dataset that small, the best you can do is LOOCV (leave one out cross validation). $\endgroup$ Oct 22, 2017 at 12:36