1
$\begingroup$

enter image description hereI am trying to understand the plot below generated in R (using the function cv.glmnet) which illustrates the cross validation process for picking the value of lambda in lasso regression. What are the numbers on the top of the plot? My understanding is that the log of the lambda vakue corresponding to the minimum MSE and 1 SE from the minimum MSE are shown by the vertical dashed lines.

Thanks for your advice.

$\endgroup$
  • 2
    $\begingroup$ The MSE range is too big. Try to limit log(lamda) max at 11 and see if you are benefiting from this regularization. $\endgroup$ – Cagdas Ozgenc Apr 24 at 14:08
1
$\begingroup$

You're correct, the dashed lines are the log$\lambda$ values corresponding to the $\lambda_{\min}$ (left dashed line) and $\lambda_{1se}$ (right dashed line).

$\lambda_{\min}$ is the value for which the model has the lowest cross-validation mean squared error. This error has a certain variance/standard error (depicted by the grey whiskers to every red point which you cannot see properly in the left part of the plot, but clearly in the right part). $\lambda_{1se}$ is a value that is a bit less prone to overfitting, you get it if you extend the height of the upper whisker of the $\lambda_{\min}$ dot to the right until you reach the last point which is still below that imaginary line (thus, within one standard error of the minimum value).

The numbers on top are the number of non-zero regression coefficients in your model (thus, the number of included features). From left to right along the x-axis, with increasing $\lambda$ you have fewer variables in the model, since the penalty for inclusion of features is weighted more heavily.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.