Skip to main content
Cross Validated

Return to Question

added 206 characters in body
Source Link
MaxYarmolinsky
MaxYarmolinsky
  • 133
  • 5

I was running some LassoCV and RidgeCV and wanted to know whether it is possible for CV MSE functions of lambda for say Ridge regression can have multiple minima.

e.g. multiple values of lambda such that derivatives $dMSE/d\lambda == 0$

(not saddle points or maxima)

typically R or python will show some graph with a minima of MSE vs $\lambda$ and I am just questioning how robust these graphs are if there are multiple minima, is it going to give you the global minimum etc.

EDIT: after reviewing the sub-satsifactory answers here, you can find out more about this issue here: What causes lasso to be unstable for feature selection?

I was running some LassoCV and RidgeCV and wanted to know whether it is possible for CV MSE functions of lambda for say Ridge regression can have multiple minima.

e.g. multiple values of lambda such that derivatives $dMSE/d\lambda == 0$

(not saddle points or maxima)

typically R or python will show some graph with a minima of MSE vs $\lambda$ and I am just questioning how robust these graphs are if there are multiple minima, is it going to give you the global minimum etc.

I was running some LassoCV and RidgeCV and wanted to know whether it is possible for CV MSE functions of lambda for say Ridge regression can have multiple minima.

e.g. multiple values of lambda such that derivatives $dMSE/d\lambda == 0$

(not saddle points or maxima)

typically R or python will show some graph with a minima of MSE vs $\lambda$ and I am just questioning how robust these graphs are if there are multiple minima, is it going to give you the global minimum etc.

EDIT: after reviewing the sub-satsifactory answers here, you can find out more about this issue here: What causes lasso to be unstable for feature selection?

[Edit removed during grace period]; edited title
Link
MaxYarmolinsky
MaxYarmolinsky
  • 133
  • 5

Can cross validatedvalidation MSE have multiple minima as function of lambda?

Source Link
MaxYarmolinsky
MaxYarmolinsky
  • 133
  • 5

Can cross validated MSE have multiple minima?

I was running some LassoCV and RidgeCV and wanted to know whether it is possible for CV MSE functions of lambda for say Ridge regression can have multiple minima.

e.g. multiple values of lambda such that derivatives $dMSE/d\lambda == 0$

(not saddle points or maxima)

typically R or python will show some graph with a minima of MSE vs $\lambda$ and I am just questioning how robust these graphs are if there are multiple minima, is it going to give you the global minimum etc.