The one standard error rule for selecting the hyperparameter value after a cross-validation search for the LASSO or ridge regression's $\lambda$ is widely known and used.

Is there an analog for this rule of thumb when there are multiple hyperparameters in the model where one used bayesian optimization/random/grid/ search?

For example, suppose I have two hyperparameters $\lambda_1$ and $\lambda_2$ for which I've done a cross-validation gridsearch. Some of these error estimates are within one standard error of the one with lowest error. How could I go about choosing the most regularized hyperparameter values that are within this one standard error range, if a slightly more regularized model is desired?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.