1
$\begingroup$

The one standard error rule for selecting the hyperparameter value after a cross-validation search for the LASSO or ridge regression's $\lambda$ is widely known and used.

Is there an analog for this rule of thumb when there are multiple hyperparameters in the model where one used bayesian optimization/random/grid/ search?

For example, suppose I have two hyperparameters $\lambda_1$ and $\lambda_2$ for which I've done a cross-validation gridsearch. Some of these error estimates are within one standard error of the one with lowest error. How could I go about choosing the most regularized hyperparameter values that are within this one standard error range, if a slightly more regularized model is desired?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.