0
$\begingroup$

I trained set of models using RandomizedSearchCV and picked the best using .best_estimator_ and then tested on my test set. However, I would like to check how any other model from the grid performed on the test set. Is it possible to do so and if yes, how can I do that?

$\endgroup$
  • $\begingroup$ Check out "cv_results_" and think about its relation with "best_estimator_" $\endgroup$ – NULL Nov 15 '19 at 13:00
  • $\begingroup$ Yeah, best_estimator_ is the model that achieved the highest mean score throught cv. I was just wondering if there is an easy way to pick for example 2nd best model and check its test score. $\endgroup$ – Łukasz Łaszczuk Nov 15 '19 at 13:35
0
$\begingroup$

RandomizedSearchCV discards the actual models it trains on each fold after evaluating them, so you won't be able to extract the fitted models from the output.

For refit=True, RandomizedSearchCV will pick the parameters that performed best on the validation sets and re-train the model with these parameters, this time on all observations. This is the model that you extract.
That would be inefficient to do for all parameter settings that were not selected, so RandomizedSearchCV does not retrain the second best model.

You could look at _cv_results to find the second best parameter settings and manually train the model with these parameters on all observations. Use method set_params() to set the parameters to the second or third best parameter set in _cv_results_.
I would not recommend this procedure to choose a set of parameters, since it negates the reason to do parameter selection using CV rather than your test set. You are likely overfit your parameter choice to the test data.

| cite | improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.