1
$\begingroup$

First question here...thank you in advance for your time.

For my thesis I am comparing the performance of different (regressor) algorithms. Since I needed to have some information on Bias and Variance I am using bootstrap repeats to get a Mean Squared Error on the predictions (using 100 repeats).

Now when I run a (e.g.) RandomForestRegressor using this method of bootstrap repeats with the "Warm Start" parameter set to True it will result in a much better performance compared to setting the warm start to False.

Now, if I do the exact same experiment using Cross Validation (sklearn K-fold cross_val_score) the "warm start" is not taken into account so it seems and performance is a lot worse.

How do I handle this? Why is there a difference between both methods wrt performance? Can I use the method with the best performance or doesn't this make sense?

Thank you for your answers.

Roy

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.