First question here...thank you in advance for your time.
For my thesis I am comparing the performance of different (regressor) algorithms. Since I needed to have some information on Bias and Variance I am using bootstrap repeats to get a Mean Squared Error on the predictions (using 100 repeats).
Now when I run a (e.g.) RandomForestRegressor using this method of bootstrap repeats with the "Warm Start" parameter set to True it will result in a much better performance compared to setting the warm start to False.
Now, if I do the exact same experiment using Cross Validation (sklearn K-fold cross_val_score) the "warm start" is not taken into account so it seems and performance is a lot worse.
How do I handle this? Why is there a difference between both methods wrt performance? Can I use the method with the best performance or doesn't this make sense?
Thank you for your answers.