1
$\begingroup$

I want to know why everytime I run my algorithm (XGBoost regressor) with a different random state (applied to train/test split part) I get different values for R2 and RMSE. For example :

  • Random state = 0 -> RMSE = 4.67, R2 = 0.78
  • Random state = 42 -> RMSE = 8.58, R2 = 0.54
$\endgroup$
1
  • $\begingroup$ The machine learning is based on random splits in trees, right? $\endgroup$
    – Dave
    Nov 15 '21 at 16:17
1
$\begingroup$

Your code is doing what it should. There is a random component to your machine learning modeling, and that random component is different for the different random states, the same as how the following two code blocks will give different random numbers.

np.random.seed(2021)
np.random.normal(0, 1, 10)

vs

np.random.seed(2022)
np.random.normal(0, 1, 10)
$\endgroup$
4
  • $\begingroup$ Watch out for interpreting $R^2$ in your nonlinear model, however. As I discuss here, there is a sense where $R^2$ is totally legitimate, but $R^2$ in nonlinear models does not have the same interpretation as in vanilla OLS linear regression. $\endgroup$
    – Dave
    Nov 15 '21 at 16:22
  • $\begingroup$ I see. Thank you. So how can I fix the results so that they don't change. Is there an alternative to random_state? Is there a better metric than R2 for non linear regression? $\endgroup$
    – Ashraf.R
    Nov 15 '21 at 16:30
  • $\begingroup$ If you want the results to be the same every time, use the same random state (or whatever the correct terminology/code is for your particular implementation). // Nonlinear regression metrics would be for a separate question. $\endgroup$
    – Dave
    Nov 15 '21 at 16:40
  • $\begingroup$ Okay. Thank you for your help $\endgroup$
    – Ashraf.R
    Nov 15 '21 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.