By adding arbitrarily many trees, seems like the $R^2$ value can be as close to 1.0 as we want.

This doesn't seem correct. How do we determine the optimal number of trees? Should I use a form of validation?

  • $\begingroup$ Can you please elaborate on what is your current process? e.g., - you are splitting the data set, using cross validation/Grid search modules $\endgroup$ – Nishad Jul 20 '18 at 23:12
  • $\begingroup$ I am just trying to learn to use them, I don't have much of a process at the moment. $\endgroup$ – Baron Yugovich Jul 21 '18 at 22:53
  • $\begingroup$ The reason for R-square close to one is most likely that you are using a toy example or you are over fitting. If you have build ML models before then check this link for xgboost parameter tuning -hackerearth.com/practice/machine-learning/… $\endgroup$ – Nishad Jul 23 '18 at 16:48
  • $\begingroup$ If you are not aware of cross-validation/hyper parameter tuning then look at this - towardsdatascience.com/… $\endgroup$ – Nishad Jul 23 '18 at 16:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.