# Gradient Boosted Regression - decide number of trees?

By adding arbitrarily many trees, seems like the $R^2$ value can be as close to 1.0 as we want.

This doesn't seem correct. How do we determine the optimal number of trees? Should I use a form of validation?

• Can you please elaborate on what is your current process? e.g., - you are splitting the data set, using cross validation/Grid search modules – Nishad Jul 20 '18 at 23:12
• I am just trying to learn to use them, I don't have much of a process at the moment. – Baron Yugovich Jul 21 '18 at 22:53
• The reason for R-square close to one is most likely that you are using a toy example or you are over fitting. If you have build ML models before then check this link for xgboost parameter tuning -hackerearth.com/practice/machine-learning/… – Nishad Jul 23 '18 at 16:48
• If you are not aware of cross-validation/hyper parameter tuning then look at this - towardsdatascience.com/… – Nishad Jul 23 '18 at 16:51