I am fitting a dataset of 1500 observations and 375 features (after one hot encoding of categorical features) dealing with prediction of house prices. I am using a gradient boosting model (GBM) using the
GradientBoostedRegressor in the
sklearn library. I've kept 20% of dataset as holdout set for obtaining a unbiased score of my model. This leaves us with 1200 observations. While tuning the hyperparameters of the GBM using
gridsearchcv, I came across two sets of hyperparameter values that gave the following learning and validation curves:
Note that the hyperparameters are written on top of each curve. The validation curve is between the RMSE and the number of trees, while the learning curve is between the RMSE and the number of examples.
Out of the two sets of hyperparameters I am unable to conclude which set to choose by looking at the graphical information provided. I just need some insight to determine if there is overfitting or underfitting, or if I should select a better hyperparameter combination, by looking at the curves.
Since this analysis is done for a data science competition, from leaderboard submission scores I can tell that the two sets of parameters perform differently as one gives RMSE of .133 and the other .124 (in no specific order).