1
$\begingroup$

Background

I am fitting a dataset of 1500 observations and 375 features (after one hot encoding of categorical features) dealing with prediction of house prices. I am using a gradient boosting model (GBM) using the GradientBoostedRegressor in the sklearn library. I've kept 20% of dataset as holdout set for obtaining a unbiased score of my model. This leaves us with 1200 observations. While tuning the hyperparameters of the GBM using gridsearchcv, I came across two sets of hyperparameter values that gave the following learning and validation curves:

enter image description here

enter image description here

enter image description here

enter image description here

Note that the hyperparameters are written on top of each curve. The validation curve is between the RMSE and the number of trees, while the learning curve is between the RMSE and the number of examples.

Problem

Out of the two sets of hyperparameters I am unable to conclude which set to choose by looking at the graphical information provided. I just need some insight to determine if there is overfitting or underfitting, or if I should select a better hyperparameter combination, by looking at the curves.

Since this analysis is done for a data science competition, from leaderboard submission scores I can tell that the two sets of parameters perform differently as one gives RMSE of .133 and the other .124 (in no specific order).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.