Skip to main content
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
edited tags
Link
Richard Hardy
  • 69.5k
  • 13
  • 126
  • 278
Source Link
Hanna
  • 145
  • 4

Hyperparameter Tuning for XGBoost multi-output regressor

I'm trying to tune hyperparameters for XGBoost using RandomizedSearchCV. I have five outputs and I guess that might be the reason that I can't run RandomizedSearchCV or GridSearchCV to tune hyperparameters. Running this code:

params = { 'max_depth': [3, 5, 6, 10, 15, 20],
           'learning_rate': [0.01, 0.1, 0.2, 0.3],
           'subsample': np.arange(0.5, 1.0, 0.1),
           'colsample_bytree': np.arange(0.4, 1.0, 0.1),
           'colsample_bylevel': np.arange(0.4, 1.0, 0.1),
           'n_estimators': [100, 500, 1000]}

xgbr = xgb.XGBRegressor(seed = 20)

clf = RandomizedSearchCV(estimator=xgbr,
                         param_distributions=params,
                         scoring='neg_mean_squared_error',
                         n_iter=25,
                         verbose=1)

clf.fit(X_train, Y_train)
print("Best parameters:", clf.best_params_)
print("Lowest RMSE: ", (-clf.best_score_)**(1/2.0))

I get this error:

UserWarning: One or more of the test scores are non-finite: [nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan
 nan nan nan nan nan nan nan]
  warnings.warn(

Is having multiple outputs in Y_train the cause of this issue?