I'm trying to tune hyperparameters for XGBoost using RandomizedSearchCV. I have five outputs and I guess that might be the reason that I can't run RandomizedSearchCV or GridSearchCV to tune hyperparameters. Running this code:
params = { 'max_depth': [3, 5, 6, 10, 15, 20],
'learning_rate': [0.01, 0.1, 0.2, 0.3],
'subsample': np.arange(0.5, 1.0, 0.1),
'colsample_bytree': np.arange(0.4, 1.0, 0.1),
'colsample_bylevel': np.arange(0.4, 1.0, 0.1),
'n_estimators': [100, 500, 1000]}
xgbr = xgb.XGBRegressor(seed = 20)
clf = RandomizedSearchCV(estimator=xgbr,
param_distributions=params,
scoring='neg_mean_squared_error',
n_iter=25,
verbose=1)
clf.fit(X_train, Y_train)
print("Best parameters:", clf.best_params_)
print("Lowest RMSE: ", (-clf.best_score_)**(1/2.0))
I get this error:
UserWarning: One or more of the test scores are non-finite: [nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan
nan nan nan nan nan nan nan]
warnings.warn(
Is having multiple outputs in Y_train the cause of this issue?