# which dataset to send as eval set in xgboost, catboost, and etc, when using optuna

In some boost models there are option to send eval set while fitting the model. for example:

xgb_model = XGBRegressor(n_estimators=1000)
xgb_model.fit(train_X, train_y, early_stopping_rounds=5,
eval_set=[(test_X, test_y)], verbose=False)


In addition, when using Optuna, you should write an objective function that returns a metric score. For example:

 def objective(trial):
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
param = {
"loss_function": trial.suggest_categorical("loss_function", ["RMSE", "MAE"]),
"learning_rate": trial.suggest_loguniform("learning_rate", 1e-5, 1e0),
"l2_leaf_reg": trial.suggest_loguniform("l2_leaf_reg", 1e-2, 1e0),

}

reg = CatBoostRegressor(**param, cat_features=categorical_features_indices)
reg.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=0, early_stopping_rounds=100)
y_pred = reg.predict(X_test)
score = r2_score(y_test, y_pred)
return score


When using both a validation set and a test set, which dataset should be used as eval_set, and which dataset should be used for calculating the metric score in the objective?

• It sounds like you're asking the difference between a test and a validate set. We have lots of questions about this. Here's a search to get you started. stats.stackexchange.com/…
– Sycorax
Jun 23 at 14:46