What should the objective be when tuning hyperparameters to minimize overfitting?

Question

I'm working on a classification problem with ~90k data rows and 12 features. I'm trying to tune the hyperparamters of an XGBoost model to minimize the overfitting. I use ROC_AUC as the metric to evaluate the model performance. With the default XGBoost parameters, the 5-fold CV results show train-auc 0.782 and test-auc 0.739 respectively. This indicates overfitting since train set performs better than the test set.

I started tuning the hyperparameters because a), it should be tuned, b) hyperparameters are known to be used to reduced overfitting. However, like many others did, I set the validation-auc as the objective and implemented cross-validation in the objective function. The library used are Optuna and Hyperopt. Interestingly, for both cases, I found the train-test auc gap (indicator of overfitting) widens as the algorithm tries to push the validation-auc towards 0.745. If I plot the train and validation auc in the order of descending validation-auc, the validation-auc decreases from 0.745 to 0.729 while the train-auc drops from 0.868 to 0.742.

I'm quite puzzled by the results. Was I right to set the validation-auc as the objective for Optuna to optimize? Should I have chosen (train-auc - validation-auc) instead, but I haven't found any examples like this online. And should I look at the result and pick the lowest validation-auc and the associated hyperparameter values as the 'best params' since the train-validation metric gap is the lowest?

Please share your thoughts here since as I was typing, I've become unsure about my understanding of overfitting was even correct.

My code:

X, y = df[features], df[target]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2,
                                                random_state = 42, stratify = y)
dtrain = xgb.DMatrix(X_train, label = y_train, enable_categorical = True)
dtest = xgb.DMatrix(X_test, label = y_test, enable_categorical = True)

def objective(trial):

    params = {'max_depth': trial.suggest_int('max_depth', 3, 10),
          'min_child_weight': trial.suggest_int('min_child_weight', 1, 100),
          'gamma': trial.suggest_float('gamma', 0, 2),
          'subsample': trial.suggest_float('subsample', 0.5, 1),
          'colsample_bytree': trial.suggest_float('colsample_bytree', 0.5, 1),
          'reg_alpha': trial.suggest_float('reg_alpha', 1e-8, 10, log = True),
          'reg_lambda': trial.suggest_float('reg_lambda', 1e-8, 10, log = True),
          'learning_rate': trial.suggest_float('learning_rate', 0.001, 0.3),
          'objective': 'binary:logistic'}

    cv_results = xgb.cv(
    params, dtrain, num_boost_round = 10000, early_stopping_rounds = 50, 
        metrics = 'auc', nfold = 5, stratified = True, shuffle = False
 )

    trial.set_user_attr('n_estimators', len(cv_results))
    trial.set_user_attr('train-auc', cv_results['train-auc-mean'].iloc[-1])

    return cv_results['test-auc-mean'].iloc[-1]

study = optuna.create_study(
direction ='maximize', sampler = optuna.samplers.TPESampler(seed = 42))

study.optimize(objective, n_trials = 500, n_jobs = -1)

Answer 1

First, a quick glossary (assisted by ChatGPT):

Train Data: The dataset used to teach the model by adjusting its parameters to minimize error. During cross-validation, different subsets of the data serve as training data in each fold.

Validation Data: A subset of the data used during cross-validation to tune hyperparameters and evaluate the model's performance on unseen data. Each fold in cross-validation uses a different part of the data as validation, helping ensure robust hyperparameter selection.

Test Data: A separate, untouched dataset used to evaluate the model's final performance after cross-validation. This data is not involved in training or hyperparameter tuning, providing an unbiased assessment of the model's generalization.

The short answer is that your goal is never to reduce overfitting. Your goal is to make accurate predictions on new data, and you should only worry about overfitting if it impacts your ability to do that. You should choose the hyperparameters that give you the best performance on your validation set, full stop. You don't need to worry about the difference between train and validation AUC, except that it a large gap due to overfitting would suggest that you could improve validation set performance by reducing the flexibility of the model.

Also, you should never change your algorithm based on how it performs against the test set. The purpose of a test set is to estimate, once you've chosen your final algorithm, how well it would perform on totally new data. Updating your algorithm based on this means it's not new data any more, and you need to go and collect a new test set.

Finally, AUC of 0.782 in the training set and 0.739 in the validation set doesn't even indicate overfitting. You'll always have at least slightly better performance on the data you've trained the model on, and this gap is tiny. Don't worry about it.