The short answer is that your goal is never to reduce overfitting. Your goal is to make accurate predictions on new data, and you should only worry about overfitting if it impacts your ability to do that. You should choose the hyperparameters that give you the best performance on your test set, full stop. You don't need to worry about the difference between train and test AUC, except that it a large gap due to overfitting would suggest that you could improve test set performance by reducing the flexibility of the model.

Also, you should never change your algorithm based on how it performs against the validation set. The purpose of a validation set is to estimate, once you've chosen your final algorithm, how well it would perform on totally new data. Updating your algorithm based on this means it's not new data any more, and you need to go and collect a new validation set.

Finally, AUC of 0.782 in the training set and 0.739 in the test set doesn't even indicate overfitting. You'll always have at least slightly better performance on the data you've trained the model on, and this gap is tiny. Don't worry about it.

First, a quick glossary (assisted by ChatGPT):

Train Data: The dataset used to teach the model by adjusting its parameters to minimize error. During cross-validation, different subsets of the data serve as training data in each fold.

Validation Data: A subset of the data used during cross-validation to tune hyperparameters and evaluate the model's performance on unseen data. Each fold in cross-validation uses a different part of the data as validation, helping ensure robust hyperparameter selection.

Test Data: A separate, untouched dataset used to evaluate the model's final performance after cross-validation. This data is not involved in training or hyperparameter tuning, providing an unbiased assessment of the model's generalization.

The short answer is that your goal is never to reduce overfitting. Your goal is to make accurate predictions on new data, and you should only worry about overfitting if it impacts your ability to do that. You should choose the hyperparameters that give you the best performance on your validation set, full stop. You don't need to worry about the difference between train and validation AUC, except that it a large gap due to overfitting would suggest that you could improve validation set performance by reducing the flexibility of the model.

Also, you should never change your algorithm based on how it performs against the test set. The purpose of a test set is to estimate, once you've chosen your final algorithm, how well it would perform on totally new data. Updating your algorithm based on this means it's not new data any more, and you need to go and collect a new test set.

Finally, AUC of 0.782 in the training set and 0.739 in the validation set doesn't even indicate overfitting. You'll always have at least slightly better performance on the data you've trained the model on, and this gap is tiny. Don't worry about it.