I'm training an XGBoost binary classifier on a difficult problem from structural microbiology. Due to the (expectedly) low performance of the predictor I plotted learning curves for my training and validation data sets:

learning curves

The graph was generated through XGBoost's evals_result() method inside a 5-fold stratified shuffled CV loop (code below). This plot looks to me like the predictor is overfitting, due to increasing gap between the training and validation curves (similar example). I tried to prevent this by adding regularization (via various values for XGBoost's subsample and colsample_bytree parameters), early stopping (using a several different early_stopping_rounds) and reducing the learning rate (eta). These either did not work at all or left me with a bad/incomplete predictor (as measured by training data performance). As soon as the training metrics improve, the validation metrics get worse.

My Question

Is this actually overfitting? If so, why aren't my countermeasures working? If not (e.g. because the model is actually not complex enough, i.e. is underfitting), why does the predictor seem to make progress on the training data? If there isn't enough information to decide, what else could I look at?

Other things to note

  • the data is fairly balanced between true and false cases with around 15k samples overall; possibly that number could be increased
  • the training hasn't converged (the training curve is still decreasing) but that can be addressed by increasing the learning rate and/or number of boosting rounds, both of which I tried and succeeded with, at the expense of an even greater disparity to the validation data
  • the features come from one-hot encoded amino acid sequences and are thus quite large (dimension around 2500). I managed to cut that down using dimensionality reduction without any change in the overall trends.
  • I didn't do a proper three-fold split of training/validation/test data to report the predictor's final performance because I'm still in the early stages of understanding the impact of my model parameters.
  • I also tried sklearn's built-in learning_curve() function based on this example but I found the graphs less enlightening because they're plotted against number of training samples used, as opposed to boosting rounds.
  • since XGBoost didn't allow me to automatically run cross-validated evals_result(), I implemented the CV loop myself so there is the possibility of a bug, although I have tried to take care with the implementation:
model = XGBClassifier(use_label_encoder=False)
cv = StratifiedShuffleSplit(n_splits=5, test_size=0.2, random_state=42)
# "test" means actually validation here
results = {
    'train': [],
    'test': [],
for train_idx, test_idx in cv.split(X, y):
    fold_model = clone(model)
    X_train = X[train_idx]
    y_train = y[train_idx]
    X_test = X[test_idx]
    y_test = y[test_idx]
    evalset = [(X_train, y_train), (X_test, y_test)]
    fold_model.fit(X_train, y_train, eval_metric='logloss', eval_set=evalset)
    fold_results = fold_model.evals_result()

combined_train = np.concatenate(results['train']).reshape((cv.n_splits, -1))
combined_test = np.concatenate(results['test']).reshape((cv.n_splits, -1))
train_scores_mean = np.mean(combined_train, axis=0)
test_scores_mean = np.mean(combined_test, axis=0)

Update: comparison with Random Forest predictor

It was suggested that this problem might be hopeless. Although it is possible that with my current model, I won't adequately solve the question, it isn't clear to me if the data I have now actually supports that argument. At least it doesn't help me to interpret my learning curves.

After Frans Rodenburg's suggestion, I trained an XGBoost Random Forest classifier with default parameters for comparison. The final logloss I get is 0.636 for training and 0.655 for validation, i.e. both rather high but similar.

Additionally, sklearn's learning_curve() function gives me the following graphs for logloss, however plotted against number of training samples.

XGBoost learning curve

Random Forest learning curve

Update 2: regularization results

Here is one example of my regularization results, by reducing eta to 0.01 from its default of 0.3:

regularization example

Although the training and validation (labeled "Test score" in the plot) graphs are much closer, neither converges and both provide terrible performance. For comparison, I can easily achieve a logloss of less than 0.1 for the training data, at which point the curve flattens out. The continuation for additional estimators of the above orange graph is flat. This solution is not a real one; its performance is simply too bad. Similarly, I also tried regularizing via the subsample and colsample_bytree parameters (both down to 0.25) but the graphs don't change their trends at all, while providing worse final performance.

  • 1
    $\begingroup$ It is definitely overfitting. The first thing I would try is to fit a completely different model to get some benchmark result. That will make it easier to see if the problem is with your data (the problem just being hard) or your classifier. For example, random forests work really well out of the box. $\endgroup$ May 31, 2021 at 4:53
  • $\begingroup$ @FransRodenburg thank you for your suggestion. I trained a default Random Forest predictor and didn't get any better results either (see updated question). Would that already suggest the problem is "hopeless"? I think it's quite possible that the current model is too simplistic but I was hoping more complexity could make it less terrible. $\endgroup$ May 31, 2021 at 5:52
  • $\begingroup$ I would start by thinking of a more efficient way to encode the amino acids. Maybe you could use embedding? $\endgroup$ May 31, 2021 at 6:08
  • $\begingroup$ I have experimented with various encoding schemes, none of which outperform simple one-hot or even random encoding, as long as enough information is preserved. Although that surprised me, others have come to similar conclusions, e.g. this paper (although it uses deep learning) $\endgroup$ May 31, 2021 at 6:29

1 Answer 1


I would say from the first diagram, there is a overfitting problem. The logic is simple:

  • For over-fitting, you have training and validation curve apart from each other, one is performing good and one is not

  • For under-fitting, both performance are bad, but two curves are close to each other.

A related discussion can be found in my other answer.

Boosting: why is the learning rate called a regularization parameter?

If so, why aren't my countermeasures working?

Because you are not regularize hard enough. Reduce the eta and number of iterations significantly you will see two curves are closer to each other.

  • $\begingroup$ Thank you for your answer. I believe I have tried to regularize fairly hard but please see my second update for details. Even with regularization, I expect at least a decent performance on the test data. $\endgroup$ May 31, 2021 at 7:37

Not the answer you're looking for? Browse other questions tagged or ask your own question.