Look at the following sample code to understand the problem. I am facing this problem in a real dataset but to represent the issue I am creating a random dataframe below.

random_df <- data.frame(replicate(10,sample(0:1,100000,rep=TRUE)))
pred <- sample(0:1,100000,prob=c(0.99,0.01),rep=T)
random_df[] <- lapply(random_df, as.numeric)

basic_train <-xgboost(data=data.matrix(random_df),label=pred,
                  eta = 0.1,
                  max_depth = 1,
                  subsample = 0.5,
                  eval_metric = "auc",
                  objective = "binary:logistic")


When I run this I get

[1] 0.04864751

Clearly the dataset I created has a mean of 1%. Why would the mean of probabilities predicted by this model equal 4%? Am I doing something wrong here or xgboost just doesn't handle class imbalance in binary classification well?

Just for comparison, use GBM

gbm_train = gbm.fit(random_df,pred



And you get the expected result!

[1] 0.01008938
  • $\begingroup$ Does the phenomenon persist if you add more trees or increase depth? Also, try setting subsample=1 (default), or try setting subsample=0.5 for the gbm to make the comparison fairer. $\endgroup$ – Alex R. Feb 20 '18 at 21:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.