# XGBoost in R - predictions have severe bias with class imbalanced data

Look at the following sample code to understand the problem. I am facing this problem in a real dataset but to represent the issue I am creating a random dataframe below.

random_df <- data.frame(replicate(10,sample(0:1,100000,rep=TRUE)))
pred <- sample(0:1,100000,prob=c(0.99,0.01),rep=T)
random_df[] <- lapply(random_df, as.numeric)

basic_train <-xgboost(data=data.matrix(random_df),label=pred,
eta = 0.1,
max_depth = 1,
nround=25,
subsample = 0.5,
eval_metric = "auc",
objective = "binary:logistic")

mean(predict(basic_train,data.matrix(random_df)))


When I run this I get

[1] 0.04864751


Clearly the dataset I created has a mean of 1%. Why would the mean of probabilities predicted by this model equal 4%? Am I doing something wrong here or xgboost just doesn't handle class imbalance in binary classification well?

Just for comparison, use GBM

gbm_train = gbm.fit(random_df,pred

,shrinkage=0.1
,interaction.depth=1
,n.trees=25
,distribution='bernoulli')

mean(predict(gbm_train,random_df,n.trees=25,type='response'))


And you get the expected result!

[1] 0.01008938

• Does the phenomenon persist if you add more trees or increase depth? Also, try setting subsample=1 (default), or try setting subsample=0.5 for the gbm to make the comparison fairer. – Alex R. Feb 20 '18 at 21:37