Look at the following sample code to understand the problem. I am facing this problem in a real dataset but to represent the issue I am creating a random dataframe below.
random_df <- data.frame(replicate(10,sample(0:1,100000,rep=TRUE))) pred <- sample(0:1,100000,prob=c(0.99,0.01),rep=T) random_df <- lapply(random_df, as.numeric) basic_train <-xgboost(data=data.matrix(random_df),label=pred, eta = 0.1, max_depth = 1, nround=25, subsample = 0.5, eval_metric = "auc", objective = "binary:logistic") mean(predict(basic_train,data.matrix(random_df)))
When I run this I get
Clearly the dataset I created has a mean of 1%. Why would the mean of probabilities predicted by this model equal 4%? Am I doing something wrong here or xgboost just doesn't handle class imbalance in binary classification well?
Just for comparison, use GBM
gbm_train = gbm.fit(random_df,pred ,shrinkage=0.1 ,interaction.depth=1 ,n.trees=25 ,distribution='bernoulli') mean(predict(gbm_train,random_df,n.trees=25,type='response'))
And you get the expected result!