# AUROC: Regression vs Classification results

I'm trying to evaluate the probability of a rare occurrence. My training data is a binary 1/0 for output and a TFIDF Vector of words for input.

Which seems to lend itself to a regression, I've been using xgboost's XGBRegressor with decent results but something is confusing me.

The feature_importances for XGBRegessor seem to pick ngrams which seem to me not that important.

When I ran the exact same test but instead used XGBClassifier the important phrases it chose made more sense to me.

The results of sklearns roc_auc score indicate similarly, it ranks the classifier as better than the regressor. Which I don't understand.

Can I compare the roc_auc score of a classifier and regressor as apples to apples?

Does it make sense that a classifier could perform better at predicting a probability than a regressor?

• Can the results of the roc_auc can be taken as apples to apples across regression and calssification? I'm leaning towards making the switch it's just I started out confident it should be taken as a regression task because everyone online talks about rare probability events need to be evaluated as a regression task. I guess if XGB is all regression under the hood it makes sense. – Scott Thompson Jan 27 at 23:06