I am using gbm to predict an imbalanced binary outcome, with the intent of obtaining a ranking by class probability estimation that produces a strong class separation on out-of-sample data. (I am combining this class probability with other predictions, including from logistic regression, in an ensemble model.)

According to this gbm vignette (Ridgeway, 2007), under "common user options" for loss functions:

This should be easily dictated by the application. For most classification problems either bernoulli or adaboost will be appropriate, the former being recommended. (p. 5)

There's no explanation provided for favoring bernoulli over adaboost nor any mention of the option for huberized loss function, although this function may have been added at a later date.

Related question, but broader than mine: Choosing between loss functions for binary classification. This answer references Bartlett (2006) which is a challenging read for me.

Although performance is satisfactory under the bernoulli loss function, I am having a hard time understanding the justification for selecting one over another. I'm trying all of them, but are there any theoretical justifications that are at least somewhat intuitive?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.