If we think back to linear models for a moment, we have Ordinary Least Squares (OLS) versus Generalized Linear Models (GLM). Without going too in-depth, it can be said that GLMs "improve" upon OLS by relaxing some of the assumptions, making it more robust to different types of data. The underlying training algorithm is also somewhat different; OLS minimizes the root mean squared error (RMSE) while GLMs minimize deviance. (I realize that RMSE is a special case of deviance). This allows us to build linear models based on, say, the gamma distribution, inverse gaussian, etc.

My question is: does the same logic hold true for gradient boosted trees? Since we're working with tree based algorithms now, I'd think that it's not subject to the same assumptions/distributional restrictions as linear models. In the XGBoost package, for example, the default objective function for regression is RMSE. You can define a custom objective if you wish, but does it matter? Does it make sense to do so?

In other words, can we possibly improve our predictive power by setting XGBoost to minimize deviance (say, of a gamma distribution) versus RMSE?

  • $\begingroup$ I'm not sure I understand your distinction between RMSE and deviance. Deviance in minimized at the MLE, so the MLE of a GLM gives you minimal deviance. The difference in robustness comes from the setup of a GLM, which allows for variance that depends on expectation (through the link function), as opposed to OLS where the variance is assumed to be the same. To me it sounds like you want to use tree models to estimate expectation and variance, where your tree takes place of the link function. $\endgroup$
    – Alex R.
    Commented Jun 8, 2016 at 19:36
  • $\begingroup$ @Alex R.: Let's say I run an OLS, and then a GLM with a gamma distribution. If there is no link function for the GLM (identity link), are you saying that the two will result in identical estimates? I don't think so, since OLS is equivalent to a GLM with a normal distribution, not a gamma distribution. $\endgroup$
    – AdmiralWen
    Commented Jun 8, 2016 at 21:16
  • 2
    $\begingroup$ With a gamma distribution of course they are not the same. OLS is equivalent to MLE for gaussian errors. The "improvement" on OLS is to use MLE of a different form. $\endgroup$
    – Alex R.
    Commented Jun 8, 2016 at 21:28
  • $\begingroup$ @Alex R.: That's what I was trying to ask - about the usage of different distributions/forms. I guess I wasn't clear in my question - will make the edit. What if we set the XGBoost objective to minimize the deviance function of a gamma distribution, instead of minimize RMSE? Are there conceivable scenarios where you'd want to do that? (Or if not gamma deviance, what other objectives might you minimize for a regression problem?) $\endgroup$
    – AdmiralWen
    Commented Jun 8, 2016 at 21:56
  • $\begingroup$ Gini coefficient perhaps? $\endgroup$
    – Firebug
    Commented Jun 8, 2016 at 22:08

3 Answers 3


does the same logic hold true for gradient boosted trees?

Yes, by any mean. Gradient boosting can be used to minimize any sensible loss function, and it is very effective in doing it.

It is worth saying that generalised linear models are generally picked considering not the loss/utility function (that answers the question: how well is doing my model/how bad are its errors), but the kind of random variable you want to model. Then, for instance, if you have a target variable which is the number of some kind of events registered in some time, it makes sense to use a Poisson model. In case you have a rich complex dataset, Xgboost can model a Poisson response much better than a GLM.

You can define a custom objective if you wish, but does it matter?

Of course it does, but I'd like to point out that trees are perfectly non-linear (there is no constraint on the functional form) and thus a model trained with MSE loss can often do quite well even if judged with quite different score functions, even for classification tasks! However, MSE is always symmetric and when circumstances require weighting one tail more than the other (like for gamma regression, or for binary regression, when close to extremes) MSE is not optimal and does not perform as well as the most fitting loss function.

But what is it?

This depends on your goal. For ordinary regression MSE is such an appreciated choice because it models the conditional mean of the target variable, which is often the objective, it benefits from the conceptual link with gaussian variable and central limit theorem, it is fast, and it is actually quite robust. This of course doesn't mean you have to use it, it's just a good standard, but every problem is different and many many times you don't want to predict the conditional mean. For instance you could need to predict the order of magnitude of some measure, then MSE should be applied on the logarithm of that variable, or you could have a situation where outliers are common and shouldn't affect the predictions more than other residuals, in that case MAE is a better loss. You can't list them all because there are infinitely many!


To answer your question you need to define what is "better". If your goal is to reach a smaller distance to a gamma distributed variable measured in squared distance than you should build your objective function in that way. A very common problem is to find a solution that minimizes Mean Absolute Error. That cost function is different from RMSE and using a different objective function can help in minimizing the MAE.


Well, in the original paper of Gradient Boosting, Friedman proposes some alternatives to RMSE (or L2 loss) for the loss function in the implementation of Boosted Trees.

These are Least Absolute Deviation (equivalent to MAE, as expressed in the previous answers), and M-Regression, which uses instead a Huber Loss, which might for example be better in case of outliers.

In general, you can define gradient boosted trees with any loss you want, as long as it respects some continuity constraints and you are able to compute its derivative.
It is also quite important to notice that you also use a loss function within the tree-building process and not only the boosting one, so changes there also affect the way trees are built, and it would make sense to use the same loss for both steps


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.